(upbeat music) >> The ascendancy of cloud and SAS has shown new light on how organizations think about, pay for, and value hardware. Once sought after skills for practitioners with expertise in hardware troubleshooting, configuring ports, tuning storage arrays, and maximizing server utilization has been superseded by demand for cloud architects, DevOps pros, developers with expertise in microservices, container, application development, and like. Even a company like Dell, the largest hardware company in enterprise tech touts that it has more software engineers than those working in hardware. Begs the question, is hardware going the way of Coball? Well, not likely. Software has to run on something, but the labor needed to deploy, and troubleshoot, and manage hardware infrastructure is shifting. At the same time, we've seen the value flow also shifting in hardware. Once a world dominated by X86 processors value is flowing to alternatives like Nvidia and arm based designs. Moreover, other componentry like NICs, accelerators, and storage controllers are becoming more advanced, integrated, and increasingly important. The question is, does it matter? And if so, why does it matter and to whom? What does it mean to customers, workloads, OEMs, and the broader society? Hello and welcome to this week's Wikibon theCUBE Insights powered by ETR. In this breaking analysis, we've organized a special power panel of industry analysts and experts to address the question, does hardware still matter? Allow me to introduce the panel. Bob O'Donnell is president and chief analyst at TECHnalysis Research. Zeus Kerravala is the founder and principal analyst at ZK Research. David Nicholson is a CTO and tech expert. Keith Townson is CEO and founder of CTO Advisor. And Marc Staimer is the chief dragon slayer at Dragon Slayer Consulting and oftentimes a Wikibon contributor. Guys, welcome to theCUBE. Thanks so much for spending some time here. >> Good to be here. >> Thanks. >> Thanks for having us. >> Okay before we get into it, I just want to bring up some data from ETR. This is a survey that ETR does every quarter. It's a survey of about 1200 to 1500 CIOs and IT buyers and I'm showing a subset of the taxonomy here. This XY axis and the vertical axis is something called net score. That's a measure of spending momentum. It's essentially the percentage of customers that are spending more on a particular area than those spending less. You subtract the lesses from the mores and you get a net score. Anything the horizontal axis is pervasion in the data set. Sometimes they call it market share. It's not like IDC market share. It's just the percentage of activity in the data set as a percentage of the total. That red 40% line, anything over that is considered highly elevated. And for the past, I don't know, eight to 12 quarters, the big four have been AI and machine learning, containers, RPA and cloud and cloud of course is very impressive because not only is it elevated in the vertical access, but you know it's very highly pervasive on the horizontal. So what I've done is highlighted in red that historical hardware sector. The server, the storage, the networking, and even PCs despite the work from home are depressed in relative terms. And of course, data center collocation services. Okay so you're seeing obviously hardware is not... People don't have the spending momentum today that they used to. They've got other priorities, et cetera, but I want to start and go kind of around the horn with each of you, what is the number one trend that each of you sees in hardware and why does it matter? Bob O'Donnell, can you please start us off? >> Sure Dave, so look, I mean, hardware is incredibly important and one comment first I'll make on that slide is let's not forget that hardware, even though it may not be growing, the amount of money spent on hardware continues to be very, very high. It's just a little bit more stable. It's not as subject to big jumps as we see certainly in other software areas. But look, the important thing that's happening in hardware is the diversification of the types of chip architectures we're seeing and how and where they're being deployed, right? You refer to this in your opening. We've moved from a world of x86 CPUs from Intel and AMD to things like obviously GPUs, DPUs. We've got VPU for, you know, computer vision processing. We've got AI-dedicated accelerators, we've got all kinds of other network acceleration tools and AI-powered tools. There's an incredible diversification of these chip architectures and that's been happening for a while but now we're seeing them more widely deployed and it's being done that way because workloads are evolving. The kinds of workloads that we're seeing in some of these software areas require different types of compute engines than traditionally we've had. The other thing is (coughs), excuse me, the power requirements based on where geographically that compute happens is also evolving. This whole notion of the edge, which I'm sure we'll get into a little bit more detail later is driven by the fact that where the compute actually sits closer to in theory the edge and where edge devices are, depending on your definition, changes the power requirements. It changes the kind of connectivity that connects the applications to those edge devices and those applications. So all of those things are being impacted by this growing diversity in chip architectures. And that's a very long-term trend that I think we're going to continue to see play out through this decade and well into the 2030s as well. >> Excellent, great, great points. Thank you, Bob. Zeus up next, please. >> Yeah, and I think the other thing when you look at this chart to remember too is, you know, through the pandemic and the work from home period a lot of companies did put their office modernization projects on hold and you heard that echoed, you know, from really all the network manufacturers anyways. They always had projects underway to upgrade networks. They put 'em on hold. Now that people are starting to come back to the office, they're looking at that now. So we might see some change there, but Bob's right. The size of those market are quite a bit different. I think the other big trend here is the hardware companies, at least in the areas that I look at networking are understanding now that it's a combination of hardware and software and silicon that works together that creates that optimum type of performance and experience, right? So some things are best done in silicon. Some like data forwarding and things like that. Historically when you look at the way network devices were built, you did everything in hardware. You configured in hardware, they did all the data for you, and did all the management. And that's been decoupled now. So more and more of the control element has been placed in software. A lot of the high-performance things, encryption, and as I mentioned, data forwarding, packet analysis, stuff like that is still done in hardware, but not everything is done in hardware. And so it's a combination of the two. I think, for the people that work with the equipment as well, there's been more shift to understanding how to work with software. And this is a mistake I think the industry made for a while is we had everybody convinced they had to become a programmer. It's really more a software power user. Can you pull things out of software? Can you through API calls and things like that. But I think the big frame here is, David, it's a combination of hardware, software working together that really make a difference. And you know how much you invest in hardware versus software kind of depends on the performance requirements you have. And I'll talk about that later but that's really the big shift that's happened here. It's the vendors that figured out how to optimize performance by leveraging the best of all of those. >> Excellent. You guys both brought up some really good themes that we can tap into Dave Nicholson, please. >> Yeah, so just kind of picking up where Bob started off. Not only are we seeing the rise of a variety of CPU designs, but I think increasingly the connectivity that's involved from a hardware perspective, from a kind of a server or service design perspective has become increasingly important. I think we'll get a chance to look at this in more depth a little bit later but when you look at what happens on the motherboard, you know we're not in so much a CPU-centric world anymore. Various application environments have various demands and you can meet them by using a variety of components. And it's extremely significant when you start looking down at the component level. It's really important that you optimize around those components. So I guess my summary would be, I think we are moving out of the CPU-centric hardware model into more of a connectivity-centric model. We can talk more about that later. >> Yeah, great. And thank you, David, and Keith Townsend I really interested in your perspectives on this. I mean, for years you worked in a data center surrounded by hardware. Now that we have the software defined data center, please chime in here. >> Well, you know, I'm going to dig deeper into that software-defined data center nature of what's happening with hardware. Hardware is meeting software infrastructure as code is a thing. What does that code look like? We're still trying to figure out but servicing up these capabilities that the previous analysts have brought up, how do I ensure that I can get the level of services needed for the applications that I need? Whether they're legacy, traditional data center, workloads, AI ML, workloads, workloads at the edge. How do I codify that and consume that as a service? And hardware vendors are figuring this out. HPE, the big push into GreenLake as a service. Dale now with Apex taking what we need, these bare bone components, moving it forward with DDR five, six CXL, et cetera, and surfacing that as cold or as services. This is a very tough problem. As we transition from consuming a hardware-based configuration to this infrastructure as cold paradigm shift. >> Yeah, programmable infrastructure, really attacking that sort of labor discussion that we were having earlier, okay. Last but not least Marc Staimer, please. >> Thanks, Dave. My peers raised really good points. I agree with most of them, but I'm going to disagree with the title of this session, which is, does hardware matter? It absolutely matters. You can't run software on the air. You can't run it in an ephemeral cloud, although there's the technical cloud and that's a different issue. The cloud is kind of changed everything. And from a market perspective in the 40 plus years I've been in this business, I've seen this perception that hardware has to go down in price every year. And part of that was driven by Moore's law. And we're coming to, let's say a lag or an end, depending on who you talk to Moore's law. So we're not doubling our transistors every 18 to 24 months in a chip and as a result of that, there's been a higher emphasis on software. From a market perception, there's no penalty. They don't put the same pressure on software from the market to reduce the cost every year that they do on hardware, which kind of bass ackwards when you think about it. Hardware costs are fixed. Software costs tend to be very low. It's kind of a weird thing that we do in the market. And what's changing is we're now starting to treat hardware like software from an OPEX versus CapEx perspective. So yes, hardware matters. And we'll talk about that more in length. >> You know, I want to follow up on that. And I wonder if you guys have a thought on this, Bob O'Donnell, you and I have talked about this a little bit. Marc, you just pointed out that Moore's laws could have waning. Pat Gelsinger recently at their investor meeting said that he promised that Moore's law is alive and well. And the point I made in breaking analysis was okay, great. You know, Pat said, doubling transistors every 18 to 24 months, let's say that Intel can do that. Even though we know it's waning somewhat. Look at the M1 Ultra from Apple (chuckles). In about 15 months increased transistor density on their package by 6X. So to your earlier point, Bob, we have this sort of these alternative processors that are really changing things. And to Dave Nicholson's point, there's a whole lot of supporting components as well. Do you have a comment on that, Bob? >> Yeah, I mean, it's a great point, Dave. And one thing to bear in mind as well, not only are we seeing a diversity of these different chip architectures and different types of components as a number of us have raised the other big point and I think it was Keith that mentioned it. CXL and interconnect on the chip itself is dramatically changing it. And a lot of the more interesting advances that are going to continue to drive Moore's law forward in terms of the way we think about performance, if perhaps not number of transistors per se, is the interconnects that become available. You're seeing the development of chiplets or tiles, people use different names, but the idea is you can have different components being put together eventually in sort of a Lego block style. And what that's also going to allow, not only is that going to give interesting performance possibilities 'cause of the faster interconnect. So you can share, have shared memory between things which for big workloads like AI, huge data sets can make a huge difference in terms of how you talk to memory over a network connection, for example, but not only that you're going to see more diversity in the types of solutions that can be built. So we're going to see even more choices in hardware from a silicon perspective because you'll be able to piece together different elements. And oh, by the way, the other benefit of that is we've reached a point in chip architectures where not everything benefits from being smaller. We've been so focused and so obsessed when it comes to Moore's law, to the size of each individual transistor and yes, for certain architecture types, CPUs and GPUs in particular, that's absolutely true, but we've already hit the point where things like RF for 5g and wifi and other wireless technologies and a whole bunch of other things actually don't get any better with a smaller transistor size. They actually get worse. So the beauty of these chiplet architectures is you could actually combine different chip manufacturing sizes. You know you hear about four nanometer and five nanometer along with 14 nanometer on a single chip, each one optimized for its specific application yet together, they can give you the best of all worlds. And so we're just at the very beginning of that era, which I think is going to drive a ton of innovation. Again, gets back to my comment about different types of devices located geographically different places at the edge, in the data center, you know, in a private cloud versus a public cloud. All of those things are going to be impacted and there'll be a lot more options because of this silicon diversity and this interconnect diversity that we're just starting to see. >> Yeah, David. David Nicholson's got a graphic on that. They're going to show later. Before we do that, I want to introduce some data. I actually want to ask Keith to comment on this before we, you know, go on. This next slide is some data from ETR that shows the percent of customers that cited difficulty procuring hardware. And you can see the red is they had significant issues and it's most pronounced in laptops and networking hardware on the far right-hand side, but virtually all categories, firewalls, peripheral servers, storage are having moderately difficult procurement issues. That's the sort of pinkish or significant challenges. So Keith, I mean, what are you seeing with your customers in the hardware supply chains and bottlenecks? And you know we're seeing it with automobiles and appliances but so it goes beyond IT. The semiconductor, you know, challenges. What's been the impact on the buyer community and society and do you have any sense as to when it will subside? >> You know, I was just asked this question yesterday and I'm feeling the pain. People question, kind of a side project within the CTO advisor, we built a hybrid infrastructure, traditional IT data center that we're walking with the traditional customer and modernizing that data center. So it was, you know, kind of a snapshot of time in 2016, 2017, 10 gigabit, ARISTA switches, some older Dell's 730 XD switches, you know, speeds and feeds. And we said we would modern that with the latest Intel stack and connected to the public cloud and then the pandemic hit and we are experiencing a lot of the same challenges. I thought we'd easily migrate from 10 gig networking to 25 gig networking path that customers are going on. The 10 gig network switches that I bought used are now double the price because you can't get legacy 10 gig network switches because all of the manufacturers are focusing on the more profitable 25 gig for capacity, even the 25 gig switches. And we're focused on networking right now. It's hard to procure. We're talking about nine to 12 months or more lead time. So we're seeing customers adjust by adopting cloud. But if you remember early on in the pandemic, Microsoft Azure kind of gated customers that didn't have a capacity agreement. So customers are keeping an eye on that. There's a desire to abstract away from the underlying vendor to be able to control or provision your IT services in a way that we do with VMware VP or some other virtualization technology where it doesn't matter who can get me the hardware, they can just get me the hardware because it's critically impacting projects and timelines. >> So that's a great setup Zeus for you with Keith mentioned the earlier the software-defined data center with software-defined networking and cloud. Do you see a day where networking hardware is monetized and it's all about the software, or are we there already? >> No, we're not there already. And I don't see that really happening any time in the near future. I do think it's changed though. And just to be clear, I mean, when you look at that data, this is saying customers have had problems procuring the equipment, right? And there's not a network vendor out there. I've talked to Norman Rice at Extreme, and I've talked to the folks at Cisco and ARISTA about this. They all said they could have had blowout quarters had they had the inventory to ship. So it's not like customers aren't buying this anymore. Right? I do think though, when it comes to networking network has certainly changed some because there's a lot more controls as I mentioned before that you can do in software. And I think the customers need to start thinking about the types of hardware they buy and you know, where they're going to use it and, you know, what its purpose is. Because I've talked to customers that have tried to run software and commodity hardware and where the performance requirements are very high and it's bogged down, right? It just doesn't have the horsepower to run it. And, you know, even when you do that, you have to start thinking of the components you use. The NICs you buy. And I've talked to customers that have simply just gone through the process replacing a NIC card and a commodity box and had some performance problems and, you know, things like that. So if agility is more important than performance, then by all means try running software on commodity hardware. I think that works in some cases. If performance though is more important, that's when you need that kind of turnkey hardware system. And I've actually seen more and more customers reverting back to that model. In fact, when you talk to even some startups I think today about when they come to market, they're delivering things more on appliances because that's what customers want. And so there's this kind of app pivot this pendulum of agility and performance. And if performance absolutely matters, that's when you do need to buy these kind of turnkey, prebuilt hardware systems. If agility matters more, that's when you can go more to software, but the underlying hardware still does matter. So I think, you know, will we ever have a day where you can just run it on whatever hardware? Maybe but I'll long be retired by that point. So I don't care. >> Well, you bring up a good point Zeus. And I remember the early days of cloud, the narrative was, oh, the cloud vendors. They don't use EMC storage, they just run on commodity storage. And then of course, low and behold, you know, they've trot out James Hamilton to talk about all the custom hardware that they were building. And you saw Google and Microsoft follow suit. >> Well, (indistinct) been falling for this forever. Right? And I mean, all the way back to the turn of the century, we were calling for the commodity of hardware. And it's never really happened because you can still drive. As long as you can drive innovation into it, customers will always lean towards the innovation cycles 'cause they get more features faster and things. And so the vendors have done a good job of keeping that cycle up but it'll be a long time before. >> Yeah, and that's why you see companies like Pure Storage. A storage company has 69% gross margins. All right. I want to go jump ahead. We're going to bring up the slide four. I want to go back to something that Bob O'Donnell was talking about, the sort of supporting act. The diversity of silicon and we've marched to the cadence of Moore's law for decades. You know, we asked, you know, is Moore's law dead? We say it's moderating. Dave Nicholson. You want to talk about those supporting components. And you shared with us a slide that shift. You call it a shift from a processor-centric world to a connect-centric world. What do you mean by that? And let's bring up slide four and you can talk to that. >> Yeah, yeah. So first, I want to echo this sentiment that the question does hardware matter is sort of the answer is of course it matters. Maybe the real question should be, should you care about it? And the answer to that is it depends who you are. If you're an end user using an application on your mobile device, maybe you don't care how the architecture is put together. You just care that the service is delivered but as you back away from that and you get closer and closer to the source, someone needs to care about the hardware and it should matter. Why? Because essentially what hardware is doing is it's consuming electricity and dollars and the more efficiently you can configure hardware, the more bang you're going to get for your buck. So it's not only a quantitative question in terms of how much can you deliver? But it also ends up being a qualitative change as capabilities allow for things we couldn't do before, because we just didn't have the aggregate horsepower to do it. So this chart actually comes out of some performance tests that were done. So it happens to be Dell servers with Broadcom components. And the point here was to peel back, you know, peel off the top of the server and look at what's in that server, starting with, you know, the PCI interconnect. So PCIE gen three, gen four, moving forward. What are the effects on from an interconnect versus on performance application performance, translating into new orders per minute, processed per dollar, et cetera, et cetera? If you look at the advances in CPU architecture mapped against the advances in interconnect and storage subsystem performance, you can see that CPU architecture is sort of lagging behind in a way. And Bob mentioned this idea of tiling and all of the different ways to get around that. When we do performance testing, we can actually peg CPUs, just running the performance tests without any actual database environments working. So right now we're at this sort of imbalance point where you have to make sure you design things properly to get the most bang per kilowatt hour of power per dollar input. So the key thing here what this is highlighting is just as a very specific example, you take a card that's designed as a gen three PCIE device, and you plug it into a gen four slot. Now the card is the bottleneck. You plug a gen four card into a gen four slot. Now the gen four slot is the bottleneck. So we're constantly chasing these bottlenecks. Someone has to be focused on that from an architectural perspective, it's critically important. So there's no question that it matters. But of course, various people in this food chain won't care where it comes from. I guess a good analogy might be, where does our food come from? If I get a steak, it's a pink thing wrapped in plastic, right? Well, there are a lot of inputs that a lot of people have to care about to get that to me. Do I care about all of those things? No. Are they important? They're critically important. >> So, okay. So all I want to get to the, okay. So what does this all mean to customers? And so what I'm hearing from you is to balance a system it's becoming, you know, more complicated. And I kind of been waiting for this day for a long time, because as we all know the bottleneck was always the spinning disc, the last mechanical. So people who wrote software knew that when they were doing it right, the disc had to go and do stuff. And so they were doing other things in the software. And now with all these new interconnects and flash and things like you could do atomic rights. And so that opens up new software possibilities and combine that with alternative processes. But what's the so what on this to the customer and the application impact? Can anybody address that? >> Yeah, let me address that for a moment. I want to leverage some of the things that Bob said, Keith said, Zeus said, and David said, yeah. So I'm a bit of a contrarian in some of this. For example, on the chip side. As the chips get smaller, 14 nanometer, 10 nanometer, five nanometer, soon three nanometer, we talk about more cores, but the biggest problem on the chip is the interconnect from the chip 'cause the wires get smaller. People don't realize in 2004 the latency on those wires in the chips was 80 picoseconds. Today it's 1300 picoseconds. That's on the chip. This is why they're not getting faster. So we maybe getting a little bit slowing down in Moore's law. But even as we kind of conquer that you still have the interconnect problem and the interconnect problem goes beyond the chip. It goes within the system, composable architectures. It goes to the point where Keith made, ultimately you need a hybrid because what we're seeing, what I'm seeing and I'm talking to customers, the biggest issue they have is moving data. Whether it be in a chip, in a system, in a data center, between data centers, moving data is now the biggest gating item in performance. So if you want to move it from, let's say your transactional database to your machine learning, it's the bottleneck, it's moving the data. And so when you look at it from a distributed environment, now you've got to move the compute to the data. The only way to get around these bottlenecks today is to spend less time in trying to move the data and more time in taking the compute, the software, running on hardware closer to the data. Go ahead. >> So is this what you mean when Nicholson was talking about a shift from a processor centric world to a connectivity centric world? You're talking about moving the bits across all the different components, not having the processor you're saying is essentially becoming the bottleneck or the memory, I guess. >> Well, that's one of them and there's a lot of different bottlenecks, but it's the data movement itself. It's moving away from, wait, why do we need to move the data? Can we move the compute, the processing closer to the data? Because if we keep them separate and this has been a trend now where people are moving processing away from it. It's like the edge. I think it was Zeus or David. You were talking about the edge earlier. As you look at the edge, who defines the edge, right? Is the edge a closet or is it a sensor? If it's a sensor, how do you do AI at the edge? When you don't have enough power, you don't have enough computable. People were inventing chips to do that. To do all that at the edge, to do AI within the sensor, instead of moving the data to a data center or a cloud to do the processing. Because the lag in latency is always limited by speed of light. How fast can you move the electrons? And all this interconnecting, all the processing, and all the improvement we're seeing in the PCIE bus from three, to four, to five, to CXL, to a higher bandwidth on the network. And that's all great but none of that deals with the speed of light latency. And that's an-- Go ahead. >> You know Marc, no, I just want to just because what you're referring to could be looked at at a macro level, which I think is what you're describing. You can also look at it at a more micro level from a systems design perspective, right? I'm going to be the resident knuckle dragging hardware guy on the panel today. But it's exactly right. You moving compute closer to data includes concepts like peripheral cards that have built in intelligence, right? So again, in some of this testing that I'm referring to, we saw dramatic improvements when you basically took the horsepower instead of using the CPU horsepower for the like IO. Now you have essentially offload engines in the form of storage controllers, rate controllers, of course, for ethernet NICs, smart NICs. And so when you can have these sort of offload engines and we've gone through these waves over time. People think, well, wait a minute, raid controller and NVMe? You know, flash storage devices. Does that make sense? It turns out it does. Why? Because you're actually at a micro level doing exactly what you're referring to. You're bringing compute closer to the data. Now, closer to the data meaning closer to the data storage subsystem. It doesn't solve the macro issue that you're referring to but it is important. Again, going back to this idea of system design optimization, always chasing the bottleneck, plugging the holes. Someone needs to do that in this value chain in order to get the best value for every kilowatt hour of power and every dollar. >> Yeah. >> Well this whole drive performance has created some really interesting architectural designs, right? Like Nickelson, the rise of the DPU right? Brings more processing power into systems that already had a lot of processing power. There's also been some really interesting, you know, kind of innovation in the area of systems architecture too. If you look at the way Nvidia goes to market, their drive kit is a prebuilt piece of hardware, you know, optimized for self-driving cars, right? They partnered with Pure Storage and ARISTA to build that AI-ready infrastructure. I remember when I talked to Charlie Giancarlo, the CEO of Pure about when the three companies rolled that out. He said, "Look, if you're going to do AI, "you need good store. "You need fast storage, fast processor and fast network." And so for customers to be able to put that together themselves was very, very difficult. There's a lot of software that needs tuning as well. So the three companies partner together to create a fully integrated turnkey hardware system with a bunch of optimized software that runs on it. And so in that case, in some ways the hardware was leading the software innovation. And so, the variety of different architectures we have today around hardware has really exploded. And I think it, part of the what Bob brought up at the beginning about the different chip design. >> Yeah, Bob talked about that earlier. Bob, I mean, most AI today is modeling, you know, and a lot of that's done in the cloud and it looks from my standpoint anyway that the future is going to be a lot of AI inferencing at the edge. And that's a radically different architecture, Bob, isn't it? >> It is, it's a completely different architecture. And just to follow up on a couple points, excellent conversation guys. Dave talked about system architecture and really this that's what this boils down to, right? But it's looking at architecture at every level. I was talking about the individual different components the new interconnect methods. There's this new thing called UCIE universal connection. I forget what it stands answer for, but it's a mechanism for doing chiplet architectures, but then again, you have to take it up to the system level, 'cause it's all fine and good. If you have this SOC that's tuned and optimized, but it has to talk to the rest of the system. And that's where you see other issues. And you've seen things like CXL and other interconnect standards, you know, and nobody likes to talk about interconnect 'cause it's really wonky and really technical and not that sexy, but at the end of the day it's incredibly important exactly. To the other points that were being raised like mark raised, for example, about getting that compute closer to where the data is and that's where again, a diversity of chip architectures help and exactly to your last comment there Dave, putting that ability in an edge device is really at the cutting edge of what we're seeing on a semiconductor design and the ability to, for example, maybe it's an FPGA, maybe it's a dedicated AI chip. It's another kind of chip architecture that's being created to do that inferencing on the edge. Because again, it's that the cost and the challenges of moving lots of data, whether it be from say a smartphone to a cloud-based application or whether it be from a private network to a cloud or any other kinds of permutations we can think of really matters. And the other thing is we're tackling bigger problems. So architecturally, not even just architecturally within a system, but when we think about DPUs and the sort of the east west data center movement conversation that we hear Nvidia and others talk about, it's about combining multiple sets of these systems to function together more efficiently again with even bigger sets of data. So really is about tackling where the processing is needed, having the interconnect and the ability to get where the data you need to the right place at the right time. And because those needs are diversifying, we're just going to continue to see an explosion of different choices and options, which is going to make hardware even more essential I would argue than it is today. And so I think what we're going to see not only does hardware matter, it's going to matter even more in the future than it does now. >> Great, yeah. Great discussion, guys. I want to bring Keith back into the conversation here. Keith, if your main expertise in tech is provisioning LUNs, you probably you want to look for another job. So maybe clearly hardware matters, but with software defined everything, do people with hardware expertise matter outside of for instance, component manufacturers or cloud companies? I mean, VMware certainly changed the dynamic in servers. Dell just spun off its most profitable asset and VMware. So it obviously thinks hardware can stand alone. How does an enterprise architect view the shift to software defined hyperscale cloud and how do you see the shifting demand for skills in enterprise IT? >> So I love the question and I'll take a different view of it. If you're a data analyst and your primary value add is that you do ETL transformation, talk to a CDO, a chief data officer over midsize bank a little bit ago. He said 80% of his data scientists' time is done on ETL. Super not value ad. He wants his data scientists to do data science work. Chances are if your only value is that you do LUN provisioning, then you probably don't have a job now. The technologies have gotten much more intelligent. As infrastructure pros, we want to give infrastructure pros the opportunities to shine and I think the software defined nature and the automation that we're seeing vendors undertake, whether it's Dell, HP, Lenovo take your pick that Pure Storage, NetApp that are doing the automation and the ML needed so that these practitioners don't spend 80% of their time doing LUN provisioning and focusing on their true expertise, which is ensuring that data is stored. Data is retrievable, data's protected, et cetera. I think the shift is to focus on that part of the job that you're ensuring no matter where the data's at, because as my data is spread across the enterprise hybrid different types, you know, Dave, you talk about the super cloud a lot. If my data is in the super cloud, protecting that data and securing that data becomes much more complicated when than when it was me just procuring or provisioning LUNs. So when you say, where should the shift be, or look be, you know, focusing on the real value, which is making sure that customers can access data, can recover data, can get data at performance levels that they need within the price point. They need to get at those datasets and where they need it. We talked a lot about where they need out. One last point about this interconnecting. I have this vision and I think we all do of composable infrastructure. This idea that scaled out does not solve every problem. The cloud can give me infinite scale out. Sometimes I just need a single OS with 64 terabytes of RAM and 204 GPUs or GPU instances that single OS does not exist today. And the opportunity is to create composable infrastructure so that we solve a lot of these problems that just simply don't scale out. >> You know, wow. So many interesting points there. I had just interviewed Zhamak Dehghani, who's the founder of Data Mesh last week. And she made a really interesting point. She said, "Think about, we have separate stacks. "We have an application stack and we have "a data pipeline stack and the transaction systems, "the transaction database, we extract data from that," to your point, "We ETL it in, you know, it takes forever. "And then we have this separate sort of data stack." If we're going to inject more intelligence and data and AI into applications, those two stacks, her contention is they have to come together. And when you think about, you know, super cloud bringing compute to data, that was what Haduck was supposed to be. It ended up all sort of going into a central location, but it's almost a rhetorical question. I mean, it seems that that necessitates new thinking around hardware architectures as it kind of everything's the edge. And the other point is to your point, Keith, it's really hard to secure that. So when you can think about offloads, right, you've heard the stats, you know, Nvidia talks about it. Broadcom talks about it that, you know, that 30%, 25 to 30% of the CPU cycles are wasted on doing things like storage offloads, or networking or security. It seems like maybe Zeus you have a comment on this. It seems like new architectures need to come other to support, you know, all of that stuff that Keith and I just dispute. >> Yeah, and by the way, I do want to Keith, the question you just asked. Keith, it's the point I made at the beginning too about engineers do need to be more software-centric, right? They do need to have better software skills. In fact, I remember talking to Cisco about this last year when they surveyed their engineer base, only about a third of 'em had ever made an API call, which you know that that kind of shows this big skillset change, you know, that has to come. But on the point of architectures, I think the big change here is edge because it brings in distributed compute models. Historically, when you think about compute, even with multi-cloud, we never really had multi-cloud. We'd use multiple centralized clouds, but compute was always centralized, right? It was in a branch office, in a data center, in a cloud. With edge what we creates is the rise of distributed computing where we'll have an application that actually accesses different resources and at different edge locations. And I think Marc, you were talking about this, like the edge could be in your IoT device. It could be your campus edge. It could be cellular edge, it could be your car, right? And so we need to start thinkin' about how our applications interact with all those different parts of that edge ecosystem, you know, to create a single experience. The consumer apps, a lot of consumer apps largely works that way. If you think of like app like Uber, right? It pulls in information from all kinds of different edge application, edge services. And, you know, it creates pretty cool experience. We're just starting to get to that point in the business world now. There's a lot of security implications and things like that, but I do think it drives more architectural decisions to be made about how I deploy what data where and where I do my processing, where I do my AI and things like that. It actually makes the world more complicated. In some ways we can do so much more with it, but I think it does drive us more towards turnkey systems, at least initially in order to, you know, ensure performance and security. >> Right. Marc, I wanted to go to you. You had indicated to me that you wanted to chat about this a little bit. You've written quite a bit about the integration of hardware and software. You know, we've watched Oracle's move from, you know, buying Sun and then basically using that in a highly differentiated approach. Engineered systems. What's your take on all that? I know you also have some thoughts on the shift from CapEx to OPEX chime in on that. >> Sure. When you look at it, there are advantages to having one vendor who has the software and hardware. They can synergistically make them work together that you can't do in a commodity basis. If you own the software and somebody else has the hardware, I'll give you an example would be Oracle. As you talked about with their exit data platform, they literally are leveraging microcode in the Intel chips. And now in AMD chips and all the way down to Optane, they make basically AMD database servers work with Optane memory PMM in their storage systems, not MVME, SSD PMM. I'm talking about the cards itself. So there are advantages you can take advantage of if you own the stack, as you were putting out earlier, Dave, of both the software and the hardware. Okay, that's great. But on the other side of that, that tends to give you better performance, but it tends to cost a little more. On the commodity side it costs less but you get less performance. What Zeus had said earlier, it depends where you're running your application. How much performance do you need? What kind of performance do you need? One of the things about moving to the edge and I'll get to the OPEX CapEx in a second. One of the issues about moving to the edge is what kind of processing do you need? If you're running in a CCTV camera on top of a traffic light, how much power do you have? How much cooling do you have that you can run this? And more importantly, do you have to take the data you're getting and move it somewhere else and get processed and the information is sent back? I mean, there are companies out there like Brain Chip that have developed AI chips that can run on the sensor without a CPU. Without any additional memory. So, I mean, there's innovation going on to deal with this question of data movement. There's companies out there like Tachyon that are combining GPUs, CPUs, and DPUs in a single chip. Think of it as super composable architecture. They're looking at being able to do more in less. On the OPEX and CapEx issue. >> Hold that thought, hold that thought on the OPEX CapEx, 'cause we're running out of time and maybe you can wrap on that. I just wanted to pick up on something you said about the integrated hardware software. I mean, other than the fact that, you know, Michael Dell unlocked whatever $40 billion for himself and Silverlake, I was always a fan of a spin in with VMware basically become the Oracle of hardware. Now I know it would've been a nightmare for the ecosystem and culturally, they probably would've had a VMware brain drain, but what does anybody have any thoughts on that as a sort of a thought exercise? I was always a fan of that on paper. >> I got to eat a little crow. I did not like the Dale VMware acquisition for the industry in general. And I think it hurt the industry in general, HPE, Cisco walked away a little bit from that VMware relationship. But when I talked to customers, they loved it. You know, I got to be honest. They absolutely loved the integration. The VxRail, VxRack solution exploded. Nutanix became kind of a afterthought when it came to competing. So that spin in, when we talk about the ability to innovate and the ability to create solutions that you just simply can't create because you don't have the full stack. Dell was well positioned to do that with a potential span in of VMware. >> Yeah, we're going to be-- Go ahead please. >> Yeah, in fact, I think you're right, Keith, it was terrible for the industry. Great for Dell. And I remember talking to Chad Sakac when he was running, you know, VCE, which became Rack and Rail, their ability to stay in lockstep with what VMware was doing. What was the number one workload running on hyperconverged forever? It was VMware. So their ability to remain in lockstep with VMware gave them a huge competitive advantage. And Dell came out of nowhere in, you know, the hyper-converged market and just started taking share because of that relationship. So, you know, this sort I guess it's, you know, from a Dell perspective I thought it gave them a pretty big advantage that they didn't really exploit across their other properties, right? Networking and service and things like they could have given the dominance that VMware had. From an industry perspective though, I do think it's better to have them be coupled. So. >> I agree. I mean, they could. I think they could have dominated in super cloud and maybe they would become the next Oracle where everybody hates 'em, but they kick ass. But guys. We got to wrap up here. And so what I'm going to ask you is I'm going to go and reverse the order this time, you know, big takeaways from this conversation today, which guys by the way, I can't thank you enough phenomenal insights, but big takeaways, any final thoughts, any research that you're working on that you want highlight or you know, what you look for in the future? Try to keep it brief. We'll go in reverse order. Maybe Marc, you could start us off please. >> Sure, on the research front, I'm working on a total cost of ownership of an integrated database analytics machine learning versus separate services. On the other aspect that I would wanted to chat about real quickly, OPEX versus CapEx, the cloud changed the market perception of hardware in the sense that you can use hardware or buy hardware like you do software. As you use it, pay for what you use in arrears. The good thing about that is you're only paying for what you use, period. You're not for what you don't use. I mean, it's compute time, everything else. The bad side about that is you have no predictability in your bill. It's elastic, but every user I've talked to says every month it's different. And from a budgeting perspective, it's very hard to set up your budget year to year and it's causing a lot of nightmares. So it's just something to be aware of. From a CapEx perspective, you have no more CapEx if you're using that kind of base system but you lose a certain amount of control as well. So ultimately that's some of the issues. But my biggest point, my biggest takeaway from this is the biggest issue right now that everybody I talk to in some shape or form it comes down to data movement whether it be ETLs that you talked about Keith or other aspects moving it between hybrid locations, moving it within a system, moving it within a chip. All those are key issues. >> Great, thank you. Okay, CTO advisor, give us your final thoughts. >> All right. Really, really great commentary. Again, I'm going to point back to us taking the walk that our customers are taking, which is trying to do this conversion of all primary data center to a hybrid of which I have this hard earned philosophy that enterprise IT is additive. When we add a service, we rarely subtract a service. So the landscape and service area what we support has to grow. So our research focuses on taking that walk. We are taking a monolithic application, decomposing that to containers, and putting that in a public cloud, and connecting that back private data center and telling that story and walking that walk with our customers. This has been a super enlightening panel. >> Yeah, thank you. Real, real different world coming. David Nicholson, please. >> You know, it really hearkens back to the beginning of the conversation. You talked about momentum in the direction of cloud. I'm sort of spending my time under the hood, getting grease under my fingernails, focusing on where still the lions share of spend will be in coming years, which is OnPrem. And then of course, obviously data center infrastructure for cloud but really diving under the covers and helping folks understand the ramifications of movement between generations of CPU architecture. I know we all know Sapphire Rapids pushed into the future. When's the next Intel release coming? Who knows? We think, you know, in 2023. There have been a lot of people standing by from a practitioner's standpoint asking, well, what do I do between now and then? Does it make sense to upgrade bits and pieces of hardware or go from a last generation to a current generation when we know the next generation is coming? And so I've been very, very focused on looking at how these connectivity components like rate controllers and NICs. I know it's not as sexy as talking about cloud but just how these opponents completely change the game and actually can justify movement from say a 14th-generation architecture to a 15th-generation architecture today, even though gen 16 is coming, let's say 12 months from now. So that's where I am. Keep my phone number in the Rolodex. I literally reference Rolodex intentionally because like I said, I'm in there under the hood and it's not as sexy. But yeah, so that's what I'm focused on Dave. >> Well, you know, to paraphrase it, maybe derivative paraphrase of, you know, Larry Ellison's rant on what is cloud? It's operating systems and databases, et cetera. Rate controllers and NICs live inside of clouds. All right. You know, one of the reasons I love working with you guys is 'cause have such a wide observation space and Zeus Kerravala you, of all people, you know you have your fingers in a lot of pies. So give us your final thoughts. >> Yeah, I'm not a propeller heady as my chip counterparts here. (all laugh) So, you know, I look at the world a little differently and a lot of my research I'm doing now is the impact that distributed computing has on customer employee experiences, right? You talk to every business and how the experiences they deliver to their customers is really differentiating how they go to market. And so they're looking at these different ways of feeding up data and analytics and things like that in different places. And I think this is going to have a really profound impact on enterprise IT architecture. We're putting more data, more compute in more places all the way down to like little micro edges and retailers and things like that. And so we need the variety. Historically, if you think back to when I was in IT you know, pre-Y2K, we didn't have a lot of choice in things, right? We had a server that was rack mount or standup, right? And there wasn't a whole lot of, you know, differences in choice. But today we can deploy, you know, these really high-performance compute systems on little blades inside servers or inside, you know, autonomous vehicles and things. I think the world from here gets... You know, just the choice of what we have and the way hardware and software works together is really going to, I think, change the world the way we do things. We're already seeing that, like I said, in the consumer world, right? There's so many things you can do from, you know, smart home perspective, you know, natural language processing, stuff like that. And it's starting to hit businesses now. So just wait and watch the next five years. >> Yeah, totally. The computing power at the edge is just going to be mind blowing. >> It's unbelievable what you can do at the edge. >> Yeah, yeah. Hey Z, I just want to say that we know you're not a propeller head and I for one would like to thank you for having your master's thesis hanging on the wall behind you 'cause we know that you studied basket weaving. >> I was actually a physics math major, so. >> Good man. Another math major. All right, Bob O'Donnell, you're going to bring us home. I mean, we've seen the importance of semiconductors and silicon in our everyday lives, but your last thoughts please. >> Sure and just to clarify, by the way I was a great books major and this was actually for my final paper. And so I was like philosophy and all that kind of stuff and literature but I still somehow got into tech. Look, it's been a great conversation and I want to pick up a little bit on a comment Zeus made, which is this it's the combination of the hardware and the software and coming together and the manner with which that needs to happen, I think is critically important. And the other thing is because of the diversity of the chip architectures and all those different pieces and elements, it's going to be how software tools evolve to adapt to that new world. So I look at things like what Intel's trying to do with oneAPI. You know, what Nvidia has done with CUDA. What other platform companies are trying to create tools that allow them to leverage the hardware, but also embrace the variety of hardware that is there. And so as those software development environments and software development tools evolve to take advantage of these new capabilities, that's going to open up a lot of interesting opportunities that can leverage all these new chip architectures. That can leverage all these new interconnects. That can leverage all these new system architectures and figure out ways to make that all happen, I think is going to be critically important. And then finally, I'll mention the research I'm actually currently working on is on private 5g and how companies are thinking about deploying private 5g and the potential for edge applications for that. So I'm doing a survey of several hundred us companies as we speak and really looking forward to getting that done in the next couple of weeks. >> Yeah, look forward to that. Guys, again, thank you so much. Outstanding conversation. Anybody going to be at Dell tech world in a couple of weeks? Bob's going to be there. Dave Nicholson. Well drinks on me and guys I really can't thank you enough for the insights and your participation today. Really appreciate it. Okay, and thank you for watching this special power panel episode of theCube Insights powered by ETR. Remember we publish each week on Siliconangle.com and wikibon.com. All these episodes they're available as podcasts. DM me or any of these guys. I'm at DVellante. You can email me at David.Vellante@siliconangle.com. Check out etr.ai for all the data. This is Dave Vellante. We'll see you next time. (upbeat music)

(bright music) >> Well, welcome back to theCUBE's coverage of AWS re:Invent 2021. We're here joined by Ian Buck, general manager and vice president of Accelerated Computing at NVIDIA. I'm John Furrrier, host of theCUBE. Ian, thanks for coming on. >> Oh, thanks for having me. >> So NVIDIA, obviously, great brand. Congratulations on all your continued success. Everyone who does anything in graphics knows that GPU's are hot, and you guys have a great brand, great success in the company. But AI and machine learning, we're seeing the trend significantly being powered by the GPU's and other systems. So it's a key part of everything. So what's the trends that you're seeing in ML and AI that's accelerating computing to the cloud? >> Yeah. I mean, AI is kind of driving breakthroughs and innovations across so many segments, so many different use cases. We see it showing up with things like credit card fraud prevention, and product and content recommendations. Really, it's the new engine behind search engines, is AI. People are applying AI to things like meeting transcriptions, virtual calls like this, using AI to actually capture what was said. And that gets applied in person-to-person interactions. We also see it in intelligence assistance for contact center automation, or chat bots, medical imaging, and intelligence stores, and warehouses, and everywhere. It's really amazing what AI has been demonstrating, what it can do, and its new use cases are showing up all the time. >> You know, Ian, I'd love to get your thoughts on how the world's evolved, just in the past few years alone, with cloud. And certainly, the pandemic's proven it. You had this whole kind of fullstack mindset, initially, and now you're seeing more of a horizontal scale, but yet, enabling this vertical specialization in applications. I mean, you mentioned some of those apps. The new enablers, this kind of, the horizontal play with enablement for, you know, specialization with data, this is a huge shift that's going on. It's been happening. What's your reaction to that? >> Yeah. The innovation's on two fronts. There's a horizontal front, which is basically the different kinds of neural networks or AIs, as well as machine learning techniques, that are just being invented by researchers and the community at large, including Amazon. You know, it started with these convolutional neural networks, which are great for image processing, but has expanded more recently into recurrent neural networks, transformer models, which are great for language and language and understanding, and then the new hot topic, graph neural networks, where the actual graph now is trained as a neural network. You have this underpinning of great AI technologies that are being invented around the world. NVIDIA's role is to try to productize that and provide a platform for people to do that innovation. And then, take the next step and innovate vertically. Take it and apply it to a particular field, like medical, like healthcare and medical imaging, applying AI so that radiologists can have an AI assistant with them and highlight different parts of the scan that may be troublesome or worrying, or require some more investigation. Using it for robotics, building virtual worlds where robots can be trained in a virtual environment, their AI being constantly trained and reinforced, and learn how to do certain activities and techniques. So that the first time it's ever downloaded into a real robot, it works right out of the box. To activate that, we are creating different vertical solutions, vertical stacks, vertical products, that talk the languages of those businesses, of those users. In medical imaging, it's processing medical data, which is obviously a very complicated, large format data, often three-dimensional voxels. In robotics, it's building, combining both our graphics and simulation technologies, along with the AI training capabilities and difference capabilities, in order to run in real time. Those are just two simple- >> Yeah, no. I mean, it's just so cutting-edge, it's so relevant. I mean, I think one of the things you mentioned about the neural networks, specifically, the graph neural networks, I mean, we saw, I mean, just go back to the late 2000s, how unstructured data, or object storage created, a lot of people realized a lot of value out of that. Now you got graph value, you got network effect, you got all kinds of new patterns. You guys have this notion of graph neural networks that's out there. What is a graph neural network, and what does it actually mean from a deep learning and an AI perspective? >> Yeah. I mean, a graph is exactly what it sounds like. You have points that are connected to each other, that establish relationships. In the example of Amazon.com, you might have buyers, distributors, sellers, and all of them are buying, or recommending, or selling different products. And they're represented in a graph. If I buy something from you and from you, I'm connected to those endpoints, and likewise, more deeply across a supply chain, or warehouse, or other buyers and sellers across the network. What's new right now is, that those connections now can be treated and trained like a neural network, understanding the relationship, how strong is that connection between that buyer and seller, or the distributor and supplier, and then build up a network to figure out and understand patterns across them. For example, what products I may like, 'cause I have this connection in my graph, what other products may meet those requirements? Or, also, identifying things like fraud, When patterns and buying patterns don't match what a graph neural networks should say would be the typical kind of graph connectivity, the different kind of weights and connections between the two, captured by the frequency of how often I buy things, or how I rate them or give them stars, or other such use cases. This application, graph neural networks, which is basically capturing the connections of all things with all people, especially in the world of e-commerce, is very exciting to a new application of applying AI to optimizing business, to reducing fraud, and letting us, you know, get access to the products that we want. They have our recommendations be things that excite us and want us to buy things, and buy more. >> That's a great setup for the real conversation that's going on here at re:Invent, which is new kinds of workloads are changing the game, people are refactoring their business with, not just re-platforming, but actually using this to identify value. And also, your cloud scale allows you to have the compute power to, you know, look at a note in an arc and actually code that. It's all science, it's all computer science, all at scale. So with that, that brings up the whole AWS relationship. Can you tell us how you're working with AWS, specifically? >> Yeah, AWS have been a great partner, and one of the first cloud providers to ever provide GPUs to the cloud. More recently, we've announced two new instances, the G5 instance, which is based on our A10G GPU, which supports the NVIDIA RTX technology, our rendering technology, for real-time ray tracing in graphics and game streaming. This is our highest performance graphics enhanced application, allows for those high-performance graphics applications to be directly hosted in the cloud. And, of course, runs everything else as well. It has access to our AI technology and runs all of our AI stacks. We also announced, with AWS, the G5 G instance. This is exciting because it's the first Graviton or Arm-based processor connected to a GPU and successful in the cloud. The focus here is Android gaming and machine learning inference. And we're excited to see the advancements that Amazon is making and AWS is making, with Arm in the cloud. And we're glad to be part of that journey. >> Well, congratulations. I remember, I was just watching my interview with James Hamilton from AWS 2013 and 2014. He was teasing this out, that they're going to build their own, get in there, and build their own connections to take that latency down and do other things. This is kind of the harvest of all that. As you start looking at these new interfaces, and the new servers, new technology that you guys are doing, you're enabling applications. What do you see this enabling? As this new capability comes out, new speed, more performance, but also, now it's enabling more capabilities so that new workloads can be realized. What would you say to folks who want to ask that question? >> Well, so first off, I think Arm is here to stay. We can see the growth and explosion of Arm, led of course, by Graviton and AWS, but many others. And by bringing all of NVIDIA's rendering graphics, machine learning and AI technologies to Arm, we can help bring that innovation that Arm allows, that open innovation, because there's an open architecture, to the entire ecosystem. We can help bring it forward to the state of the art in AI machine learning and graphics. All of our software that we release is both supportive, both on x86 and on Arm equally, and including all of our AI stacks. So most notably, for inference, the deployment of AI models, we have the NVIDIA Triton inference server. This is our inference serving software, where after you've trained a model, you want to deploy it at scale on any CPU, or GPU instance, for that matter. So we support both CPUs and GPUs with Triton. It's natively integrated with SageMaker and provides the benefit of all those performance optimizations. Features like dynamic batching, it supports all the different AI frameworks, from PyTorch to TensorFlow, even a generalized Python code. We're activating, and help activating, the Arm ecosystem, as well as bringing all those new AI use cases, and all those different performance levels with our partnership with AWS and all the different cloud instances. >> And you guys are making it really easy for people to use use the technology. That brings up the next, kind of, question I wanted to ask you. I mean, a lot of people are really going in, jumping in big-time into this. They're adopting AI, either they're moving it from prototype to production. There's always some gaps, whether it's, you know, knowledge, skills gaps, or whatever. But people are accelerating into the AI and leaning into it hard. What advancements has NVIDIA made to make it more accessible for people to move faster through the system, through the process? >> Yeah. It's one of the biggest challenges. You know, the promise of AI, all the publications that are coming out, all the great research, you know, how can you make it more accessible or easier to use by more people? Rather than just being an AI researcher, which is obviously a very challenging and interesting field, but not one that's directly connected to the business. NVIDIA is trying to provide a fullstack approach to AI. So as we discover or see these AI technologies become available, we produce SDKs to help activate them or connect them with developers around the world. We have over 150 different SDKs at this point, serving industries from gaming, to design, to life sciences, to earth sciences. We even have stuff to help simulate quantum computing. And of course, all the work we're doing with AI, 5G, and robotics. So we actually just introduced about 65 new updates, just this past month, on all those SDKs. Some of the newer stuff that's really exciting is the large language models. People are building some amazing AI that's capable of understanding the corpus of, like, human understanding. These language models that are trained on literally the content of the internet to provide general purpose or open-domain chatbots, so the customer is going to have a new kind of experience with the computer or the cloud. We're offering those large language models, as well as AI frameworks, to help companies take advantage of this new kind of technology. >> You know, Ian, every time I do an interview with NVIDIA or talk about NVIDIA, my kids and friends, first thing they say is, "Can you get me a good graphics card?" They all want the best thing in their rig. Obviously the gaming market's hot and known for that. But there's a huge software team behind NVIDIA. This is well-known. Your CEO is always talking about it on his keynotes. You're in the software business. And you do have hardware, you are integrating with Graviton and other things. But it's a software practice. This is software. This is all about software. >> Right. >> Can you share, kind of, more about how NVIDIA culture and their cloud culture, and specifically around the scale, I mean, you hit every use case. So what's the software culture there at NVIDIA? >> Yeah, NVIDIA's actually a bigger, we have more software people than hardware people. But people don't often realize this. And in fact, that it's because of, it just starts with the chip, and obviously, building great silicon is necessary to provide that level of innovation. But it's expanded dramatically from there. Not just the silicon and the GPU, but the server designs themselves. We actually do entire server designs ourselves, to help build out this infrastructure. We consume it and use it ourselves, and build our own supercomputers to use AI to improve our products. And then, all that software that we build on top, we make it available, as I mentioned before, as containers on our NGC container store, container registry, which is accessible from AWS, to connect to those vertical markets. Instead of just opening up the hardware and letting the ecosystem develop on it, they can, with the low-level and programmatic stacks that we provide with CUDA. We believe that those vertical stacks are the ways we can help accelerate and advance AI. And that's why we make them so available. >> And programmable software is so much easier. I want to get that plug in for, I think it's worth noting that you guys are heavy hardcore, especially on the AI side, and it's worth calling out. Getting back to the customers who are bridging that gap and getting out there, what are the metrics they should consider as they're deploying AI? What are success metrics? What does success look like? Can you share any insight into what they should be thinking about, and looking at how they're doing? >> Yeah. For training, it's all about time-to-solution. It's not the hardware that's the cost, it's the opportunity that AI can provide to your business, and the productivity of those data scientists which are developing them, which are not easy to come by. So what we hear from customers is they need a fast time-to-solution to allow people to prototype very quickly, to train a model to convergence, to get into production quickly, and of course, move on to the next or continue to refine it. >> John Furrier: Often. >> So in training, it's time-to-solution. For inference, it's about your ability to deploy at scale. Often people need to have real-time requirements. They want to run in a certain amount of latency, in a certain amount of time. And typically, most companies don't have a single AI model. They have a collection of them they want to run for a single service or across multiple services. That's where you can aggregate some of your infrastructure. Leveraging the Triton inference server, I mentioned before, can actually run multiple models on a single GPU saving costs, optimizing for efficiency, yet still meeting the requirements for latency and the real-time experience, so that our customers have a good interaction with the AI. >> Awesome. Great. Let's get into the customer examples. You guys have, obviously, great customers. Can you share some of the use cases examples with customers, notable customers? >> Yeah. One great part about working at NVIDIA is, as technology company, you get to engage with such amazing customers across many verticals. Some of the ones that are pretty exciting right now, Netflix is using the G4 instances to do a video effects and animation content from anywhere in the world, in the cloud, as a cloud creation content platform. We work in the energy field. Siemens energy is actually using AI combined with simulation to do predictive maintenance on their energy plants, preventing, or optimizing, onsite inspection activities and eliminating downtime, which is saving a lot of money for the energy industry. We have worked with Oxford University. Oxford University actually has over 20 million artifacts and specimens and collections, across its gardens and museums and libraries. They're actually using NVIDIA GPU's and Amazon to do enhanced image recognition to classify all these things, which would take literally years going through manually, each of these artifacts. Using AI, we can quickly catalog all of them and connect them with their users. Great stories across graphics, across industries, across research, that it's just so exciting to see what people are doing with our technology, together with Amazon. >> Ian, thank you so much for coming on theCUBE. I really appreciate it. A lot of great content there. We probably could go another hour. All the great stuff going on at NVIDIA. Any closing remarks you want to share, as we wrap this last minute up? >> You know, really what NVIDIA's about, is accelerating cloud computing. Whether it be AI, machine learning, graphics, or high-performance computing and simulation. And AWS was one of the first with this, in the beginning, and they continue to bring out great instances to help connect the cloud and accelerated computing with all the different opportunities. The integrations with EC2, with SageMaker, with EKS, and ECS. The new instances with G5 and G5 G. Very excited to see all the work that we're doing together. >> Ian Buck, general manager and vice president of Accelerated Computing. I mean, how can you not love that title? We want more power, more faster, come on. More computing. No one's going to complain with more computing. Ian, thanks for coming on. >> Thank you. >> Appreciate it. I'm John Furrier, host of theCUBE. You're watching Amazon coverage re:Invent 2021. Thanks for watching. (bright music)

>>Yeah. >>Hello. Welcome to the Special Cube conversation here in Palo Alto, California. I'm John for a host of the Cube. We have a conversation around a I for the enterprise. What this means I got two great guests. John Finelli, Vice President, virtual GPU at NVIDIA and Maurizio D V D C T o University of Pisa in Italy. Uh, Practitioner, customer partner, um, got VM world coming up. A lot of action happening in the enterprise. John. Great to see you. Nice to meet you. Remotely coming in from Italy for this remote. >>John. Thanks for having us on again. >>Yeah. Nice to meet >>you. I wish we could be in person face to face, but that's coming soon. Hopefully, John, you were talking. We were just talking about before we came on camera about AI for the enterprise. And the last time I saw you in person was in Cuba interview. We were talking about some of the work you guys were doing in AI. It's gotten so much stronger and broader and the execution of an video, the success you're having set the table for us. What is the ai for the enterprise conversation frame? >>Sure. So, um, we, uh we've been working with enterprises today on how they can deliver a I or explore AI or get involved in a I, um uh, in a standard way in the way that they're used to managing and operating their data centre. Um, writing on top of you know, they're Dell servers with B M or V sphere. Um, so that AI feels like a standard workload that night organisation can deliver to their engineers and data scientists. And then the flip side of that, of course, is ensuring that engineers and data scientists get the workloads position to them or have access to them in the way that they need them. So it's no longer a trouble ticket that you have to submit to, I t and you know, count the hours or days or weeks until you you can get new hardware, right By being able to pull it into the mainstream data centre. I can enable self service provisioning for those folks. So we actually we make a I more consumable or easier to manage for I t administrators and then for the engineers and the data scientists, etcetera. We make it easy for them to get access to those resources so they can get to their work right away. >>Quite progress in the past two years. Congratulations on that and looking. It's only the beginning is Day one Mercy. I want to ask you about what's going on as the CTO University piece of what's happening down there. Tell us a little bit about what's going on. You have the centre of excellence there. What does that mean? What does that include? >>Uh, you know, uh, University of Peace. Are you one of one of the biggest and oldest in Italy? Uh, if you have to give you some numbers is around 50 K students and 3000 staff between, uh, professors resurgence and that cabinet receive staff. So I we are looking into data operation of the centres and especially supports for scientific computing. And, uh, this is our our daily work. Let's say this, uh, taking us a lot of times, but, you know, we are able to, uh, reserve a merchant percentage of our time, Uh, for r and D, And this is where the centre of excellence is, Uh, is coming out. Uh, so we are always looking into new kinds of technologies that we can put together to build new solutions to do next generation computing gas. We always say we are looking for the right partners to do things together. And at the end of the day is the work that is good for us is good for our partners and typically, uh, ends in a production system for our university. So is the evolution of the scientific computing environment that we have. >>Yeah. And you guys have a great track record and reputation of, you know, R and D, testing software, hardware combinations and sharing those best practises, you know, with covid impact in the world. Certainly we see it on the supply chain side. Uh, and John, we heard Jensen, your CEO and video talk multiple keynotes. Now about software, uh, and video being a software company. Dell, you mentioned Dale and VM Ware. You know, Covid has brought this virtualisation world back. And now hybrid. Those are words that we used basically in the text industry. Now it's you're hearing hybrid and virtualisation kicked around in real world. So it's ironic that vm ware and El, uh, and the Cube eventually all of us together doing more virtual stuff. So with covid impacting the world, how does that change you guys? Because software is more important. You gotta leverage the hardware you got, Whether it's Dell or in the cloud, this is a huge change. >>Yeah. So, uh, as you mentioned organisations and enterprises, you know, they're looking at things differently now, Um, you know, the idea of hybrid. You know, when you talk to tech folks and we think about hybrid, we always think about you know, how the different technology works. Um, what we're hearing from customers is hybrid, you know, effectively translates into, you know, two days in the office, three days remote, you know, in the future when they actually start going back to the office. So hybrid work is actually driving the need for hybrid I t. Or or the ability to share resources more effectively. Um, And to think about having resources wherever you are, whether you're working from home or you're in the office that day, you need to have access to the same resources. And that's where you know the the ability to virtualize those resources and provide that access makes that hybrid part seamless >>mercy What's your world has really changed. You have students and faculty. You know, Things used to be easy in the old days. Physical in this network. That network now virtual there. You must really be having him having impact. >>Yeah, we have. We have. Of course. As you can imagine, a big impact, Uh, in any kind of the i t offering, uh, from, uh, design new networking technologies, deploying new networking technologies, uh, new kind of operation we find. We found it at them. We were not able anymore to do burr metal operations directly, but, uh, from the i t point of view, uh, we were how can I say prepared in the sense that, uh, we ran from three or four years parallel, uh, environment. We have bare metal and virtual. So as you can imagine, traditional bare metal HPC cluster D g d g X machines, uh, multi GPU s and so on. But in parallel, we have developed, uh, visual environment that at the beginning was, as you can imagine, used, uh, for traditional enterprise application, or VD. I, uh, we have a significant significant arise on a farm with the grid for remote desktop remote pull station that we are using for, for example, uh, developing a virtual classroom or visual go stations. And so this is was typical the typical operation that we did the individual world. But in the same infrastructure, we were able to develop first HPC individual borders of utilisation of the HPC resources for our researchers and, uh, at the end, ai ai offering and ai, uh, software for our for our researchers, you can imagine our vehicle infrastructure as a sort of white board where we are able to design new solution, uh, in a fast way without losing too much performance. And in the case of the AI, we will see that we the performance are almost the same at the bare metal. But with all the flexibility that we needed in the covid 19 world and in the future world, too. >>So a couple things that I want to get John's thoughts as well performance you mentioned you mentioned hybrid virtual. How does VM Ware and NVIDIA fit into all this as you put this together, okay, because you bring up performance. That's now table stakes. He's leading scale and performance are really on the table. everyone's looking at it. How does VM ware an NVIDIA John fit in with the university's work? >>Sure. So, um, I think you're right when it comes to, uh, you know, enterprises or mainstream enterprises beginning their initial foray into into a I, um there are, of course, as performance in scale and also kind of ease of use and familiarity are all kind of things that come into play in terms of when an enterprise starts to think about it. And, um, we have a history with VM Ware working on this technology. So in 2019, we introduced our virtual compute server with VM Ware, which allowed us to effectively virtual is the Cuda Compute driver at last year's VM World in 2020 the CEOs of both companies got together and made an announcement that we were going to bring a I R entire video AI platform to the Enterprise on top of the sphere. And we did that, Um, starting in March this year, we we we finalise that with the introduction of GM wears V, Sphere seven, update two and the early access at the time of NVIDIA ai Enterprise. And, um, we have now gone to production with both of those products. And so customers, Um, like the University of Pisa are now using our production capabilities. And, um, whenever you virtualize in particular and in something like a I where performances is really important. Um, the first question that comes up is, uh doesn't work and And how quickly does it work Or or, you know, from an I t audience? A lot of times you get the How much did it slow down? And and and so we We've worked really closely from an NVIDIA software perspective and a bm wear perspective. And we really talk about in media enterprise with these fair seven as optimist, certified and supported. And the net of that is, we've been able to run the standard industry benchmarks for single node as well as multi note performance, with about maybe potentially a 2% degradation in performance, depending on the workload. Of course, it's very different, but but effectively being able to trade that performance for the accessibility, the ease of use, um, and even using things like we realise, automation for self service for the data scientists, Um and so that's kind of how we've been pulling it together for the market. >>Great stuff. Well, I got to ask you. I mean, people have that reaction of about the performance. I think you're being polite. Um, around how you said that shows the expectation. It's kind of sceptical, uh, and so I got to ask you, the impact of this is pretty significant. What is it now that customers can do that? They couldn't or couldn't feel they had before? Because if the expectations as well as it worked well, I mean, there's a fast means. It works, but like performance is always concerned. What's different now? What what's the bottom line impact on what country do now that they couldn't do before. >>So the bottom line impact is that AI is now accessible for the enterprise across there. Called their mainstream data centre, enterprises typically use consistent building blocks like the Dell VX rail products, right where they have to use servers that are common standard across the data centre. And now, with NVIDIA Enterprise and B M R V sphere, they're able to manage their AI in the same way that they're used to managing their data centre today. So there's no retraining. There's no separate clusters. There isn't like a shadow I t. So this really allows an enterprise to efficiently deploy um, and cost effectively Deploy it, uh, it without because there's no performance degradation without compromising what their their their data scientists and researchers are looking for. And then the flip side is for the data science and researcher, um, using some of the self service automation that I spoke about earlier, they're able to get a virtual machine today that maybe as a half a GPU as their models grow, they do more exploring. They might get a full GPU or or to GPS in a virtual machine. And their environment doesn't change because it's all connected to the back end storage. And so for the for the developer and the researcher, um, it makes it seamless. So it's really kind of a win for both Nike and for the user. And again, University of Pisa is doing some amazing things in terms of the workloads that they're doing, Um, and, uh and, uh, and are validating that performance. >>Weigh in on this. Share your opinion on or your reaction to that, What you can do now that you couldn't do before. Could you share your experience? >>Our experience is, uh, of course, if you if you go to your, uh, data scientists or researchers, the idea of, uh, sacrificing four months to flexibility at the beginning is not so well accepted. It's okay for, uh, for the Eid management, As John was saying, you have people that is know how to deal with the virtual infrastructure, so nothing changed for them. But at the end of the day, we were able to, uh, uh, test with our data. Scientists are researchers veteran The performance of us almost similar around really 95% of the performance for the internal developer developer to our work clothes. So we are not dealing with benchmarks. We have some, uh, work clothes that are internally developed and apply to healthcare music generator or some other strange project that we have inside and were able to show that the performance on the beautiful and their metal world were almost the same. We, the addition that individual world, you are much more flexible. You are able to reconfigure every finger very fast. You are able to design solution for your researcher, uh, in a more flexible way. An effective way we are. We were able to use the latest technologies from Dell Technologies and Vidia. You can imagine from the latest power edge the latest cuts from NVIDIA. The latest network cards from NVIDIA, like the blue Field to the latest, uh, switches to set up an infrastructure that at the end of the day is our winning platform for our that aside, >>a great collaboration. Congratulations. Exciting. Um, get the latest and greatest and and get the new benchmarks out their new playbooks. New best practises. I do have to ask you marriage, if you don't mind me asking why Look at virtualizing ai workloads. What's the motivation? Why did you look at virtualizing ai work clothes? >>Oh, for the sake of flexibility Because, you know, uh, in the latest couple of years, the ai resources are never enough. So we are. If you go after the bare metal, uh, installation, you are going into, uh, a world that is developing very fastly. But of course, you can afford all the bare metal, uh, infrastructure that your data scientists are asking for. So, uh, we decided to integrate our view. Dual infrastructure with AI, uh, resources in order to be able to, uh, use in different ways in a more flexible way. Of course. Uh, we have a We have a two parallels world. We still have a bare metal infrastructure. We are growing the bare metal infrastructure. But at the same time, we are growing our vehicle infrastructure because it's flexible, because we because our our stuff, people are happy about how the platform behaviour and they know how to deal them so they don't have to learn anything new. So it's a sort of comfort zone for everybody. >>I mean, no one ever got hurt virtualizing things that makes it makes things go better faster building on on that workloads. John, I gotta ask you, you're on the end video side. You You see this real up close than video? Why do people look at virtualizing ai workloads is the unification benefit. I mean, ai implies a lot of things, implies you have access to data. It implies that silos don't exist. I mean, that doesn't mean that's hard. I mean, is this real people actually looking at this? How is it working? >>Yeah. So? So again, um you know for all the benefits and activity today AI brings a I can be pretty complex, right? It's complex software to set up and to manage. And, um, within the day I enterprise, we're really focusing in on ensuring that it's easier for organisations to use. For example Um, you know, I mentioned you know, we we had introduced a virtual compute server bcs, um uh, two years ago and and that that has seen some some really interesting adoption. Some, uh, enterprise use cases. But what we found is that at the driver level, um, it still wasn't accessible for the majority of enterprises. And so what we've done is we've built upon that with NVIDIA Enterprise and we're bringing in pre built containers that remove some of the complexities. You know, AI has a lot of open source components and trying to ensure that all the open source dependencies are resolved so you can get the AI developers and researchers and data scientists. Actually doing their work can be complex. And so what we've done is we've brought these pre built containers that allow you to do everything from your initial data preparation data science, using things like video rapids, um, to do your training, using pytorch and tensorflow to optimise those models using tensor rt and then to deploy them using what we call in video Triton Server Inference in server. Really helping that ai loop become accessible, that ai workflow as something that an enterprise can manage as part of their common core infrastructure >>having the performance and the tools available? It's just a huge godsend people love. That only makes them more productive and again scales of existing stuff. Okay, great stuff. Great insight. I have to ask, What's next one's collaboration? This is one of those better together situations. It's working. Um, Mauricio, what's next for your collaboration with Dell VM Ware and video? >>We will not be for sure. We will not stop here. Uh, we are just starting working on new things, looking for new development, uh, looking for the next beast. Come, uh, you know, the digital world is something that is moving very fast. Uh, and we are We will not We will not stop here because because they, um the outcome of this work has been a very big for for our research group. And what John was saying This the fact that all the software stock for AI are simplified is something that has been, uh, accepted. Very well, of course you can imagine researching is developing new things. But for people that needs, uh, integrated workflow. The work that NVIDIA has done in the development of software package in developing containers, that gives the end user, uh, the capabilities of running their workloads is really something that some years ago it was unbelievable. Now, everything is really is really easy to manage. >>John mentioned open source, obviously a big part of this. What are you going to? Quick, Quick follow if you don't mind. Are you going to share your results so people can can look at this so they can have an easier path to AI? >>Oh, yes, of course. All the all the work, The work that is done at an ideal level from University of Visa is here to be shared. So we we as, uh, as much as we have time to write down we are. We are trying to find a way to share the results of the work that we're doing with our partner, Dell and NVIDIA. So for sure will be shared >>well, except we'll get that link in the comments, John, your thoughts. Final thoughts on the on the on the collaboration, uh, with the University of Pisa and Delvian, where in the video is is all go next? >>Sure. So So with University of Pisa, We're you know, we're absolutely, uh, you know, grateful to Morocco and his team for the work they're doing and the feedback they're sharing with us. Um, we're learning a lot from them in terms of things we can do better and things that we can add to the product. So that's a fantastic collaboration. Um, I believe that Mauricio has a session at the M World. So if you want to actually learn about some of the workloads, um, you know, they're doing, like, music generation. They're doing, you know, covid 19 research. They're doing deep, multi level, uh, deep learning training. So there's some really interesting work there, and so we want to continue that partnership. University of Pisa, um, again, across all four of us, uh, university, NVIDIA, Dell and VM Ware. And then on the tech side, you know, for our enterprise customers, um, you know, one of the things that we actually didn't speak much about was, um I mentioned that the product is optimised certified and supported, and I think that support cannot be understated. Right? So as enterprises start to move into these new areas, they want to know that they can pick up the phone and call in video or VM ware. Adele, and they're going to get support for these new workloads as they're running them. Um, we were also continuing, uh, you know, to to think about we spent a lot of time today on, like, the developer side of things and developing ai. But the flip side of that, of course, is that when those ai apps are available or ai enhanced apps, right, Pretty much every enterprise app today is adding a I capabilities all of our partners in the enterprise software space and so you can think of a beady eye enterprises having a runtime component so that as you deploy your applications into the data centre, they're going to be automatically take advantage of the GPS that you have there. And so we're seeing this, uh, future as you're talking about the collaboration going forward, where the standard data centre building block still maintains and is going to be something like a VX rail two U server. But instead of just being CPU storage and RAM, they're all going to go with CPU, GPU, storage and RAM. And that's going to be the norm. And every enterprise application is going to be infused with AI and be able to take advantage of GPS in that scenario. >>Great stuff, ai for the enterprise. This is a great QB conversation. Just the beginning. We'll be having more of these virtualizing ai workloads is real impacts data scientists impacts that compute the edge, all aspects of the new environment we're all living in. John. Great to see you, Maurizio here to meet you and all the way in Italy looking for the meeting in person and good luck in your session. I just got a note here on the session. It's at VM World. Uh, it's session 22 63 I believe, um And so if anyone's watching, Want to check that out? Um, love to hear more. Thanks for coming on. Appreciate it. >>Thanks for having us. Thanks to >>its acute conversation. I'm John for your host. Thanks for watching. We'll talk to you soon. Yeah,

>> As I've said many times on theCUBE for years, decades even we've marched to the cadence of Moore's law relying on the doubling of performance every 18 months or so, but no longer is this the main spring of innovation for technology rather it's the combination of data applying machine intelligence and the cloud supported by the relentless reduction of the cost of compute and storage and the build-out of a massively distributed computer network. Very importantly, the last several years alternative processors have emerged to support offloading work and performing specific tests. GPUs are the most widely known example of this trend with the ascendancy of Nvidia for certain applications like gaming and crypto mining and more recently machine learning. But in the middle of last decade we saw the early development focused on the DPU, the data processing unit, which is projected to make a huge impact on data centers in the coming years as we move into the next era of cloud. And with me is Pradeep Sindhu who's the co-founder and CEO of Fungible, a company specializing in the design and development of DPUs. Pradeep, welcome to theCUBE. Great to see you. >> Thank-you, Dave and thank-you for having me. >> You're very welcome. So okay, my first question is don't CPUs and GPUs process data already. Why do we need a DPU? >> That is a natural question to ask. And CPUs have been around in one form or another for almost 55, maybe 60 years. And this is when general purpose computing was invented and essentially all CPUs went to x86 architecture by and large and of course is used very heavily in mobile computing, but x86 is primarily used in data center which is our focus. Now, you can understand that that architecture of a general purpose CPUs has been refined heavily by some of the smartest people on the planet. And for the longest time improvements you refer to Moore's law, which is really the improvements of the price, performance of silicon over time that combined with architectural improvements was the thing that was pushing us forward. Well, what has happened is that the architectural refinements are more or less done. You're not going to get very much, you're not going to squeeze more blood out of that storm from the general purpose computer architecture. what has also happened over the last decade is that Moore's law which is essentially the doubling of the number of transistors on a chip has slowed down considerably and to the point where you're only getting maybe 10, 20% improvements every generation in speed of the transistor if that. And what's happening also is that the spacing between successive generations of technology is actually increasing from two, two and a half years to now three, maybe even four years. And this is because we are reaching some physical limits in CMOS. These limits are well-recognized. And we have to understand that these limits apply not just to general purposive use but they also apply to GPUs. Now, general purpose CPUs do one kind of competition they're really general and they can do lots and lots of different things. It is actually a very, very powerful engine. And then the problem is it's not powerful enough to handle all computations. So this is why you ended up having a different kind of a processor called the GPU which specializes in executing vector floating-point arithmetic operations much, much better than CPU maybe 20, 30, 40 times better. Well, GPUs have now been around for probably 15, 20 years mostly addressing graphics computations, but recently in the last decade or so they have been used heavily for AI and analytics computations. So now the question is, well, why do you need another specialized engine called the DPU? Well, I started down this journey about almost eight years ago and I recognize I was still at Juniper Networks which is another company that I founded. I recognize that in the data center as the workload changes to addressing more and more, larger and larger corpuses of data, number one and as people use scale-out as these standard technique for building applications, what happens is that the amount of east-west traffic increases greatly. And what happens is that you now have a new type of workload which is coming. And today probably 30% of the workload in a data center is what we call data-centric. I want to give you some examples of what is a data-centric workload. >> Well, I wonder if I could interrupt you for a second. >> Of course. >> Because I want those examples and I want you to tie it into the cloud 'cause that's kind of the topic that we're talking about today and how you see that evolving. I mean, it's a key question that we're trying to answer in this program. Of course, early cloud was about infrastructure, little compute, little storage, little networking and now we have to get to your point all this data in the cloud. And we're seeing, by the way the definition of cloud expand into this distributed or I think a term you use is disaggregated network of computers. So you're a technology visionary and I wonder how you see that evolving and then please work in your examples of that critical workload, that data-centric workload. >> Absolutely happy to do that. So if you look at the architecture of our cloud data centers the single most important invention was scale-out of identical or near identical servers all connected to a standard IP ethernet network. That's the architecture. Now, the building blocks of this architecture is ethernet switches which make up the network, IP ethernet switches. And then the server is all built using general purpose x86 CPUs with DRAM, with SSD, with hard drives all connected to inside the CPU. Now, the fact that you scale these server nodes as they're called out was very, very important in addressing the problem of how do you build very large scale infrastructure using general purpose compute. But this architecture did is it compute centric architecture and the reason it's a compute centric architecture is if you open this server node what you see is a connection to the network typically with a simple network interface card. And then you have CPUs which are in the middle of the action. Not only are the CPUs processing the application workload but they're processing all of the IO workload, what we call data-centric workload. And so when you connect SSDs, and hard drives, and GPUs, and everything to the CPU, as well as to the network you can now imagine the CPUs is doing two functions. It's running the applications but it's also playing traffic cop for the IO. So every IO has to go through the CPU and you're executing instructions typically in the operating system and you're interrupting the CPU many, many millions of times a second. Now, general purpose CPUs and the architecture CPUs was never designed to play traffic cop because the traffic cop function is a function that requires you to be interrupted very, very frequently. So it's critical that in this new architecture where there's a lot of data, a lot of these stress traffic the percentage of workload, which is data-centric has gone from maybe one to 2% to 30 to 40%. I'll give you some numbers which are absolutely stunning. If you go back to say 1987 and which is the year in which I bought my first personal computer the network was some 30 times slower than the CPU. The CPU is running at 15 megahertz, the network was running at three megabits per second. Or today the network runs at a 100 gigabits per second and the CPU clock speed of a single core is about three to 2.3 gigahertz. So you've seen that there's a 600X change in the ratio of IO to compute just the raw clock speed. Now, you can tell me that, hey, typical CPUs have lots, lots of cores, but even when you factor that in there's been close to two orders of magnitude change in the amount of IO to compute. There is no way to address that without changing the architecture and this is where the DPU comes in. And the DPU actually solves two fundamental problems in cloud data centers. And these are fundamental there's no escaping it. No amount of clever marketing is going to get around these problems. Problem number one is that in a compute centric cloud architecture the interactions between server nodes are very inefficient. That's number one, problem number one. Problem number two is that these data-centric computations and I'll give you those four examples. The network stack, the storage stack, the virtualization stack, and the security stack. Those four examples are executed very inefficiently by CPUs. Needless to say that if you try to execute these on GPUs you will run into the same problem probably even worse because GPUs are not good at executing these data-centric computations. So what we were looking to do at Fungible is to solve these two basic problems. And you don't solve them by just taking older architectures off the shelf and applying them to these problems because this is what people have been doing for the last 40 years. So what we did was we created this new microprocessor that we call DPU from ground up. It's a clean sheet design and it solves those two problems fundamentally. >> So I want to get into that. And I just want to stop you for a second and just ask you a basic question which is if I understand it correctly, if I just took the traditional scale out, if I scale out compute and storage you're saying I'm going to hit a diminishing returns. It's not only is it not going to scale linearly I'm going to get inefficiencies. And that's really the problem that you're solving. Is that correct? >> That is correct. And the workloads that we have today are very data-heavy. You take AI for example, you take analytics for example it's well known that for AI training the larger the corpus of relevant data that you're training on the better the result. So you can imagine where this is going to go. >> Right. >> Especially when people have figured out a formula that, hey the more data I collect I can use those insights to make money- >> Yeah, this is why I wanted to talk to you because the last 10 years we've been collecting all this data. Now, I want to bring in some other data that you actually shared with me beforehand. Some market trends that you guys cited in your research. And the first thing people said is they want to improve their infrastructure and they want to do that by moving to the cloud. And they also, there was a security angle there as well. That's a whole another topic we could discuss. The other stat that jumped out at me, there's 80% of the customers that you surveyed said there'll be augmenting their x86 CPU with alternative processing technology. So that's sort of, I know it's self-serving, but it's right on the conversation we're having. So I want to understand the architecture. >> Sure. >> And how you've approached this. You've clearly laid out this x86 is not going to solve this problem. And even GPUs are not going to solve the problem. >> They re not going to solve the problem. >> So help us understand the architecture and how you do solve this problem. >> I'll be very happy to. Remember I use this term traffic cop. I use this term very specifically because, first let me define what I mean by a data-centric computation because that's the essence of the problem we're solving. Remember I said two problems. One is we execute data-centric workloads at least an order of magnitude more efficiently than CPUs or GPUs, probably 30 times more efficient. And the second thing is that we allow nodes to interact with each other over the network much, much more efficiently. Okay, so let's keep those two things in mind. So first let's look at the data-centric piece. The data-centric piece for workload to qualify as being data-centric four things have to be true. First of all, it needs to come over the network in the form of packets. Well, this is all workloads so I'm not saying anything. Secondly, this workload is heavily multiplex in that there are many, many, many computations that are happening concurrently, thousands of them, okay? That's the number two. So a lot of multiplexing. Number three is that this workload is stateful. In other words you can't process back it's out of order. You have to do them in order because you're terminating network sessions. And the last one is that when you look at the actual computation the ratio of IO to arithmetic is medium to high. When you put all four of them together you actually have a data-centric workload, right? And this workload is terrible for general purpose CPUs. Not only the general purpose CPU is not executed properly the application that is running on the CPU also suffers because data center workloads are interfering workloads. So unless you designed specifically to them you're going to be in trouble. So what did we do? Well, what we did was our architecture consists of very, very heavily multi-threaded general purpose CPUs combined with very heavily threaded specific accelerators. I'll give you examples of some of those accelerators, DMA accelerators, then ratio coding accelerators, compression accelerators, crypto accelerators, compression accelerators. These are just some, and then look up accelerators. These are functions that if you do not specialize you're not going to execute them efficiently. But you cannot just put accelerators in there, these accelerators have to be multi-threaded to handle. We have something like a 1,000 different treads inside our DPU to address these many, many, many computations that are happening concurrently but handle them efficiently. Now, the thing that is very important to understand is that given the velocity of transistors I know that we have hundreds of billions of transistors on a chip, but the problem is that those transistors are used very inefficiently today if the architecture of a CPU or a GPU. What we have done is we've improved the efficiency of those transistors by 30 times, okay? >> So you can use a real estate much more effectively? >> Much more effectively because we were not trying to solve a general purpose computing problem. Because if you do that we're going to end up in the same bucket where general purpose CPUs are today. We were trying to solve a specific problem of data-centric computations and of improving the note to note efficiency. So let me go to point number two because that's equally important. Because in a scalar or architecture the whole idea is that I have many, many notes and they're connected over a high performance network. It might be shocking for your listeners to hear that these networks today run at a utilization of no more than 20 to 25%. Question is why? Well, the reason is that if I tried to run them faster than that you start to get back at drops because there are some fundamental problems caused by congestion on the network which are unsolved as we speak today. There are only one solution which is to use TCP. Well, TCP is a well-known, is part of the TCP IP suite. TCP was never designed to handle the latencies and speeds inside data center. It's a wonderful protocol but it was invented 43 years ago now. >> Yeah, very reliable and tested and proven. It's got a good track record but you're right. >> Very good track record, unfortunately eats a lot of CPU cycles. So if you take the idea behind TCP and you say, okay, what's the essence of TCP? How would you apply it to the data center? That's what we've done with what we call FCP which is a fabric control protocol, which we intend to open. We intend to publish the standards and make it open. And when you do that and you embed FCP in hardware on top of this standard IP ethernet network you end up with the ability to run at very large-scale networks where the utilization of the network is 90 to 95%, not 20 to 25%. >> Wow, okay. >> And you end up with solving problems of congestion at the same time. Now, why is this important today? That's all geek speak so far. The reason this stuff is important is that it such a network allows you to disaggregate, pull and then virtualize the most important and expensive resources in the data center. What are those? It's computer on one side, storage on the other side. And increasingly even things like DRAM wants to be disaggregated. And well, if I put everything inside a general purpose server the problem is that those resources get stranded because they're stuck behind a CPU. Well, once you disaggregate those resources and we're saying hyper disaggregate meaning the hyper and the hyper disaggregate simply means that you can disaggregate almost all the resources. >> And then you going to reaggregate them, right? I mean, that's obviously- >> Exactly and the network is the key in helping. >> Okay. >> So the reason the company is called Fungible is because we are able to disaggregate, virtualize and then pull those resources. And you can get for so scale-out companies the large AWS, Google, et cetera they have been doing this aggregation tooling for some time but because they've been using a compute centric architecture their disaggregation is not nearly as efficient as we can make. And they're off by about a factor of three. When you look at enterprise companies they are off by another factor of four because the utilization of enterprise is typically around 8% of overall infrastructure. The utilization in the cloud for AWS, and GCP, and Microsoft is closer to 35 to 40%. So there is a factor of almost four to eight which you can gain by dis-aggregating and pulling. >> Okay, so I want to interrupt you again. So these hyperscalers are smart. They have a lot of engineers and we've seen them. Yeah, you're right they're using a lot of general purpose but we've seen them make moves toward GPUs and embrace things like Arm. So I know you can't name names, but you would think that this is with all the data that's in the cloud, again, our topic today. You would think the hyperscalers are all over this. >> Well, the hyperscalers recognized here that the problems that we have articulated are important ones and they're trying to solve them with the resources that they have and all the clever people that they have. So these are recognized problems. However, please note that each of these hyperscalers has their own legacy now. They've been around for 10, 15 years. And so they're not in a position to all of a sudden turn on a dime. This is what happens to all companies at some point. >> They have technical debt, you mean? (laughs) >> I'm not going to say they have technical debt, but they have a certain way of doing things and they are in love with the compute centric way of doing things. And eventually it will be understood that you need a third element called the DPU to address these problems. Now, of course, you've heard the term SmartNIC. >> Yeah, right. >> Or your listeners must've heard that term. Well, a SmartNIC is not a DPU. What a SmartNIC is, is simply taking general purpose ARM cores, putting the network interface and a PCI interface and integrating them all on the same chip and separating them from the CPU. So this does solve a problem. It solves the problem of the data center workload interfering with the application workload, good job, but it does not address the architectural problem of how to execute data center workloads efficiently. >> Yeah, so it reminds me of, I understand what you're saying I was going to ask you about SmartNICs. It's almost like a bridge or a band-aid. >> Band-aid? >> It almost reminds me of throwing a high flash storage on a disc system that was designed for spinning disc. Gave you something but it doesn't solve the fundamental problem. I don't know if it's a valid analogy but we've seen this in computing for a longtime. >> Yeah, this analogy is close because okay, so let's take a hyperscaler X, okay? We won't name names. You find that half my CPUs are crippling their thumbs because they're executing this data-centric workload. Well, what are you going to do? All your code is written in C++ on x86. Well, the easiest thing to do is to separate the cores that run this workload. Put it on a different let's say we use Arm simply because x86 licenses are not available to people to build their own CPUs so Arm was available. So they put a bunch of Arm cores, they stick a PCI express and a network interface and you bought that code from x86 to Arm. Not difficult to do but and it does you results. And by the way if for example this hyperscaler X, shall we called them, if they're able to remove 20% of the workload from general purpose CPUs that's worth billions of dollars. So of course, you're going to do that. It requires relatively little innovation other than to port code from one place to another place. >> Pradeep, that's what I'm saying. I mean, I would think again, the hyperscalers why can't they just do some work and do some engineering and then give you a call and say, okay, we're going to attack these workloads together. That's similar to how they brought in GPUs. And you're right it's worth billions of dollars. You could see when the hyperscalers Microsoft, and Azure, and AWS bolt announced, I think they depreciated servers now instead of four years it's five years. And it dropped like a billion dollars to their bottom line. But why not just work directly with you guys? I mean, let's see the logical play. >> Some of them are working with us. So that's not to say that they're not working with us. So all of the hyperscalers they recognize that the technology that we're building is a fundamental that we have something really special and moreover it's fully programmable. So the whole trick is you can actually build a lump of hardware that is fixed function. But the difficulty is that in the place where the DPU would sit which is on the boundary of a server and the network, is literally on that boundary, that place the functionality needs to be programmable. And so the whole trick is how do you come up with an architecture where the functionality is programmable but it is also very high speed for this particular set of applications. So the analogy with GPUs is nearly perfect because GPUs and particularly Nvidia implemented or they invented CUDA which is the programming language for GPUs. And it made them easy to use, made it fully programmable without compromising performance. Well, this is what we're doing with DPUs. We've invented a new architecture, we've made them very easy to program. And they're these workloads, not workloads, computation that I talked about which is security, virtualization, storage and then network. Those four are quintessential examples of data center workloads and they're not going away. In fact, they're becoming more, and more, and more important over time. >> I'm very excited for you guys, I think, and really appreciate Pradeep, we have your back because I really want to get into some of the secret sauce. You talked about these accelerators, eraser code and crypto accelerators. But I want to understand that. I know there's NBMe in here, there's a lot of hardware and software and intellectual property, but we're seeing this notion of programmable infrastructure extending now into this domain, this build-out of this, I like this term disaggregated, massive disaggregated network. >> Hyper disaggregated. >> It's so hyper disaggregated even better. And I would say this and then I got to go. But what got us here the last decade is not the same as what's going to take us through the next decade. >> That's correct. >> Pradeep, thanks so much for coming on theCUBE. It's a great conversation. >> Thank-you for having me it's really a pleasure to speak with you and get the message of Fungible out there. >> Yeah, I promise we'll have you back. And keep it right there everybody we've got more great content coming your way on theCUBE on cloud. This is Dave Vellante. Stay right there. >> Thank-you, Dave.

>>artificial intelligence. The words are full of possibility. Yet to many it may seem complex, expensive and hard to know where to get started. How do you make AI really for your business? At Dell Technologies, we see AI enhancing business, enriching lives and improving the world. Dell Technologies is dedicated to making AI easy, so more people can use it to make a real difference. So you can adopt and run AI anywhere with your current skill. Sets with AI Solutions powered by power edge servers and made portable across hybrid multi clouds with VM ware. Plus solved I O bottlenecks with breakthrough performance delivered by Dell EMC Ready solutions for HPC storage and Data Accelerator. And enjoy automated, effortless management with open manage systems management so you can keep business insights flowing across a multi cloud environment. With an AI portfolio that spans from workstations to supercomputers, Dell Technologies can help you get started with AI easily and grow seamlessly. AI has the potential to profoundly change our lives with Dell Technologies. AI is easy to adopt, easy to manage and easy to scale. And there's nothing artificial about that. Yeah, yeah, from >>the Cube Studios in Palo Alto and Boston >>connecting with >>thought leaders all around the world. This is a cube conversation. Hi, I'm Stew Minimum. And welcome to this special launch with our friends at Dell Technologies. We're gonna be talking about AI and the reality of making artificial intelligence real happy to welcome to the program. Two of our Cube alumni Rob, depending 90. He's the senior vice president of server product management and very Pellegrino vice president, data centric workloads and solutions in high performance computing, both with Dell Technologies. Thank you both for joining thanks to you. So you know, is the industry we watch? You know, the AI has been this huge buzz word, but one of things I've actually liked about one of the differences about what I see when I listen to the vendor community talking about AI versus what I saw too much in the big data world is you know, it used to be, you know Oh, there was the opportunity. And data is so important. Yes, that's really But it was. It was a very wonky conversation. And the promise and the translation of what has been to the real world didn't necessarily always connect and We saw many of the big data solutions, you know, failed over time with AI on. And I've seen this in meetings from Dell talking about, you know, the business outcomes in general overall in i t. But you know how ai is helping make things real. So maybe we can start there for another product announcements and things we're gonna get into. But Robbie Interior talk to us a little bit about you know, the customers that you've been seeing in the impact that AI is having on their business. >>Sure, Teoh, I'll take us a job in it. A couple of things. For example, if you start looking at, uh, you know, the autonomous vehicles industry of the manufacturing industry where people are building better tools for anything they need to do on their manufacturing both. For example, uh, this is a good example of where that honors makers and stuff you've got Xeon ut It's actually a world war balcony. Now it is using our whole product suite right from the hardware and software to do multiple iterations off, ensuring that the software and the hardware come together pretty seamlessly and more importantly, ingesting, you know, probably tens of petabytes of data to ensure that we've got the right. They're training and gardens in place. So that's a great example of how we are helping some of our customers today in ensuring that we can really meet is really in terms of moving away from just a morning scenario in something that customers are able to use like today. >>Well, if I can have one more, Ah Yanai, one of our core and more partners than just customers in Italy in the energy sector have been been really, really driving innovation with us. We just deployed a pretty large 8000 accelerator cluster with them, which is the largest commercial cluster in the world. And where they're focusing on is the digital transformation and the development of energy sources. And it's really important not be an age. You know, the plan. It's not getting younger, and we have to be really careful about the type of energies that we utilize to do what we do every day on they put a lot of innovation. We've helped set up the right solution for them, and we'll talk some more about what they've done with that cluster. Later, during our chat, but it is one of the example that is tangible with the appointment that is being used to help there. >>Great. Well, we love starting with some of the customer stories. Really glad we're gonna be able to share some of those, you know, actual here from some of the customers a little bit later in this launch. But, Robbie, you know, maybe give us a little bit as to what you're hearing from customers. You know, the overall climate in AI. You know, obviously you know, so many challenges facing, you know, people today. But you know, specifically around ai, what are some of the hurdles that they might need to overcome Be able to make ai. Really? >>I think the two important pieces I can choose to number one as much as we talk about AI machine learning. One of the biggest challenges that customers have today is ensuring that they have the right amount and the right quality of data to go out and do the analytics percent. Because if you don't do it, it's giggle garbage in garbage out. So the one of the biggest challenges our customers have today is ensuring that they have the most pristine data to go back on, and that takes quite a bit of an effort. Number two. A lot of times, I think one of the challenges they also have is having the right skill set to go out and have the execution phase of the AI pod. You know, work done. And I think those are the two big challenges we hear off. And that doesn't seem to be changing in the very near term, given the very fact that nothing Forbes recently had an article that said that less than 15% off, our customers probably are using AI machine learning today so that talks to the challenges and the opportunities ahead for me. All right, >>So, Ravi, give us the news. Tell us the updates from Dell Technologies how you're helping customers with AI today, >>going back to one of the challenges, as I mentioned, which is not having the right skin set. One of the things we are doing at Dell Technologies is making sure that we provide them not just the product but also the ready solutions that we're working with. For example, Tier and his team. We're also working on validated and things are called reference architectures. The whole idea behind this is we want to take the guesswork out for our customers and actually go ahead and destroying things that we have already tested to ensure that the integration is right. There's rightsizing attributes, so they know exactly the kind of a product that would pick up our not worry about me in time and the resources needed you get to that particular location. So those are probably the two of the biggest things we're doing to help our customers make the right decision and execute seamlessly and on time. >>Excellent. So teary, maybe give us a little bit of a broader look as to, you know, Dell's part participation in the overall ecosystem when it comes to what's happening in AI on and you know why is this a unique time for what's happening in the in the industry? >>Yeah, I mean, I think we all live it. I mean, I'm right here in my home, and I'm trying to ensure that the business continues to operate, and it's important to make sure that we're also there for our customers, right? The fight against covered 19 is eyes changing what's happening around the quarantines, etcetera. So Dell, as a participant not only in the AI the world that we live in on enabling AI is also a participant in all of the community's s. So we've recently joined the covered 19 High Performance Computing Consortium on. We also made a lot of resources available to researchers and scientists leveraging AI in order to make progress towards you're and potentially the vaccine against Corbyn. 19 examples are we have our own supercomputers in the lab here in Austin, Texas, and we've given access to some of our partners. T. Gen. Is one example. The beginning of our chat I mentioned and I So not only did they have barely deport the cluster with us earlier this year that could 19 started hitting, so they've done what's the right thing to do for community and humanity is they made the resource available to scientists in Europe on tack just down the road here, which had the largest I can't make supercomputer that we deployed with them to. Ai's doing exactly the same thing. So this is one of the real examples that are very timely, and it's it's it's happening right now we hadn't planned for it. A booth there with our customers, the other pieces. This is probably going to be a trend, but healthcare is going through and version of data you mentioned in the beginning. You're talking about 2.3000 exabytes, about 3000 times the content of the Library of Congress. It's incredible, and that data is useless. I mean, it's great we can We can put that on our great ice on storage, but you can also see it as an opportunity to get business value out of it. That's going to be we're a lot more resource is with AI so a lot happening here. That's that's really if I can get into more of the science of it because it's healthcare, because it's the industry we see now that our family members at the M. Ware, part of the Dell Technologies Portfolio, are getting even more relevance in the discussion. The industry is based on virtualization, and the M ware is the number one virtualization solution for the industry. So now we're trying to weave in the reality in the I T environment with the new nodes of AI and data science and HPC. So you will see the VM Ware just added kubernetes control plane. This fear Andi were leveraging that to have a very flexible environment On one side, we can do some data science on the other side. We can go back to running some enterprise class hardware class software on top of it. So this is is great. And we're capitalizing on it with validates solutions, validated design on. And I think that's going to be adding a lot of ah power in the hands of our customers and always based on their feedback. And they asked back, >>Yeah, I may ask you just to build on that interesting comment that you made on we're actually looking at very shortly will be talking about how we're gonna have the ability to, for example, read or V Sphere and Allah servers begin. That essentially means that we're going to cut down the time our customers need to go ahead and deploy on their sites. >>Yeah, excellent. Definitely been, you know, very strong feedback from the community. We did videos around some of the B sphere seven launch, you know, theory. You know, we actually had done an interview with you. Ah, while back at your big lab, Jeff Frick. Otto, See the supercomputers behind what you were doing. Maybe bring us in a little bit inside as who? You know, some of the new pieces that help enable AI. You know, it often gets lost on the industry. You know, it's like, Oh, yeah, well, we've got the best hardware to accelerate or enable these kind of workloads. So, you know, bring us in its But what, You know, the engineering solution sets that are helping toe make this a reality >>of today. Yeah, and truly still you've been there. You've seen the engineers in the lab, and that's more than AI being real. That that is double real because we spend a lot of time analyzing workloads customer needs. We have a lot of PhD engineers in there, and what we're working on right now is kind of the next wave of HPC enablement Azaz. We all know the consumption model or the way that we want to have access to resources is evolving from something that is directly in front of us. 1 to 1 ratio to when virtualization became more prevalent. We had a one to many ratio on genes historically have been allocated on a per user. Or sometimes it is study modified view to have more than one user GP. But with the addition of big confusion to the VM our portfolio and be treated not being part of these fear. We're building up a GPU as a service solutions through a VM ware validated design that we are launching, and that's gonna give them flexibility. And the key here is flexibility. We have the ability, as you know, with the VM Ware environment, to bring in also some security, some flexibility through moving the workloads. And let's be honest with some ties into cloud models on, we have our own set of partners. We all know that the big players in the industry to But that's all about flexibility and giving our customers what they need and what they expect in the world. But really, >>Yeah, Ravi, I guess that brings us to ah, you know, one of the key pieces we need to look at here is how do we manage across all of these environments? Uh, and you know, how does AI fit into this whole discussion between what Dell and VM ware doing things like v Sphere, you know, put pulling in new workloads >>stew, actually a couple of things. So there's really nothing artificial about the real intelligence that comes through with all that foolish intelligence we're working out. And so one of the crucial things I think we need to, you know, ensure that we talk about is it's not just about the fact that it's a problem. So here are our stories there, but I think the crucial thing is we're looking at it from an end to end perspective from everything from ensuring that we have direct workstations, right servers, the storage, making sure that is well protected and all the way to working with an ecosystem of software renders. So first and foremost, that's the whole integration piece, making sure they realized people system. But more importantly, it's also ensuring that we help our customers by taking the guess work out again. I can't emphasize the fact that there are customers who are looking at different aliens off entry, for example, somebody will be looking at an F G. A. Everybody looking at GP use. API is probably, as you know, are great because they're price points and normal. Or should I say that our needs our lot lesser than the GP use? But on the flip side, there's a need for them to have a set of folks who can actually program right. It is why it's called the no programming programmable gate arrays of Saas fee programmable. My point being in all this, it's important that we actually provide dried end to end perspective, making sure that we're able to show the integration, show the value and also provide the options, because it's really not a cookie cutter approach of where you can take a particular solution and think that it will put the needs of every single customer. He doesn't even happen in the same industry, for that matter. So the flexibility that we provide all the way to the services is truly our attempt. At Dell Technologies, you get the entire gamut of solutions available for the customer to go out and pick and choose what says their needs the best. >>Alright, well, Ravi interior Thank you so much for the update. So we're gonna turn it over to actually hear from some of your customers. Talk about the power of ai. You're from their viewpoint, how real these solutions are becoming. Love the plan words there about, you know, enabling really artificial intelligence. Thanks so much for joining after the customers looking forward to the VM Ware discussion, we want to >>put robots into the world's dullest, deadliest and dirtiest jobs. We think that if we can have machines doing the work that put people at risk than we can allow people to do better work. Dell Technologies is the foundation for a lot of the >>work that we've done here. Every single piece of software that we developed is simulated dozens >>or hundreds of thousands of times. And having reliable compute infrastructure is critical for this. Yeah, yeah, A lot of technology has >>matured to actually do something really useful that can be used by non >>experts. We try to predict one system fails. We try to predict the >>business impatience things into images. On the end of the day, it's that >>now we have machines that learn how to speak a language from from zero. Yeah, everything >>we do really, at Epsilon centered around data and our ability >>to get the right message to >>the right person at the right >>time. We apply machine learning and artificial intelligence. So in real time you can adjust those campaigns to ensure that you're getting the most optimized message theme. >>It is a joint venture between Well, cars on the Amir are your progress is automated driving on Advanced Driver Assistance Systems Centre is really based on safety on how we can actually make lives better for you. Typically gets warned on distracted in cars. If you can take those kind of situations away, it will bring the accidents down about 70 to 80%. So what I appreciate it with Dell Technologies is the overall solution that they have to live in being able to deliver the full package. That has been a major differentiator compared to your competitors. >>Yeah. Yeah, alright, welcome back to help us dig into this discussion and happy to welcome to the program Chris Facade. He is the senior vice president and general manager of the B sphere business and just Simon, chief technologist for the High performance computing group, both of them with VM ware. Gentlemen, thanks so much for joining. Thank >>you for having us. >>All right, Krish. When vm Ware made the bit fusion acquisition. Everybody was looking the You know what this will do for space Force? GPU is we're talking about things like AI and ML. So bring us up to speed. As to you know, the news today is the what being worth doing with fusion. Yeah. >>Today we have a big announcement. I'm excited to announce that, you know, we're taking the next big step in the AI ML and more than application strategy. With the launch off bit fusion, we're just now being fully integrated with VCF. They're in black home, and we'll be releasing this very shortly to the market. As you said when we acquire institution A year ago, we had a showcase that's capable days as part of the animal event. And at that time we laid out a strategy that part of our institution as the cornerstone off our capabilities in the black home in the Iot space. Since then, we have had many customers take a look at the technology and we have had feedback from them as well as from partners and analysts. And the feedback has been tremendous. >>Excellent. Well, Chris, what does this then mean for customers? You know What's the value proposition that diffusion brings the VC? Yeah, >>if you look at our customers, they are in the midst of a big ah journey in digital transformation. And basically, what that means is customers are building a ton of applications and most of those applications some kind of data analytics or machine learning embedded in it. And what this is doing is that in the harbor and infrastructure industry, this is driving a lot of innovation. So you see the advent off a lot off specialized? Absolutely. There's custom a six FPs. And of course, the views being used to accelerate the special algorithms that these AI ml type applications need. And unfortunately, customer environment. Most of these specialized accelerators uh um bare metal kind of set up, but they're not taking advantage off optimization and everything that it brings to that. Also, with fusion launched today, we are essentially doing the accelerator space. What we need to compute several years ago and that is essentially bringing organization to the accelerators. But we take it one step further, which is, you know, we use the customers the ability to pull these accelerators and essentially going to be couple it from the server so you can have a pool of these accelerators sitting in the network. And customers are able to then target their workloads and share the accelerators get better utilization by a lot of past improvements and, in essence, have a smaller pool that they can use for a whole bunch of different applications across the enterprise. That is a huge angle for our customers. And that's the tremendous positive feedback that we get getting both from customers as well. >>Excellent. Well, I'm glad we've got Josh here to dig into some of the thesis before we get to you. They got Chris. Uh, part of this announcement is the partnership of VM Ware in Dell. So tell us about what the partnership is in the solutions for for this long. Yeah. >>We have been working with the Dell in the in the AI and ML space for a long time. We have ah, good partnership there. This just takes the partnership to the next level and we will have ah, execution solution. Support in some of the key. I am el targeted words like the sea for 1 40 the r 7 40 Those are the centers that would be partnering with them on and providing solutions. >>Excellent. Eso John. You know, we've watched for a long time. You know, various technologies. Oh, it's not a fit for virtualized environment. And then, you know, VM Ware does does what it does. Make sure you know, performance is there. And make sure all the options there bring us inside a little bit. You know what this solution means for leveraging GPS? Yeah. So actually, before I before us, answer that question. Let me say that the the fusion acquisition and the diffusion technology fits into a larger strategy at VM Ware around AI and ML. That I think matches pretty nicely the overall Dell strategy as well, in the sense that we are really focused on delivering AI ml capabilities or the ability for our customers to run their am ai and ml workloads from edge before the cloud. And that means running it on CPU or running it on hardware accelerators like like G fuse. Whatever is really required by the customer in this specific case, we're quite excited about using technology as it really allows us. As Chris was describing to extend our capabilities especially in the deep learning space where GPU accelerators are critically important. And so what this technology really brings to the table is the ability to, as Chris was outlining, to pull those resources those hardware resource together and then allow organizations to drive up the utilization of those GP Resource is through that pooling and also increase the degree of sharing that we support that supported for the customer. Okay, Jeff, take us in a little bit further as how you know the mechanisms of diffusion work. Sure, Yeah, that's a great question. So think of it this way. There there is a client component that we're using a server component. The server component is running on a machine that actually has the physical GPU is installed in it. The client machine, which is running the bit fusion client software, is where the user of the data scientist is actually running their machine machine learning application. But there's no GPU actually in that host. And what is happening with fusion technology is that it is essentially intercepting the cuda calls that are being made by that machine learning app, patience and promoting those protocols over to the bit fusion server and then injecting them into the local GPU on the server. So it's actually, you know, we call it into a position in the ability that remote these protocols, but it's actually much more sophisticated than that. There are a lot of underlying capabilities that are being deployed in terms of optimization who takes maximum advantage of the the networking link that sits between the client machine and the server machine. But given all of that, once we've done it with diffusion, it's now possible for the data scientist. Either consume multiple GP use for single GPU use or even fractional defuse across that Internet using the using technology. Okay, maybe it would help illustrate some of these technologies. If you got a couple of customers, Sure, so one example would be a retail customer. I'm thinking of who is. Actually it's ah, grocery chain. That is the flowing, ah, large number of video cameras into their to their stores in order to do things like, um, watch for pilfering, uh, identify when storage store shelves could be restocked and even looking for cases where, for example, maybe a customer has fallen down in denial on someone needs to go and help those multiple video streams and then multiple app patients that are being run that part are consuming the data from those video streams and doing analytics and ml on them would be perfectly suited for this type of environment where you would like to be ableto have these multiple independent applications running but having them be able to efficiently share the hardware resources of the GP use. Another example would be retailers who are deploying ml Howard Check out registers who helped reduce fraud customers who are buying, buying things with, uh, fake barcodes, for example. So in that case, you would not necessarily want to employ a single dedicated GPU for every single check out line. Instead, what you would prefer to do is have a full set of resource. Is that each inference operation that's occurring within each one of those check out lines could then consume collectively. That would be two examples of the use of this kind of pull in technology. Okay, great. So, Josh, a lot last question for you is this technology is this only for use and anything else. You can give us a little bit of a look forward to as to what we should be expecting from the big fusion technology. Yeah. So currently, the target is specifically NVIDIA GPU use with Cuda. The team, actually even prior to acquisition, had done some work on enablement of PJs and also had done some work on open CL, which is more open standard for a device that so what you will see over time is an expansion of the diffusion capabilities to embrace devices like PJs. The domain specific a six that first was referring to earlier will roll out over time. But we are starting with the NVIDIA GPU, which totally makes sense, since that is the primary hardware acceleration and for deep learning currently excellent. Well, John and Chris, thank you so much for the updates to the audience. If you're watching this live, please throwing the crowd chat and ask your questions. This faith, If you're watching this on demand, you can also go to crowdchat dot net slash make ai really to be able to see the conversation that we had. Thanks so much for joining. >>Thank you very much. >>Thank you. Managing your data center requires around the clock. Attention Dell, EMC open manage mobile enables I t administrators to monitor data center issues and respond rapidly toe unexpected events anytime, anywhere. Open Manage Mobile provides a wealth of features within a comprehensive user interface, including >>server configuration, push notifications, remote desktop augmented reality and more. The latest release features an updated Our interface Power and Thermal Policy Review. Emergency Power Reduction, an internal storage monitoring download Open Manage Mobile today.

>> Narrator: Live from Las Vegas, it's theCUBE covering AWS re:Invent 2017 presented by AWS, Intel, and our ecosystem of partners. >> We are life back here in Las Vegas at the Sands Expo as we continue our coverage here on theCUBE of re:Invent, AWS here on the fourth day of what has been a very successful show. I still hear a lot of buzz, a lot of activity on the show floor. It certainly indicative of what happened here in terms of bringing this community together in a very positive way. I'm with Justin Warren, I'm John Walls. We go from Justin to Dustin, Dustin Kirkland, who is the vice president of product development for Ubuntu on the Canonical. It's good to see you again. >> Likewise, John. >> I should let the two of you probably chat about Australia. We heard these great diving stories about your adventures, your home, your native country. >> Yep. >> Maybe afterwards will get a little photos, travel thing going on. >> Yeah that's right. (laughing) >> All right, 17 years you have been diving. Were going to have to get into that a little bit later on. First off, let's talk about Ubuntu, and maybe the footprint within AWS. Maybe not only what brings you here, but what gets you there? What are you doing there? >> First of all, this is a fantastic conference. Hundreds of these organizations here are involved in Ubuntu, using Ubuntu in AWS and taking advantage of open source, using it for lots of scale out services. To date this year in 2017, over 125 million instances of Ubuntu have launched in AWS alone just this year, and the year is not even over yet. We see anything from media entertainment. Netflix is here. I spent some time with them. One of Netflix's performance engineers gave a talk yesterday about how Netflix tunes their Ubuntu instances in Amazon to the tune of 100,000 instances of Ubuntu running in Amazon to deliver the Netflix experience that I'm sure all of us have. >> John: 100,000? >> Yeah. >> That's amazing. >> It's crazy, yeah. >> I'm a big fan of Ubuntu because I am a mad person. I've been running it as my primary desktop for something like 10 years. >> All right! >> I run it on a laptop. >> Okay. >> I love it, it's great. >> Good. >> People use Ubuntu all the time but it's like it just became the de facto, it seems like overnight of pretty much, hey, if you want to run Linux in cloud, you just spin up in Ubuntu. Just run it up, so what is it about Ubuntu itself, where are you taking the product for people who are using it in cloud? We are hearing a lot about all these different services, and we are hearing about serverless, so how does Ubuntu fit into that AWS world? >> That's a great question. First of all, it's not overnight. We have been doing this since 2004, so we are going on 14 years of building the thing that is Ubuntu. We brought Ubuntu into Amazon in about 2008, which is right when I got involved at Canonical. I was working on Ubuntu before that, but working for Canonical, and that was relatively early in the entire Amazon adventure. You said Ubuntu on the desktop. That's certainly where Ubuntu got its start, but it was Amazon that really busted Ubuntu out into the server space, and so now as you said, if you are starting a new company or a new technology, you almost by default start on Ubuntu. Now where are we taking that? Here we are talking about cloud, but the other half of cloud is the edge. The edge being embedded devices, embedded IOT connected devices. The thing about every IOT device, the I in IOT is Internet. The connected part of a connected device means it has to be connected to something, and what is going to be connected to? The cloud. Every smart autonomous driving vehicle, every oil rig out in West Texas, every airplane, every boat, every ship, every place where you are going to find compute in these next couple of years as we move into the 5G revolution, are connected to services on the backend, the majority of those hosted in Amazon, and the majority of those running Ubuntu. >> When you talk about IOT though, what kind of challenges that that bring into your world? Because you are talking about this, I mean, I can't even think about the order of growth. >> Yeah, billions, literally billions. >> It's just massive connectivity, and in a mobile environment, throw that on top of that, so what does that do for you then in terms of what you are looking at down the road and the kind of capabilities that you have got to build in? >> Security, I mean it starts with security. When we think about devices in our homes accompanying our kids to school, devices that are inside of buses and hospitals, it's all about security, and security is first and foremost. We put a lot of effort into securing Ubuntu. We've created new features as part of where we are taking Ubuntu. Many of the new features we created around Ubuntu are about updates, security updates, being able to make those updates active without rebooting the system, so zero downtime kernel updates is something we call a live patch service which we deliver in Amazon for Ubuntu Amazon users. Extended security maintenance. Security for Ubuntu after end-of-life, say you said you've been using Ubuntu for a long time. Each Ubuntu release has basically a five year lifecycle but some enterprises actually need to run Ubuntu for much longer than five years, and for those enterprises, we provide security updates after the end of life, after that five-year end-of-life, and in many cases, that helps them bridge that gap until the next release of Ubuntu. We've also worked with IBM and the US government to provide FIPS certified cryptography for Ubuntu also available in Amazon, so the Department of Defense contractors, many federal contractors are required to use FIPS bits, and this actually allows them to bring their Ubuntu usage into compliance with what's required for government regulation. >> I'm so glad that you went from IOT to security in, like, a nanosecond. That was going to be my next question. >> Well that is the only answer to that. It's the only right answer to that question in my mind. >> Not enough companies put that much focus on security and you follow it up with specific concrete examples of things that are going to work. The live kernel patching without rebooting things so that you can have the-- I mean, services in the cloud, it has to be always on. You can't take a maintenance window when something is down four hours or a weekend. That's just not acceptable in the cloud world anymore. >> Especially in the retail season. We are just now getting into the retail-- you know, Black Friday was last week, Cyber Monday this week, and the roll up all the way to Christmas, Canonical works with quite a bit with the largest retailers in the world, Walmart, Best Buy, other ones like that, and downtime is just not acceptable. At the same time, security is of the utmost importance. When you are taking people's credit cards, you are placing large amounts of money on the line every time these transactions take place. Security has to be utmost, and being able to do that without impacting the downtime. Downtime is seriously hundreds of thousands of dollars per second on some of these sites during the major holiday rush. >> You just mentioned some of the big names you're working with, so what kind of assurance can you give them that you can sleep with both eyes closed? You don't have to keep that one eye open. Don't worry, if there is an incident of some kind, we are going to take care of it. If you have a problem, rest assured, we are going to be there because, as you pointed out, with the volume involved and the issues of security infiltrations being what they are today, it's hard to rest. >> Right, the return on value, the return on investment of the live patch is easily apparent. Any time someone does the math and realizes, "Let's actually look at how much it costs us "to reboot a data center, "or how much it costs us to wake up the dev ops team "on a Saturday and have them work through a weekend "to roll out this update," whereas with the live patch, at least the patch is applied in milliseconds without downtime, and then we get back on Monday and we rollout a comprehensive plan as to what do we actually need to do about this going forward? That is for the kernel side of things. The other half of it is the user space side of Ubuntu. In the user space side of Ubuntu, we continue to make Ubuntu smaller, smaller and smaller. That might be one of the reasons you are attracted to Ubuntu on your laptop early on is because we really did a good job of making a Linux that was consumable, usable, but also very small and secure. We've actually taken that same approach in the cloud where we continue to minimize the footprint of Ubuntu. That has a security impact in that if you simply leave software out of the default image you are not vulnerable to problems in that software, so we've heard that quite a bit around the container space, the work we do in the container space. We will be in Austin next week for CUBE Con talking about containers. I will save the container talk for next week, but minimizing Ubuntu is an important of that security story as well. >> All right, just reducing that attack surface is fabulous. It also means that when you are actually doing this patching, it's less things to patch, there are less opportunities for downtime, there are less things that can go wrong and cause outages in the rest of the place. Simple is better. >> Dustin : That's right, that's exactly right. >> What else are you doing? We've talked about security a lot here. What are some of the other things that you are doing around supporting the services that we are hearing here at AWS? We've heard a lot about things like serverless. We've heard a lot about high performance computing. We've had guests here on theCUBE talking about what they are doing around data analytics and machine learning, so maybe you could give us a little bit of color around that. >> Let's start with that last point, machine learning and data analytics. We work very closely with both Amazon and Nvidia to enable the GPGPUs, the general-purpose graphics processing units that Nvidia produces which go into servers and Amazon exposes in the some of the largest machine learning type instances. Those instances powered by Ubuntu are working directly with that GPU out of the box by default, and that's something that we've worked very hard on and closely with both Amazon and Nvidia to make sure the Ubuntu experience when using the graphics accelerated instance types just work, and just work out of the box. Those are important for the machine learning and the data analytics because many of those algorithms take advantage of CUDA. CUDA is a set of libraries that allows developers to write applications that scale very, very wide across the CUDA cores, so a given Nvidia GPU may have several thousand Nvidia CUDA cores. Each of those are running little process bits and then the answers are summed up, basically, at the end. That is at the heart of everything that's happening in the AI space, and that I will tie that back to our IOT space in that for those connected devices where memory discs, CPU, power are very constrained, part of the important part of that connection is that they are talking to a cloud that has essentially infinite resources, infinite data at its disposal, enough memory to load those entire data sets and crunch those. The fastest CPUs and the fastest GPUs that can crunch that data, so to really take that and make that real, that's exactly what's powering every autonomous vehicle in the world, essentially, is a little unit inside of the car, a majority of those autonomous vehicles are running Ubuntu on the auto driving unit. Tesla, Google, Uber, all running Ubuntu inside of that car. Every one of those cars are talking to a cloud. Some clouds are Amazon, other, in Google's case, certainly the Google cloud, but they are talking to GPU Nvidia powered AI instances that are crunching all the data that these Tesla cars and GM, and Ford cars are sending to the cloud and constantly making the inference engine better. What gets downloaded to the car is an updated inference engine. That inference engine comes down to the car, and that's how that car decides is it safe to change lanes right now or not? That answer has to be determined inside of the car, not in the cloud, but you can understand why data training and modeling in the cloud is powerful, far more powerful than what can happen inside of a little CPU in a the car. >> John: Let's just keep it on the right side of the road. Can we do that? (laughing) >> Well, you need to talk to this gentleman about that. >> Yeah, I drive on the left side. (laughing) >> Or the left side of the road. >> Don't cross the streams. >> How about the correct side of the road? >> Don't cross the streams. >> Dustin thanks for the time. >> Thank you, John. >> Always good seeing you. >> Likewise. >> And we'll see you next week as well. Down in your hometown, a little barbecue in Austin. >> That sounds good. >> All right, back with more here at re:Invent. We are live in Las Vegas back with more on theCUBE in just a bit.