(bright upbeat music) >> Hello, everyone. Welcome to SiliconANGLE News breaking story here. Amazon Web Services expanding their relationship with Hugging Face, breaking news here on SiliconANGLE. I'm John Furrier, SiliconANGLE reporter, founder, and also co-host of theCUBE. And I have with me, Swami, from Amazon Web Services, vice president of database, analytics, machine learning with AWS. Swami, great to have you on for this breaking news segment on AWS's big news. Thanks for coming on and taking the time. >> Hey, John, pleasure to be here. >> You know- >> Looking forward to it. >> We've had many conversations on theCUBE over the years, we've watched Amazon really move fast into the large data modeling, SageMaker became a very smashing success, obviously you've been on this for a while. Now with ChatGPT OpenAI, a lot of buzz going mainstream, takes it from behind the curtain inside the ropes, if you will, in the industry to a mainstream. And so this is a big moment, I think, in the industry, I want to get your perspective, because your news with Hugging Face, I think is another tell sign that we're about to tip over into a new accelerated growth around making AI now application aware, application centric, more programmable, more API access. What's the big news about, with AWS Hugging Face, you know, what's going on with this announcement? >> Yeah. First of all, they're very excited to announce our expanded collaboration with Hugging Face, because with this partnership, our goal, as you all know, I mean, Hugging Face, I consider them like the GitHub for machine learning. And with this partnership, Hugging Face and AWS, we'll be able to democratize AI for a broad range of developers, not just specific deep AI startups. And now with this, we can accelerate the training, fine tuning and deployment of these large language models, and vision models from Hugging Face in the cloud. And the broader context, when you step back and see what customer problem we are trying to solve with this announcement, essentially if you see these foundational models, are used to now create like a huge number of applications, suggest like tech summarization, question answering, or search image generation, creative, other things. And these are all stuff we are seeing in the likes of these ChatGPT style applications. But there is a broad range of enterprise use cases that we don't even talk about. And it's because these kind of transformative, generative AI capabilities and models are not available to, I mean, millions of developers. And because either training these elements from scratch can be very expensive or time consuming and need deep expertise, or more importantly, they don't need these generic models, they need them to be fine tuned for the specific use cases. And one of the biggest complaints we hear is that these models, when they try to use it for real production use cases, they are incredibly expensive to train and incredibly expensive to run inference on, to use it at a production scale. So, and unlike web search style applications, where the margins can be really huge, here in production use cases and enterprises, you want efficiency at scale. That's where Hugging Face and AWS share our mission. And by integrating with Trainium and Inferentia, we're able to handle the cost efficient training and inference at scale, I'll deep dive on it. And by teaming up on the SageMaker front, now the time it takes to build these models and fine tune them is also coming down. So that's what makes this partnership very unique as well. So I'm very excited. >> I want to get into the time savings and the cost savings as well on the training and inference, it's a huge issue, but before we get into that, just how long have you guys been working with Hugging Face? I know there's a previous relationship, this is an expansion of that relationship, can you comment on what's different about what's happened before and then now? >> Yeah. So, Hugging Face, we have had a great relationship in the past few years as well, where they have actually made their models available to run on AWS, you know, fashion. Even in fact, their Bloom Project was something many of our customers even used. Bloom Project, for context, is their open source project which builds a GPT-3 style model. And now with this expanded collaboration, now Hugging Face selected AWS for that next generation office generative AI model, building on their highly successful Bloom Project as well. And the nice thing is, now, by direct integration with Trainium and Inferentia, where you get cost savings in a really significant way, now, for instance, Trn1 can provide up to 50% cost to train savings, and Inferentia can deliver up to 60% better costs, and four x more higher throughput than (indistinct). Now, these models, especially as they train that next generation generative AI models, it is going to be, not only more accessible to all the developers, who use it in open, so it'll be a lot cheaper as well. And that's what makes this moment really exciting, because we can't democratize AI unless we make it broadly accessible and cost efficient and easy to program and use as well. >> Yeah. >> So very exciting. >> I'll get into the SageMaker and CodeWhisperer angle in a second, but you hit on some good points there. One, accessibility, which is, I call the democratization, which is getting this in the hands of developers, and/or AI to develop, we'll get into that in a second. So, access to coding and Git reasoning is a whole nother wave. But the three things I know you've been working on, I want to put in the buckets here and comment, one, I know you've, over the years, been working on saving time to train, that's a big point, you mentioned some of those stats, also cost, 'cause now cost is an equation on, you know, bundling whether you're uncoupling with hardware and software, that's a big issue. Where do I find the GPUs? Where's the horsepower cost? And then also sustainability. You've mentioned that in the past, is there a sustainability angle here? Can you talk about those three things, time, cost, and sustainability? >> Certainly. So if you look at it from the AWS perspective, we have been supporting customers doing machine learning for the past years. Just for broader context, Amazon has been doing ML the past two decades right from the early days of ML powered recommendation to actually also supporting all kinds of generative AI applications. If you look at even generative AI application within Amazon, Amazon search, when you go search for a product and so forth, we have a team called MFi within Amazon search that helps bring these large language models into creating highly accurate search results. And these are created with models, really large models with tens of billions of parameters, scales to thousands of training jobs every month and trained on large model of hardware. And this is an example of a really good large language foundation model application running at production scale, and also, of course, Alexa, which uses a large generator model as well. And they actually even had a research paper that showed that they are more, and do better in accuracy than other systems like GPT-3 and whatnot. So, and we also touched on things like CodeWhisperer, which uses generative AI to improve developer productivity, but in a responsible manner, because 40% of some of the studies show 40% of this generated code had serious security flaws in it. This is where we didn't just do generative AI, we combined with automated reasoning capabilities, which is a very, very useful technique to identify these issues and couple them so that it produces highly secure code as well. Now, all these learnings taught us few things, and which is what you put in these three buckets. And yeah, like more than 100,000 customers using ML and AI services, including leading startups in the generative AI space, like stability AI, AI21 Labs, or Hugging Face, or even Alexa, for that matter. They care about, I put them in three dimension, one is around cost, which we touched on with Trainium and Inferentia, where we actually, the Trainium, you provide to 50% better cost savings, but the other aspect is, Trainium is a lot more power efficient as well compared to traditional one. And Inferentia is also better in terms of throughput, when it comes to what it is capable of. Like it is able to deliver up to three x higher compute performance and four x higher throughput, compared to it's previous generation, and it is extremely cost efficient and power efficient as well. >> Well. >> Now, the second element that really is important is in a day, developers deeply value the time it takes to build these models, and they don't want to build models from scratch. And this is where SageMaker, which is, even going to Kaggle uses, this is what it is, number one, enterprise ML platform. What it did to traditional machine learning, where tens of thousands of customers use StageMaker today, including the ones I mentioned, is that what used to take like months to build these models have dropped down to now a matter of days, if not less. Now, a generative AI, the cost of building these models, if you look at the landscape, the model parameter size had jumped by more than thousand X in the past three years, thousand x. And that means the training is like a really big distributed systems problem. How do you actually scale these model training? How do you actually ensure that you utilize these efficiently? Because these machines are very expensive, let alone they consume a lot of power. So, this is where SageMaker capability to build, automatically train, tune, and deploy models really concern this, especially with this distributor training infrastructure, and those are some of the reasons why some of the leading generative AI startups are actually leveraging it, because they do not want a giant infrastructure team, which is constantly tuning and fine tuning, and keeping these clusters alive. >> It sounds like a lot like what startups are doing with the cloud early days, no data center, you move to the cloud. So, this is the trend we're seeing, right? You guys are making it easier for developers with Hugging Face, I get that. I love that GitHub for machine learning, large language models are complex and expensive to build, but not anymore, you got Trainium and Inferentia, developers can get faster time to value, but then you got the transformers data sets, token libraries, all that optimized for generator. This is a perfect storm for startups. Jon Turow, a former AWS person, who used to work, I think for you, is now a VC at Madrona Venture, he and I were talking about the generator AI landscape, it's exploding with startups. Every alpha entrepreneur out there is seeing this as the next frontier, that's the 20 mile stairs, next 10 years is going to be huge. What is the big thing that's happened? 'Cause some people were saying, the founder of Yquem said, "Oh, the start ups won't be real, because they don't all have AI experience." John Markoff, former New York Times writer told me that, AI, there's so much work done, this is going to explode, accelerate really fast, because it's almost like it's been waiting for this moment. What's your reaction? >> I actually think there is going to be an explosion of startups, not because they need to be AI startups, but now finally AI is really accessible or going to be accessible, so that they can create remarkable applications, either for enterprises or for disrupting actually how customer service is being done or how creative tools are being built. And I mean, this is going to change in many ways. When we think about generative AI, we always like to think of how it generates like school homework or arts or music or whatnot, but when you look at it on the practical side, generative AI is being actually used across various industries. I'll give an example of like Autodesk. Autodesk is a customer who runs an AWS and SageMaker. They already have an offering that enables generated design, where designers can generate many structural designs for products, whereby you give a specific set of constraints and they actually can generate a structure accordingly. And we see similar kind of trend across various industries, where it can be around creative media editing or various others. I have the strong sense that literally, in the next few years, just like now, conventional machine learning is embedded in every application, every mobile app that we see, it is pervasive, and we don't even think twice about it, same way, like almost all apps are built on cloud. Generative AI is going to be part of every startup, and they are going to create remarkable experiences without needing actually, these deep generative AI scientists. But you won't get that until you actually make these models accessible. And I also don't think one model is going to rule the world, then you want these developers to have access to broad range of models. Just like, go back to the early days of deep learning. Everybody thought it is going to be one framework that will rule the world, and it has been changing, from Caffe to TensorFlow to PyTorch to various other things. And I have a suspicion, we had to enable developers where they are, so. >> You know, Dave Vellante and I have been riffing on this concept called super cloud, and a lot of people have co-opted to be multicloud, but we really were getting at this whole next layer on top of say, AWS. You guys are the most comprehensive cloud, you guys are a super cloud, and even Adam and I are talking about ISVs evolving to ecosystem partners. I mean, your top customers have ecosystems building on top of it. This feels like a whole nother AWS. How are you guys leveraging the history of AWS, which by the way, had the same trajectory, startups came in, they didn't want to provision a data center, the heavy lifting, all the things that have made Amazon successful culturally. And day one thinking is, provide the heavy lifting, undifferentiated heavy lifting, and make it faster for developers to program code. AI's got the same thing. How are you guys taking this to the next level, because now, this is an opportunity for the competition to change the game and take it over? This is, I'm sure, a conversation, you guys have a lot of things going on in AWS that makes you unique. What's the internal and external positioning around how you take it to the next level? >> I mean, so I agree with you that generative AI has a very, very strong potential in terms of what it can enable in terms of next generation application. But this is where Amazon's experience and expertise in putting these foundation models to work internally really has helped us quite a bit. If you look at it, like amazon.com search is like a very, very important application in terms of what is the customer impact on number of customers who use that application openly, and the amount of dollar impact it does for an organization. And we have been doing it silently for a while now. And the same thing is true for like Alexa too, which actually not only uses it for natural language understanding other city, even national leverages is set for creating stories and various other examples. And now, our approach to it from AWS is we actually look at it as in terms of the same three tiers like we did in machine learning, because when you look at generative AI, we genuinely see three sets of customers. One is, like really deep technical expert practitioner startups. These are the startups that are creating the next generation models like the likes of stability AIs or Hugging Face with Bloom or AI21. And they generally want to build their own models, and they want the best price performance of their infrastructure for training and inference. That's where our investments in silicon and hardware and networking innovations, where Trainium and Inferentia really plays a big role. And we can nearly do that, and that is one. The second middle tier is where I do think developers don't want to spend time building their own models, let alone, they actually want the model to be useful to that data. They don't need their models to create like high school homeworks or various other things. What they generally want is, hey, I had this data from my enterprises that I want to fine tune and make it really work only for this, and make it work remarkable, can be for tech summarization, to generate a report, or it can be for better Q&A, and so forth. This is where we are. Our investments in the middle tier with SageMaker, and our partnership with Hugging Face and AI21 and co here are all going to very meaningful. And you'll see us investing, I mean, you already talked about CodeWhisperer, which is an open preview, but we are also partnering with a whole lot of top ISVs, and you'll see more on this front to enable the next wave of generated AI apps too, because this is an area where we do think lot of innovation is yet to be done. It's like day one for us in this space, and we want to enable that huge ecosystem to flourish. >> You know, one of the things Dave Vellante and I were talking about in our first podcast we just did on Friday, we're going to do weekly, is we highlighted the AI ChatGPT example as a horizontal use case, because everyone loves it, people are using it in all their different verticals, and horizontal scalable cloud plays perfectly into it. So I have to ask you, as you look at what AWS is going to bring to the table, a lot's changed over the past 13 years with AWS, a lot more services are available, how should someone rebuild or re-platform and refactor their application of business with AI, with AWS? What are some of the tools that you see and recommend? Is it Serverless, is it SageMaker, CodeWhisperer? What do you think's going to shine brightly within the AWS stack, if you will, or service list, that's going to be part of this? As you mentioned, CodeWhisperer and SageMaker, what else should people be looking at as they start tinkering and getting all these benefits, and scale up their ups? >> You know, if we were a startup, first, I would really work backwards from the customer problem I try to solve, and pick and choose, bar, I don't need to deal with the undifferentiated heavy lifting, so. And that's where the answer is going to change. If you look at it then, the answer is not going to be like a one size fits all, so you need a very strong, I mean, granted on the compute front, if you can actually completely accurate it, so unless, I will always recommend it, instead of running compute for running your ups, because it takes care of all the undifferentiated heavy lifting, but on the data, and that's where we provide a whole variety of databases, right from like relational data, or non-relational, or dynamo, and so forth. And of course, we also have a deep analytical stack, where data directly flows from our relational databases into data lakes and data virus. And you can get value along with partnership with various analytical providers. The area where I do think fundamentally things are changing on what people can do is like, with CodeWhisperer, I was literally trying to actually program a code on sending a message through Twilio, and I was going to pull up to read a documentation, and in my ID, I was actually saying like, let's try sending a message to Twilio, or let's actually update a Route 53 error code. All I had to do was type in just a comment, and it actually started generating the sub-routine. And it is going to be a huge time saver, if I were a developer. And the goal is for us not to actually do it just for AWS developers, and not to just generate the code, but make sure the code is actually highly secure and follows the best practices. So, it's not always about machine learning, it's augmenting with automated reasoning as well. And generative AI is going to be changing, and not just in how people write code, but also how it actually gets built and used as well. You'll see a lot more stuff coming on this front. >> Swami, thank you for your time. I know you're super busy. Thank you for sharing on the news and giving commentary. Again, I think this is a AWS moment and industry moment, heavy lifting, accelerated value, agility. AIOps is going to be probably redefined here. Thanks for sharing your commentary. And we'll see you next time, I'm looking forward to doing more follow up on this. It's going to be a big wave. Thanks. >> Okay. Thanks again, John, always a pleasure. >> Okay. This is SiliconANGLE's breaking news commentary. I'm John Furrier with SiliconANGLE News, as well as host of theCUBE. Swami, who's a leader in AWS, has been on theCUBE multiple times. We've been tracking the growth of how Amazon's journey has just been exploding past five years, in particular, past three. You heard the numbers, great performance, great reviews. This is a watershed moment, I think, for the industry, and it's going to be a lot of fun for the next 10 years. Thanks for watching. (bright music)

(upbeat music) >> Okay, welcome back to theCUBE's coverage here at AWS re:Invent 2021. I'm John Furrier, your host. We're in-person, on the floor, also a virtual event, it's a hybrid event. A lot of people watching online, of course, wall-to-wall coverage. We got a great guest here, we're going to go in-depth on the keynote review with Leah Bibbo who's the vice president product and marketing behind the scenes with Adam Selipsky, the CEO, setting up the table for his first keynote as CEO. Leah, great to see you, thanks for coming on. >> Great to be here, thank you. >> So I thought Adam did a fantastic job. It was Adam's keynote, not Andy. He did a great job. Hit his points, Adam's style. But it's the same Amazonian misses, he's Amazonian. I watched a 2006 interview on YouTube, he's talking, taking away all the undifferentiated heavy lifting. He's been Amazonian all his life. He went to Tableau now came back, he's doing a good job. >> He's doing a great job, and he is his own man. But he is an Amazonian, and I think we are really lucky to have him. Our customers are in good hands and it's been fun working with him. >> So had a great one-on-one pre event interview with him for about, almost two hours with Dave Vellante. And I just published a piece, and I think what was interesting was is that, the press was kind of like, oh yeah, new regime change. The competition's heating up, but the game is still the same. Amazon has got an, the same playbook, nothing new here. It's just new capabilities, every re:Invent. It's the same thing, and you have that in the keynote where you kind of hit the history lesson, where we've come. Was that the kind of set the table for the next generation? >> Maybe, I think also we're at this special point, it's the 10th re:Invent and it's our 15th anniversary as AWS. And so I think that's a good moment to kind of look back and see how far we've come. And I think that's one of those things that for all of us, everything's grown so fast. The cloud is something that's changing the world for a lot of customers, but it wasn't that long ago that we just started this new thing. And you know, many of our customers that are here with us, we're part of that. And so we really wanted to acknowledge that. >> And on the product side, a lot of key product announcements, notable. I mean, if you're kind of in the game, you kind of know what it is, but if you're kind of watching from the outside, not inside the ropes, the Annapurna relationship and the grill chips that are coming out with the Silicon is interesting. When you start looking at how the new stacks developing out for this next generation, a big part of the core strategy is compute. >> Absolutely compute is foundational as Adam said, and it's really important. And I think that we've innovated a lot and will continue to innovate in that area because we really want to make the best price performance for any workloads that our customers want to run. Whether that's, you know, legacy workloads like mainframe applications or the workloads of the future that are going to be mobile and running on the 5G network. >> So I have to ask you while got you here before we go to the next piece is that, you're in product marketing, which is a really hard job because I can imagine, `cause there's so much to highlight on the product side. I mean, there's so much, how do you know what people are interested in? Like what's the key feature that people love about AWS? If you had to point to a few, cause there's so much, but the key is to know what resonates with the customers. >> I think that's part of our customer obsession, is that we stay close to the customers and we listen to them. And what we develop is based on what they tell us they need and what we are seeing in their usage as they are. >> Is it compute? Is it the SageMaker? Is it the ML? >> Yes. (laughs) >> You do not want to pick a favorite side >> It's all of those, it's hard to pick. It's hard to pick a favorite. I think that, you know, Adam kind of had three areas where he hit, where he talked about continued innovation and infrastructure and really pushing the edge of the cloud out. He talked about data and data's everywhere and super important to customers right now. And then of course, really doubling down on making solutions that are targeted to use cases and industries. >> And verticals too, very key point. >> Leah: Correct, vertical initiates. >> And as the machine learning really shines there in this vertical and that's the big part of his keynote is that machine learning is everywhere, but these vertical cases that's super important. >> Very much so. >> All right, so let's go through the review. What's the highlights when they announces, what's the new announcements that you guys rolled out today? >> Do you want me to go through all of them, or should I pick a few? >> Pick the highlights, the big ones. >> Well, I think you've kind of touched on a few of them when you talked about some of the work that we're doing in Silicon. And so introducing the new Trainium instance, Trainium based EC2 instance, Trn1 we're pretty excited about that, it's going to offer the best price performance for training in the cloud. I think also we got a lot of buzz around the mainframe modernization. I know that maybe it's not as exciting and forward-looking, but it's super important to a lot of customers and the whole idea that we can cut, you know, cut your migration by two-thirds is kind of a big deal. So we're excited about that and then of course, AWS private 5G. >> Yeah, one of the things that came up in my interviews with Adam was this whole connect phenomenon. And last year during the pandemic, a lot of my interviews I've done with a lot of your customers is the connect thing came up the call center where it was an example of how the call center filled a void during the pandemic when people were kind of, I won't say disabled, but they were having to shift to working at home and cause a big disruption. So this notion of this horizontally scalable use case, purpose-built use case could be offered as a platform and people were into it, it was on fire as he said, that seems to be the trend going forward. He had said that, is that something that you guys are talking more about? Can you give some color into this idea of purpose-built platforms? >> I think it is something that you'll hear us continue to talk about. I think that, you know, connect is one of our fastest growing services. Customers are loving it you're right, it was great during the pandemic. And that's an area where we're going to always continue to make building blocks for our customers. People love all of the building blocks and stitching those together. But we are looking at a lot of different ways, where we can take use cases or vertical industry use cases and, you know, create an abstraction or a new solution that makes it possible for many, many more people to interact with AWS and get the benefits of the cloud. >> A lot of your customers, I interviewed, they loved the whole edge strategy because without posts, they can now put it out in the edge. Now you've got 5G, we had Dish on earlier, we had United Airlines on, the chief digital officer, she was amazing and we, so you've got 5G, which, okay, I get it, but this is a telecom transformation. You've got healthcare, you've got telecom, You've got all these verticals, 5G is huge. How much is that impacting the products, as you guys look at the edge, what's your take on that? What's your, how do you talk to customers about this whole 5G impact? >> Well, I think that, you know, 5G is exciting to everyone. We're very excited about it. As we mentioned, there's a whole range of new applications that are going to be enabled by 5G. And it's going to be applications that are in factories, in hospitals, it's really on the edge. And so I think, I think it's really important and big, but it's still emerging and we're pretty excited about it. >> I mean, Like the whole Amazon, everywhere vibe, Amazon in factories, Amazon on farm windmills, Amazon in cars, I mean cloud's everywhere now, that's a big theme, essentially promotes this continuum, where cloud meets outcomes and that messaging is resonating. When you guys are discussing like the kind of get all this keynote content together. I mean, it's super hard. >> It's really hard. >> Like how do you guys cut it down? What's the, take us inside the ropes. What goes on when you guys have to put this, this program together? Because it's not a lot of time, only a couple of hours on keynote. I mean, seems like a long time. >> Well, I mean, it's really hard. You've kind of nailed it, right? We want to be able to have customers come up and give great presentations about what they're doing so that their peers can see all the innovations that are happening on the cloud. And then we have so much innovation and selecting the innovation is difficult and challenging. I think that we really looked at some of the areas where we feel like we're not leading and customers are super interested in and we kind of make some decisions there. And the good news is, we have a lot more coming, re:Invent is many more days and we'll be probably a lot more news. >> One of the feedback we've been hearing, is the diversity has been really strong. The speakers onstage, 50% women, 50% men. That's really good, 51% is women. So we are going to get more women on stage. I mean, that's, I mean, you had two men and two women, phenomenal. >> Yes. >> Awesome. (laughing) When's it going to be all women? I see that. (laughing) All right, so give me the bottom line. Now let's get into like, okay, what's next? As you guys look to the next couple of days, what's coming, what can people expect for the next re:Invent? Cause they're going to need the bait on that, so some announcement on that. So what's next? >> Well, we have a few more keynotes, you know, tomorrow we'll have Swami and Peter, and then we have Werner on Thursday. >> John: Yes, AI tomorrow with Swami, okay. >> Maybe a little bit of that. And I think you can expect some excitement in that keynote, as always. Peter will do his keynote and then Werner, always look forward to Werner on Thursday. >> What's your big takeaway this year? If you had to boil down this year, re:Invent into kind of a bumper sticker, what's the big theme that people should walk away with this, what's the top story in your mind? >> Well, I think a big part of it, and what's so exciting is that we're all here together. And I feel like everybody's happy to be connecting again. And the energy here is really great. So I think that's one of the big themes, is kind of the community and everything we're doing together across AWS and our customers and our partners and kind of bringing it all together is super exciting. >> And the products are continuing to do well, congratulations. >> I mean, innovation is going to be something that we continue to do. It's a core pillar of AWS and it's in our DNA. >> The number is 27,000 People was here. What's the numbers, 26,000 attendees, here, roughly? >> I do not know the exact numbers. >> More than expected, a big turnout. >> Yeah, it's good, the energy here is great. And we have the, you know, it's our hybrid event as well. So we have a lot of customers that are tuning in virtually. >> Well, thanks for coming on theCUBE. Really appreciate it, congratulations on a great keynote. Thanks for coming on theCUBE. Okay, CUBE coverage here, I'm John Furrier, the worldwide leader in tech coverage is theCUBE, we're here on the ground at AWS re:Invent. Thanks for watching. (upbeat music)