>>And welcome to this special cube conversation. I'm John for a, your host of the cube. We're here in Palo Alto, California, and I'm here with a very special guest coming down from Seattle remotely into the cube studios is the leader at AWS Amazon web services, the vice president of database analytics and machine learning Swami. Great to see you cube alumni recently taking over the database business at AWS as a leader. Congratulations. And thanks for coming on the cube. >>Hey, my pleasure to be here, John, very excited to talk to you. >>Yeah. We've had many conversations on the cube and also in person and also online around all the major mega trends. You've had your hand in all the action, going back to your days when you were in school learning and, and writing papers. And 10 years ago, Amazon web services launched AWS dynamo, DB, fast, flexible, no SQL database that everyone loves today, which has inspired a generation of what I would call database distributing cloud scale, single digit millisecond performance at scale. And again, the key scale. And again, this is 10 years ago, so it seems like yesterday, but you guys are celebrating and your name was on the original paper with CTO Verner. Vogel's your celebrity. Congratulations. >>Thank you. Not sure about the celebrating part, but I'm very excited. At least I played a hand in building such an amazing technology that has enabled so many amazing customers along the way as well. So >>Trivia on the, on the paper as you were an intern at AWS, so you're getting your PhD. And then since, since rising through the ranks and involved in a lot of products over the years, and then leading the machine learning and AI, which is now changing the game at the industry level, but I got to ask you getting back to the story here. A lot of customers have built amazing things on top of dynamo DB, not to mention lots of other AWS and Amazon tech riding on it. Can you share some of the highlights that came out of the original paper? And so with some examples, because I think this is a point in time, 10 years ago, where you start to, so the KickUp of cloud scale, not just, just for developers and building startups, you're really starting to see the scale rise. >>Yeah, I actually, I mean, as you probably know, based on what he read to explain the Genesis of dynamo DB itself had to explain the Genesis of how Amazon got into building the original dynamo, right? And this was during the time when miner, I joined Ron esteem as an intern and, and Amazon was one of the pioneers in pushing the boundary of scale. And a year over year, our Q4 holiday season tends to be really, really bad for all the right reasons. We all want our holiday shopping done during that time. And you want to be able to scale your website, arters fulfillment centers, all of them at that time. And those are the times around 2005. And the answer is when people think our database, they think of a single database server that actually runs on a box and has a certain characteristics and does a scale and availability and whatnot. >>And it's usually relational. And then when we had a major disruption during Q4 that's when yeah, ask ourselves the question, why are we actually using a relational database for some of these things when they really didn't need the data model complexity of relational database. And normally I would say most companies where to actually ask an intern or a few engineers who are early in the career saying like, what the hell are you suggesting? Just go away. But Amazon being enabling Buddhists to build what they want. And they actually let us start reimagining what a database or our scale could look like. And that led to dynamo. And since she unstained mine, then we migrated from an traditional relational database stair this one for some of the amazon.com services. And then I moved on to actually start building some butts off our storage service and then our managed relational database service, I explicitly remember. >>And one of our customer advisory board, we're just the set off some of our leading customers who actually give us feedback on roadmap. Another son, Don, who's the CEO and chief geek of spunk bargain faker. And him actually looking at the Trinity me, I was starting in the corner and saying like you all, both tomorrow and why do I need to keep shotting my, my sequel database and reshooting assigned scaling. And this is the time when the state of the art in most databases were around. Like, you start sharding your relational database and constantly reshaping. And this is when most websites are starting to experience the kind of scale which we consider a normal month. During those times it was mostly, most companies used to have a single relational database backend and start scaling that way. And that conversation led entirely under duress, unaided read, lot of AWS leaders and myself saying like, Hey, what is a cloud database reimagined without the hampering SQL look like? And that led us to start building dynamo DB, but just a key value database at that time. Now we support document might've too, but that single digit millisecond latency at any scale imagine. So >>I think about that time at that time, 10 years ago, when you were having this conversation and I know the smug mug and I, he said, he's in totally geek and he's, he's good to point that out. You also have Netflix as customers too. I'd like to hear how that's evolved, but, but I think back at the time, if you look back then I got to ask you most people we've talked about this before. No one database rules, a world that's now standard people now don't see one database back then it was a one database kind of mindset back then. Yeah. And then you had that big data movement happening with Hadoop. You had the object store developing. So you're in you're you're circling around that area. What was it like then? I mean, take, take us through that because there was obvious visibility that, Hey, let's just store this. Now you see data lakes and that's all happening. But back then object store was kind of new. Yeah. >>Ah, it's a great question. Now, one of the things I realized early on, especially when I was working with binary, when you're saying amazon.com itself as an example, that the access patterns for various applications and Amazon, but let alone AWS customers tend to be very, very, very, some of them really just needed an object store. Some of them needed a relational database. Some of them really wanted a key value store within a fast latency. Some of them really needed a durable cash. And, but it so happens when you have a giant hammer. You use that for everything looks like a map, which is essentially the story at that time. And so everyone kept using the same database, irrespective of what the problem was because nobody else, I mean, thought about like, what else can we build that is better? So this let us do, literally I remember writing a paper with Bernard internally that is widely used in Amazon explaining what are all the menu of booklets that access. >>And then how do we go about actually solving for each of these things so that they can actually grow and innovate faster. And, and this was led to actually the Genesis of not only building IDs and so forth, but also dynamo and various other non-relational data. There's a still let alone not so storage access patterns and what not. So, and this was one of the big revelations he had just that there is not a single database that is going to meet the customer, needs us. The diversity of workloads in the internet is growing. And this was a key pivotal moment because with cloud now applications can scale very more instantly than before now. Building an application for Superbowl is very easier than before. That means that on, I mean, everybody is pushing the boundaries of what scale means, and they are expecting more from their obligations. That's when you need technologies like dynamo, DB, and that's exactly what dynamo already be set out to do. And since then, we are continuing to innovate on behalf of our customers and the purpose of the database story as well. And this concept has resonated well across the board. If you see that the database industry has also embraced this method, >>It's natural that you obviously evolved into the machine learning side of it because that's data is big part of that. And you see back then you, you bringing up kind of like flashes for me where it's like those, the data conversations back then and the data movement was just beginning. So the idea that you can have diversity in access methods of the kind of databases was a use case driven by the application, not so much database saying, this is how you have to work, that the script was flipped. It it's changed from infrastructure dictating to the applications, what to do. Now, the applications are going to the infrastructure and saying, give me what I want. I want to access something here in an office store, something here in no SQL that became the Genesis of infrastructure as code at a, at a global level. And so your paper kind of set the, the, the wave, the influence for this, no SQL did big data movement. It's created tons of value, maybe a third Mongo might've been influenced by this other people have been influenced. Can you share some stories of how people adopted the concept of dynamo DB and how that's changed in the industry and how has that helped the industry evolve? >>I mean, plus file data. Most share our experience of building and dynamo style data store. Very, it is a non-relational API and showing what are some of the experiences that the Venter in building such an paper and these set out early on itself, that it is should not be just a design paper, but it should be something that we shared our experiences. So even now, when I talked to my friends and colleagues and various other companies, one thing they always tell me is they appreciated the openness with which we were sharing. Some of the examples and learnings that we learned to not optimizing for percentile latencies, and what are some of the scalability challenges, how we solved and some of the techniques around things like sloppy Cora or various other stuff. We invented a lot of towns along the way too, but people really appreciated several of some of our findings and as talking about it. >>And since then I met so many other innovations are happening in the industry and the AWS, but also across the entire academia and industry in this space, the databases I've been going through what I call as a period of Renaissance, where one of the things, if you see our own arc, when Roger and I started on the database, front Disney started over the promo saying like, if you were to build a database where cloud is the new normal, this is again in 2008, we asked ourselves that question and what the belt that led us to start building things like dynamo, DB, RDS star. I know that alone, we reimagined data viruses with Redshift and several, and then several other databases like time stream for time series workloads started running Neptune for graph and whatnot. But at the moment we started actually asking that question and working backwards from customers. Then you will start being able to innovate accordingly. And this has worked really well. Then more than a hundred thousand AWS customers have chosen dynamo DB for mobile gaming tech IOT. Many of these are fast growing businesses, such as ledge, Darryl BNB, red fan, as soon as enterprises like Samsung Toyota, capital one and so far. So these are like really some meaningful clouds, let alone amazon.com. I run this. >>We have an internal customer is always good to have that entire inside customer. You know, I really find this a really profound use case because you're just talking, you know, in Amazonian terms, I'll just translate for the audience working backwards from the customer, which is the customer obsession you guys have. So here's, what's going on off the way I see it. You got dynamo, DB, paper, you and Verner, and the team Paul was a great as a great video on your blog posts that goes into the, to the talk he gave at around that time, which is fun to watch if you look back, but you have a radical enabler here, that's disrupting and changing S3 RDS, Aurora. These are game-changing concepts inside the, the landscape of AWS at the same time, you're working backwards from the customer. So the question I have for you as a leader and as a builder, how did you balance the working backwards from the customer while bringing something brand new and radical at that time to the market? >>Yeah, this is one of the S I mean hardest things to be, as leaders need to balance on. If you see many times, then we actually worked backwards from customers. The literal later translated this, literally do what customers are asking for, which is true nine out of 10 times, but there is one or a 10 times, you got to read between the lines on what they are asking. Because many times customers when are articulate that they need to go fast. If in the right way, they might say, Hey, I wish my heart storage goes faster, but they're not going to tell you they need a car, but you need to know and be able to translate and read between the lines we call it under the bucket of innovate on behalf of customers. And that is exactly the kind of a mantra we had when we were thinking about concepts like dynamo DB, because essentially at that time, almost everybody would, if I asked, they would just say, I wish a relational database could actually be able to scale from not just like a hundred gigabyte to one terabyte are, it can take up to like 2 million transactions, a second and so forth and still be cheap and made in reality as relational databases, the way they were engineered at that time, those are not going to meet the scale needs. >>So this is fair. We hunted read between the lines on what are some of the key Mustang needs from customers and then work backwards and then innovate on behalf of these workloads, be enabled by the sun oh four, which are some of the reasons that led to us launching some of the initial sets on dynamo on a single digit millisecond latency and seamless scale. At that time, databases didn't have the elasticity to go from like 10 requests, a second to like a hundred thousand or 1 million requests a second, and then scaled right back in an hour. So that was not possible. And we kind of enabled that. And that was an, a pretty big game changer that showed the elasticity of the cloud to a database. Well, >>Yeah, I think also just to, not to nerd out on this, but it enables a lot of other kind of cool scaled concepts, like queuing storage. It's all kind of together. This database piece of that you guys are solving. And again, props to you guys on the team. Congratulations. I have to ask, you know, more generally, how has your thinking changed since the paper? I'll see, you've got more experience under your belt. You don't yet have the gray hairs yet, but we'll see those soon come in, but you know, you're, you got a lot more experience. You're running teams, you're launching a lot of products. How has your thinking changed in the industry since the paper what's happening now? What's the big evolution. What are those new things now that are in the innovate on behalf of the customer? What's between the lines now, how do you see this happening? >>I mean, now since wanting dynamo via a victim, I had the opportunity to work on various problems in the big data space. There we've worked on some are fire things that you might be aware of in the analytics all the way from Redshift to quick side, too. Then I moved on to start some of our efforts, having built systems that enabled customer to store process and credit, and then analyze them. One of the realizations, I had this, the in around 2015 or 2016, I kinda had that machine learning was hitting a critical point where now it is ready for being scaled at option. Their cloud has basically enabled limitless compute and limitless storage, which are the factors that are holding back machine learning technology. Then I realized that now we have a unique opportunity to bring machine learning BI to everybody, not just folks with PhD in machine learning. >>And that's when I moved on from database and analytics areas, they started machine learning. We're just a descent area because machine learning is powered by data and then started building capabilities like SageMaker, which is our end to end ML platform to build, train and deploy them on models. And this, what does the leading enterprise platform by several gaggled users and then also a bunch of our AI services since then, I view the reason I'm giving all this historical context is one of the biggest realization I had early on itself. And 2016 as first machine learning is one of the most disruptive technologies. She will then country in our generation. This is right after cloud. I think these still are the most amazing combination that is going to revolutionize how we build applications and how we actually reason about that. Now, the second thing is that at the end of the day, when you look at the ANC and journey, it is not just about one database or one data Varroa. >>So one data lake product, or even 1:00 AM out platform. It is about the end to end journey where a customer is storing their order database. And then they are actually building a data lake that test customer history and order history. And they want to be able to personalize. And for their viewer experience are actually forecast what products to staff in their fulfillment center, but then all these things need to work and to handle. And that view is one of the big things that struck me for the past five years. And I've been on this journey in addition to building this Emma building blocks to connect the dots so that customers can go on this modern end to end data strategy as I call it, right. It goes beyond a single database technology or data technology, but putting now all of these end to end together so that customers don't end up spending six months connecting the dots, which has been the state of the down for the last couple of years. And we are bringing it down to matter of the Sundays. Now >>He's incredible Swami. Thank you so much for spending the time with us here in the, >>Yeah, my pleasure. Thanks again, Sean. Thanks for having me.