>> theCUBE presents KubeCon and CloudNativeCon, Europe, 2022. Brought to you by Red Hat, the cloud-native computing foundation and its ecosystem partners. >> Welcome to Valencia, Spain in KubeCon, CloudNativeCon, Europe, 2022. I'm Keith Townsend with my cohosts, Enrico Signoretti, Senior IT Analyst at GigaOm. >> Exactly. >> 7,500 people I'm told, Enrico. What's the flavor of the show so far? >> It's a fantastic mood, I mean, I found a lot of people wanting to track, talk about what they're doing with Kubernetes, sharing their you know, stories, some war stories that bit tough. And you know, this is where you learn actually. Because we had a lot of Zoom calls, webinar and stuff. But it is when you talk a video, "Oh, I did it this way, and it didn't work out very well." So, and, you start a conversation like this that is really different from learning from Zoom, when, you know, everybody talks about things that work it well, they did it right. No, it's here that you learn from other experiences. >> So we're talking to amazing people the whole week, talking about those experiences here on theCUBE. Fresh on the theCUBE for the first time, Chris Voss, senior software engineer at Microsoft Xbox. Chris, welcome to the theCUBE. >> Thank you so much for having me. >> So first off, give us a high level picture of the environment that you're running at Microsoft. >> Yeah. So, you know, we've got 20 well probably close to 30 clusters at this point around the globe, you know 700 to 1,000 pods per cluster, roughly. So about 22,000 pods total. So yeah, it's pretty, pretty sizable footprint and yeah. So we've been running on Kubernetes since 2018 and well actually might be 2017, but anyways, so yeah, that's kind of our footprint. Yeah. >> So all of that, let's talk about the basics which is security across multiple I'm assuming containers, microservices, etcetera. Why did you and the team settle on Linkerd? >> Yeah, so previously we had our own kind of solution for managing TLS certs and things like that. And we found it to be pretty painful, pretty quickly. And so we knew, you know we wanted something that was a little bit more abstracted away from the developers and things like that, that allowed us to move quickly. And so we began investigating, you know, solutions to that. And a few of our colleagues went to Kubecon in San Diego in 2019, Cloudnativecon as well. And basically they just, you know, sponged it all up. And actually funny enough, my old manager was one of the people who was there and he went to the Linkerd booth and they had a thing going that was like, "Hey, get set up with MTLS in five minutes." And he was like, "This is something we want to do, why not check this out?" And he was able to do it. And so that put it on our radar. And so yeah, we investigated several others and Linkerd just perfectly fit exactly what we needed. >> So, in general we are talking about, you know, security at scale. So how you manage security scale and also flexibility. Right? So, but you know, what is the... You told us about the five minutes to start using there but you know, again, we are talking about war stories. We're talking about, you know, all these. So what kind of challenges you found at the beginning when you started adopting this technology? >> So the biggest ones were around getting up and running with like a new service, especially in the beginning, right, we were, you know, adding a new service almost every day. It felt like. And so, you know, basically it took someone going through a whole bunch of different repos, getting approvals from everyone to get the certs minted, all that fun stuff getting them put into the right environments and in the right clusters, to make sure that, you know, everybody is talking appropriately. And just the amount of work that that took alone was just a huge headache and a huge barrier to entry for us to, quickly move up the number of services we have. >> So, I'm trying to wrap my head around the scale of the challenge. When I think about certification or certificate management, I have to do it on a small scale. And every now and again, when a certificate expires it is just a troubleshooting pain. >> Yes. >> So as I think about that, it costs it's not just certificates across 22,000 pods, or it's certificates across 22,000 pods in multiple applications. How were you doing that before Linkerd? Like, what was the... And what were the pain points? Like what happens when a certificate either fails? Or expired up? Not updated? >> So, I mean, to be completely honest, the biggest thing is we're just unable to make the calls, you know, out or in, based on yeah, what is failing basically. But, you know, we saw essentially an uptick in failures around a certain service and pretty quickly, pretty quickly, we got used to the fact that it was like, oh, it's probably a cert expiration issue. And so we tried, you know, a few things in order to make that a little bit more automated and things like that. But we never came to a solution that like didn't require every engineer on the team to know essentially quite a bit about this, just to get into it, which was a huge issue. >> So talk about day two, after you've deployed Linkerd, how did this alleviate software engineers? And what was like the benefits of now having this automated way of managing certs? >> So the biggest thing is like, there is no touch from developers, everyone on our team... Well, I mean, there are a lot of people who are familiar with security and certs and all of that stuff. But no one has to know it. Like it's not a requirement. Like for instance, I knew nothing about it when I joined the team. And even when I was setting up our newer clusters, I knew very little about it. And I was still able to really quickly set up Linkerd, which was really nice. And it's been, you know, essentially we've been able to just kind of set it, and not think about it too much. Obviously, you know, there're parts of it that you have to think about, we monitor it and all that fun stuff, but yeah, it's been pretty painless almost day one. It took a long time to trust it for developers. You know, anytime there was a failure, it's like, "Oh, could this be Linkerd?" you know. But after a while, like now we don't have that immediate assumption because people have built up that trust, but. >> Also you have this massive infrastructure I mean, 30 clusters. So, I guess, that it's quite different to manage a single cluster in 30. So what are the, you know, consideration that you have to do to install this software on, you know, 30 different cluster, manage different, you know versions probably, et cetera, et cetera, et cetera. >> So, I mean, you know, as far as like... I guess, just to clarify, are you asking specifically with Linkerd? Or are you just asking in more in general? >> Well, I mean, you can take that the question in two ways. >> Okay. >> Sure, yeah, so Linkerd in particular but the 30 cluster also quite interesting. >> Yeah. So, I mean, you know, more generally, you know how we manage our clusters and things like that. We have, you know, a CLI tool that we use in order to like change context very quickly, and switch and communicate with whatever cluster we're trying to connect to and you know, are we debugging or getting logs, whatever. And then, you know, with Linkerd it's nice because again, you know, we aren't having to worry about like, oh, how is this cert being inserted in the right node? Or not the right node, but in the right cluster or things like that. Whereas with Linkerd, we don't really have that concern. When we spin up our clusters, essentially we get the route certificate and everything like that packaged up, passed along to Linkerd on installation. And then essentially, there's not much we have to do after that. >> So talk to me about your upcoming section here at Kubecon. what's the high level talking points? Like what attendees learn? >> Yeah. So it's a journey. Those are the sorts of talks that I find useful. Having not been, you know, I'm not a deep Kubernetes expert from, you know decades or whatever of experience, but-- >> I think nobody is. >> (indistinct). >> True, yes. >> That's also true. >> That's another story >> That's a job posting decades of requirements for-- >> Of course, yeah. But so, you know, it's a journey. It's really just like, hey, what made us decide on a service mesh in the first place? What made us choose Linkerd? And then what are the ways in which, you know, we use Linkerd? So what are those, you know we use some of the extra plugins and things like that. And then finally, a little bit about more what we're going to do in the future. >> Let's talk about not just necessarily the future as in two or three days from now, or two or three years from now. Well, the future after you immediately solve the low level problems with Linkerd, what were some of the surprises? Because Linkerd in service mesh and in general have side benefits. Do you experience any of those side benefits as well? >> Yeah, it's funny, you know, writing the blog post, you know, I hadn't really looked at a lot of the data in years on, you know when we did our investigations and things like that. And we had seen that we like had very low latency and low CPU utilization and things like that. And looking at some of that, I found that we were actually saving time off of requests. And I couldn't really think of why that was and I was talking with someone else and the biggest, unfortunately all that data's gone now, like the source data. So I can't go back and verify this but it makes sense, you know, there's the availability zone routing that Linkerd supports. And so I think that's actually doing it where, you know essentially, if a node is closer to another node, it's essentially, you know, routing to those ones. So when one service is talking to another service and maybe they're on the same node, you know, it short circuits that and allows us to gain some time there. It's not huge, but it adds up after, you know, 10, 20 calls down the line. >> Right. In general, so you are saying that it's smooth operations at this very, you know, simplifying your life. >> And again, we didn't have to really do anything for that. It handled that for us. >> It was there? >> Yep. Yeah, exactly. >> So we know one thing when I do it on my laptop it works fine. When I do it with across 22,000 pods, that's a different experience. What were some of the lessons learned coming out of Kubecon 2018 in San Diego? I was there. I wish I would've ran into the Microsoft folks, but what were some of the hard lessons learned scaling Linkerd across the 22,000 nodes? >> So, you know, the first one and this seems pretty obvious, but was just not something I knew about was the high availability mode of Linkerd. So obviously makes sense. You would want that in, you know a large scale environment. So like, that's one of the big lessons that like, we didn't ride away. No. Like one of the mistakes we made in one of our pre-production clusters was not turning that on. And we were kind of surprised. We were like, whoa, like all of these pods are spinning up but they're having issues, like actually getting injected and things like that. And we found, oh, okay. Yeah, you need to actually give it some more resources. But it's still very lightweight considering, you know, they have high availability mode but it's just a few instances still. >> So from, even from, you know, binary perspective and running Linkerd how much overhead is it? >> That is a great question. So I don't remember off the top of my head, the numbers but it's very lightweight. We evaluated a few different service missions and it was the lightest weight that we encountered at that point. >> And then from a resource perspective, is it a team of Linkerd people? Is it a couple of people? Like how? >> To be completely honest for a long time, it was one person Abraham, who actually is the person who proposed this talk. He couldn't make it to Valencia, but he essentially did probably 95% of the work to get into production. And then this was before, we even had a team dedicated to our infrastructure. And so we have, now we have a team dedicated, we're all kind of Linkerd folks, if not Linkerd experts, we at least can troubleshoot basically. And things like that. So it's, I think a group of six people on our team and then, you know various people who've had experience with it on other teams. >> But others, dedicated just to that. >> No one is dedicated just to it. No, it's pretty like pretty light touch once it's up and running. It took a very long time for us to really understand it and to, you know, get like not getting started, but like getting to where we really felt comfortable letting it go in production. But once it was there, like, it is very, very light touch. >> Well, I really appreciate you stopping by Chris. It's been an amazing conversation to hear how Microsoft is using a open source project. >> Exactly. >> At scale, it's just a few years ago when you would've heard the concept of Microsoft and open source together and like OS, just, you know-- >> They have changed a lot in the last few years. Now, there are huge contributors. And, you know, if you go to Azure, it's full of open source stuff, everywhere so. >> Yeah. >> Wow. The Kubecon 2022, how the world has changed in so many ways. From Valencia Spain, I'm Keith Townsend, along with Enrico Signoretti. You're watching theCUBE, the leader in high tech coverage. (upbeat music)

>> Narrator: theCUBE presents KubeCon and Cloud NativeCon Europe, 2022 brought to you by Red Hat, the Cloud Native Computing Foundation and its ecosystem partners. >> Welcome to Valencia, Spain. A coverage of KubeCon, Cloud NativeCon, Europe, 2022. I'm Keith Townsend. Your host of theCUBE, along with Paul Gillum, Senior Editor Enterprise Architecture for Silicon Angle, Enrico, Senior IT Analyst for GigaOm . This has been a full day, 7,500 attendees. I might have seen them run out of food, this is just unexpected. I mean, it escalated from what I understand, it went from capping it off at 4,000 gold, 5,000 gold in it off finally at 7,500 people. I'm super excited for... Today's been a great dead coverage. I'm super excited for tomorrow's coverage from theCUBE, but first off, we'll let the the new person on stage take the first question of the wrap up of the day of coverage, Enrico, what's different about this year versus other KubeCons or Cloud Native conversations. >> I think in general, it's the maturity. So we talk a lot about day two operations, observability, monitoring, going deeper and deeper in the security aspects of the application. So this means that for many enterprises, Kubernetes is becoming real critical. They want to get more control of it. And of course you have the discussion around FinOps, around cost control, because we are deploying Kubernetes everywhere. And if you don't have everything optimized, control, monitored, costs go to the roof and think about deploying the Public Cloud . If your application is not optimized, you're paying more. But also in that, on-premises if you are not optimized, you don't have any clear idea what is going to happen. So capacity planning become the nightmare, that we know from the past. So there is a lot of going on around these topics, really exciting actually, less infrastructure, more application. That is what Kubernetes is in here. >> Paul help me separate some of the signal from the noise. There is a lot going on a lot of overlap. What are some of the big themes of takeaways for day one that Enterprise Architects, Executives, need to take home and really chew on? >> Well, the Kubernetes was a turning point. Docker was introduced nine years ago, and for the first three or four years it was an interesting technology that was not very widely adopted. Kubernetes came along and gave developers a reason to use containers. What strikes me about this conference is that this is a developer event, ordinarily you go to conferences and it's geared toward IT Managers, towards CIOs, this is very much geared toward developers. When you have the hearts and minds of developers the rest of the industry is sort of pulled along with it. So this is ground zero for the hottest area of the entire computing industry right now, is in this area building Distributed services, Microservices based, Cloud Native applications. And it's the developers who are leading the way. I think that's a significant shift. I don't see the Managers here, the CIOs here. These are the people who are pulling this industry into the next generation. >> One of the interesting things that I've seen when we've always said, Kubernetes is for the developers, but we talk with an icon from MoneyGram, who's a end user, he's an enterprise architect, and he brought Kubernetes to his front end developers, and they rejected it. They said, what is this? I just want to develop code. So when we say Kubernetes is for developers or the developers are here, how do we reconcile that mismatch of experience? We have Enterprise Architect here. I hear constantly that the Kubernetes is for developers, but is it a certain kind of developer that Kubernetes is for? >> Well, yes and no. I mean, so the paradigm is changing. Okay. So, and maybe a few years back, it was tough to understand how make your application different. So microservices, everything was new for everybody, but actually, everything has changed to a point and now the developer understands, is neural. So, going through the application, APIs, automation, because the complexity of this application is huge, and you have, 724 kind of development sort of deployment. So you have to stay always on, et cetera, et cetera. And actually, to the point of developers bringing this new generation of decision makers in there. So they are actually decision, they are adopting technology. Maybe it's a sort of shadow IT at the very beginning. So they're adopting it, they're using it. And they're starting to use a lot of open source stuff. And then somebody upper in the stack, the Executive, says what are... They discover that the technology is already in place is a critical component, and then it's transformed in something enterprise, meaning paying enterprise services on top of it to be sure support contract and so on. So it's a real journey. And these guys are the real decision makers, or they are at the base of the decision making process, at least >> Cloud Native is something we're going to learn to take for granted. When you remember back, remember the Fail Whale in the early days of Twitter, when periodically the service would just crash from traffic, or Amazon went through the same thing. Facebook went through the same thing. We don't see that anymore because we are now learning to take Cloud Native for granted. We assume applications are going to be available. They're going to be performant. They're going to scale. They're going to handle anything we throw at them. That is Cloud Native at work. And I think we forget sometimes how refreshing it is to have an internet that really works for you. >> Yeah, I think we're much earlier in the journey. We had Microsoft on, the Xbox team talked about 22,000 pods running Linkerd some of the initial problems and pain points around those challenges. Much of my hallway track conversation has been centered around as we talk about the decision makers, the platform teams. And this is what I'm getting excited to talk about in tomorrow's coverage. Who's on the ground doing this stuff. Is it developers as we see or hear or told? Or is it what we're seeing from the Microsoft example, the MoneyGram example, where central IT is getting it. And not only are they getting it, they're enabling developers to simply write code, build it, and Kubernetes is invisible. It seems like that's become the Holy Grail to make Kubernetes invisible and Cloud Native invisible, and the experience is much closer to Cloud. >> So I think that, it's an interesting, I mean, I had a lot of conversation in the past year is that it's not that the original traditional IT operations are disappearing. So it's just that traditional IT operation are giving resources to these new developers. Okay, so it's a sort of walled garden, you don't see the wall, but it's a walled garden. So they are giving you resources and you use these resources like an internal Cloud. So a few years back, we were talking about private Cloud, the private Cloud as let's say the same identical paradigm of the Public Cloud is not possible, because there are no infinite resources or well, whatever we think are infinite resources. So what you're doing today is giving these developers enough resources to think that they are unlimited and they can do automatic operationing and do all these kind of things. So they don't think about infrastructure at all, but actually it's there. So IT operation are still there providing resources to let developers be more free and agile and everything. So we are still in a, I think an interesting time for all of it. >> Kubernetes and Cloud Native in general, I think are blurring the lines, traditional lines development and operations always were separate entities. Obviously with DevOps, those two are emerging. But now we're moving when you add in shift left testing, shift right testing, DevSecOps, you see the developers become much more involved in the infrastructure and they want to be involved in infrastructure because that's what makes their applications perform. So this is going to cause, I think IT organizations to have to do some rethinking about what those traditional lines are, maybe break down those walls and have these teams work much closer together. And that should be a good thing because the people who are developing applications should also have intimate knowledge of the infrastructure they're going to run on. >> So Paul, another recurring theme that we've heard here is the impact of funding on resources. What have your discussions been around founders and creators when it comes to sourcing talent and the impact of the markets on just their day to day? >> Well, the sourcing talent has been a huge issue for the last year, of course, really, ever since the pandemic started. Interestingly, one of our guests earlier today said that with the meltdown in the tech stock market, actually talent has become more available, because people who were tied to their companies because of their stock options are now seeing those options are underwater and suddenly they're not as loyal to the companies they joined. So that's certainly for the startups, there are many small startups here, they're seeing a bit of a windfall now from the tech stock bust. Nevertheless, skills are a long term problem. The US educational system is turning out about 10% of the skilled people that the industry needs every year. And no one I know, sees an end to that issue anytime soon. >> So Enrico, last question to you. Let's talk about what that means to the practitioner. There's a lot of opportunity out there. 200 plus sponsors I hear, I think is worth the projects is 200 plus, where are the big opportunities as a practitioner, as I'm thinking about the next thing that I'm going to learn to help me survive the next 10 or 15 years of my career? Where you think the focus should be? Should it be that low level Cloud builder? Or should it be at those levels of extraction that we're seeing and reading about? >> I think that it's a good question. The answer is not that easy. I mean, being a developer today, for sure, grants you a salary at the end of the month. I mean, there is high demand, but actually there are a lot of other technical figures in the data center, in the Cloud, that could really find easily a job today. So, developers is the first in my mind also because they are more, they can serve multiple roles. It means you can be a developer, but actually you can be also with the new roles that we have, especially now with the DevOps, you can be somebody that supports operation because you know automation, you know a few other things. So you can be a sysadmin of the next generation even if you are a developer, even if when you start as a developer. >> KubeCon 2022, is exciting. I don't care if you're a developer, practitioner, a investor, IT decision maker, CIO, CXO, there's so much to learn and absorb here and we're going to be covering it for the next two days. Me and Paul will be shoulder to shoulder, I'm not going to say you're going to get sick of this because it's just, it's all great information, we'll help sort all of this. From Valencia, Spain. I'm Keith Townsend, along with my host Enrico Signoretti, Paul Gillum, and you're watching theCUBE, the leader in high tech coverage. (upbeat music)

>> Instructor: "theCUBE" presents KubeCon and CloudNativeCon Europe 2022, brought to you by Red Hat, the Cloud Native Computing Foundation and its ecosystem partners. >> Welcome to Valencia, Spain and we're at KubeCon, CloudNativeCon Europe 2022. I'm Keith Townsend, and my co-host, Enrico Signoretti. Enrico's really proud of me. I've called him Enrico instead of Enrique every session. >> Every day. >> Senior IT analyst at GigaOm. We're talking to fantastic builders at KubeCon, CloudNativeCon Europe 2022 about the projects and their efforts. Enrico, up to this point, it's been all about provisioning, insecurity, what conversation have we been missing? >> Well, I mean, I think that we passed the point of having the conversation of deployment, of provisioning. Everybody's very skilled, actually everything is done at day two. They are discovering that, well, there is a security problem. There is an observability problem a and in fact, we are meeting with a lot of people and there are a lot of conversation with people really needing to understand what is happening. I mean, in their cluster work, why it is happening and all the questions that come with it. And the more I talk with people in the show floor here or even in the various sessions is about, we are growing so that our clusters are becoming bigger and bigger, applications are becoming bigger as well. So we need to now understand better what is happening. As it's not only about cost, it's about everything at the end. >> So I think that's a great set up for our guests, Matt Provo, founder and CEO of StormForge and Patrick Brixton? >> Bergstrom. >> Bergstrom. >> Yeah. >> I spelled it right, I didn't say it right, Bergstrom, CTO. We're at KubeCon, CloudNativeCon where projects are discussed, built and StormForge, I've heard the pitch before, so forgive me. And I'm kind of torn. I have service mesh. What do I need more, like what problem is StormForge solving? >> You want to take it? >> Sure, absolutely. So it's interesting because, my background is in the enterprise, right? I was an executive at UnitedHealth Group before that I worked at Best Buy and one of the issues that we always had was, especially as you migrate to the cloud, it seems like the CPU dial or the memory dial is your reliability dial. So it's like, oh, I just turned that all the way to the right and everything's hunky-dory, right? But then we run into the issue like you and I were just talking about, where it gets very very expensive very quickly. And so my first conversations with Matt and the StormForge group, and they were telling me about the product and what we're dealing with. I said, that is the problem statement that I have always struggled with and I wish this existed 10 years ago when I was dealing with EC2 costs, right? And now with Kubernetes, it's the same thing. It's so easy to provision. So realistically what it is, is we take your raw telemetry data and we essentially monitor the performance of your application, and then we can tell you using our machine learning algorithms, the exact configuration that you should be using for your application to achieve the results that you're looking for without over-provisioning. So we reduce your consumption of CPU, of memory and production which ultimately nine times out of 10, actually I would say 10 out of 10, reduces your cost significantly without sacrificing reliability. >> So can your solution also help to optimize the application in the long run? Because, yes, of course-- >> Yep. >> The lowering fluid as you know optimize the deployment. >> Yeah. >> But actually the long-term is optimizing the application. >> Yes. >> Which is the real problem. >> Yep. >> So, we're fine with the former of what you just said, but we exist to do the latter. And so, we're squarely and completely focused at the application layer. As long as you can track or understand the metrics you care about for your application, we can optimize against it. We love that we don't know your application, we don't know what the SLA and SLO requirements are for your app, you do, and so, in our world it's about empowering the developer into the process, not automating them out of it and I think sometimes AI and machine learning sort of gets a bad rap from that standpoint. And so, at this point the company's been around since 2016, kind of from the very early days of Kubernetes, we've always been, squarely focused on Kubernetes, using our core machine learning engine to optimize metrics at the application layer that people care about and need to go after. And the truth of the matter is today and over time, setting a cluster up on Kubernetes has largely been solved. And yet the promise of Kubernetes around portability and flexibility, downstream when you operationalize, the complexity smacks you in the face and that's where StormForge comes in. And so we're a vertical, kind of vertically oriented solution, that's absolutely focused on solving that problem. >> Well, I don't want to play, actually. I want to play the devils advocate here and-- >> You wouldn't be a good analyst if you didn't. >> So the problem is when you talk with clients, users, there are many of them still working with Java, something that is really tough. I mean, all of us loved Java. >> Yeah, absolutely. >> Maybe 20 years ago. Yeah, but not anymore, but still they have developers, they have porting applications, microservices. Yes, but not very optimized, et cetera, cetera, et cetera. So it's becoming tough. So how you can interact with this kind of old hybrid or anyway, not well engineered applications. >> Yeah. >> We do that today. We actually, part of our platform is we offer performance testing in a lower environment and stage and we, like Matt was saying, we can use any metric that you care about and we can work with any configuration for that application. So perfect example is Java, you have to worry about your heap size, your garbage collection tuning and one of the things that really struck me very early on about the StormForge product is because it is true machine learning. You remove the human bias from that. So like a lot of what I did in the past, especially around SRE and performance tuning, we were only as good as our humans were because of what they knew. And so, we kind of got stuck in these paths of making the same configuration adjustments, making the same changes to the application, hoping for different results. But then when you apply machine learning capability to that the machine will recommend things you never would've dreamed of. And you get amazing results out of that. >> So both me and Enrico have been doing this for a long time. Like, I have battled to my last breath the argument when it's a bare metal or a VM, look, I cannot give you any more memory. >> Yeah. >> And the argument going all the way up to the CIO and the CIO basically saying, you know what, Keith you're cheap, my developer resources are expensive, buy bigger box. >> Yeah. >> Yap. >> Buying a bigger box in the cloud to your point is no longer a option because it's just expensive. >> Yeah. >> Talk to me about the carrot or the stick as developers are realizing that they have to be more responsible. Where's the culture change coming from? Is it the shift in responsibility? >> I think the center of the bullseye for us is within those sets of decisions, not in a static way, but in an ongoing way, especially as the development of applications becomes more and more rapid and the management of them. Our charge and our belief wholeheartedly is that you shouldn't have to choose. You should not have to choose between costs or performance. You should not have to choose where your applications live, in a public private or hybrid cloud environment. And so, we want to empower people to be able to sit in the middle of all of that chaos and for those trade offs and those difficult interactions to no longer be a thing. We're at a place now where we've done hundreds of deployments and never once have we met a developer who said, "I'm really excited to get out of bed and come to work every day and manually tune my application." One side, secondly, we've never met, a manager or someone with budget that said, please don't increase the value of my investment that I've made to lift and shift us over to the cloud or to Kubernetes or some combination of both. And so what we're seeing is the converging of these groups, their happy place is the lack of needing to be able to make those trade offs, and that's been exciting for us. >> So, I'm listening and looks like that your solution is right in the middle in application performance, management, observability. >> Yeah. >> And, monitoring. >> Yeah. >> So it's a little bit of all of this. >> Yeah, so we want to be, the intel inside of all of that, we often get lumped into one of those categories, it used to be APM a lot, we sometimes get, are you observability or and we're really not any of those things, in and of themselves, but we instead we've invested in deep integrations and partnerships with a lot of that tooling 'cause in a lot of ways, the tool chain is hardening in a cloud native and in Kubernetes world. And so, integrating in intelligently, staying focused and great at what we solve for, but then seamlessly partnering and not requiring switching for our users who have already invested likely, in a APM or observability. >> So to go a little bit deeper. What does it mean integration? I mean, do you provide data to this, other applications in the environment or are they supporting you in the work that you do. >> Yeah, we're a data consumer for the most part. In fact, one of our big taglines is take your observability and turn it into action ability, right? Like how do you take that, it's one thing to collect all of the data, but then how do you know what to do with it, right? So to Matt's point, we integrate with folks like Datadog, we integrate with Prometheus today. So we want to collect that telemetry data and then do something useful with it for you. >> But also we want Datadog customers, for example, we have a very close partnership with Datadog so that in your existing Datadog dashboard, now you have-- >> Yeah. >> The StormForge capability showing up in the same location. >> Yep. >> And so you don't have to switch out. >> So I was just going to ask, is it a push pull? What is the developer experience when you say you provide developer this resolve ML learnings about performance, how do they receive it? Like, what's the developer experience. >> They can receive it, for a while we were CLI only, like any good developer tool. >> Right. >> And, we have our own UI. And so it is a push in a lot of cases where I can come to one spot, I've got my applications and every time I'm going to release or plan for a release or I have released and I want to pull in observability data from a production standpoint, I can visualize all of that within the StormForge UI and platform, make decisions, we allow you to set your, kind of comfort level of automation that you're okay with. You can be completely set and forget or you can be somewhere along that spectrum and you can say, as long as it's within, these thresholds, go ahead and release the application or go ahead and apply the configuration. But we also allow you to experience the same, a lot of the same functionality right now, in Grafana, in Datadog and a bunch of others that are coming. >> So I've talked to Tim Crawford who talks to a lot of CIOs and he's saying one of the biggest challenges or if not, one of the biggest challenges CIOs are facing are resource constraints. >> Yeah. >> They cannot find the developers to begin with to get this feedback. How are you hoping to address this biggest pain point for CIOs-- >> Yeah.6 >> And developers? >> You should take that one. >> Yeah, absolutely. So like my background, like I said at UnitedHealth Group, right. It's not always just about cost savings. In fact, the way that I look about at some of these tech challenges, especially when we talk about scalability there's kind of three pillars that I consider, right? There's the tech scalability, how am I solving those challenges? There's the financial piece 'cause you can only throw money at a problem for so long and it's the same thing with the human piece. I can only find so many bodies and right now that pool is very small, and so, we are absolutely squarely in that footprint of we enable your team to focus on the things that they matter, not manual tuning like Matt said. And then there are other resource constraints that I think that a lot of folks don't talk about too. Like, you were talking about private cloud for instance and so having a physical data center, I've worked with physical data centers that companies I've worked for have owned where it is literally full, wall to wall. You can't rack any more servers in it, and so their biggest option is, well, I could spend $1.2 billion to build a new one if I wanted to, or if you had a capability to truly optimize your compute to what you needed and free up 30% of your capacity of that data center. So you can deploy additional name spaces into your cluster, like that's a huge opportunity. >> So I have another question. I mean, maybe it doesn't sound very intelligent at this point, but, so is it an ongoing process or is it something that you do at the very beginning, I mean you start deploying this. >> Yeah. >> And maybe as a service. >> Yep. >> Once in a year I say, okay, let's do it again and see if something change it. >> Sure. >> So one spot, one single.. >> Yeah, would you recommend somebody performance test just once a year? Like, so that's my thing is, at previous roles, my role was to do performance test every single release, and that was at a minimum once a week and if your thing did not get faster, you had to have an executive exception to get it into production and that's the space that we want to live in as well as part of your CICD process, like this should be continuous verification, every time you deploy, we want to make sure that we're recommending the perfect configuration for your application in the name space that you're deploying into. >> And I would be as bold as to say that we believe that we can be a part of adding, actually adding a step in the CICD process that's connected to optimization and that no application should be released, monitored, and sort of analyzed on an ongoing basis without optimization being a part of that. And again, not just from a cost perspective, but for cost and performance. >> Almost a couple of hundred vendors on this floor. You mentioned some of the big ones Datadog, et cetera, but what happens when one of the up and comings out of nowhere, completely new data structure, some imaginative way to click to telemetry data. >> Yeah. >> How do, how do you react to that? >> Yeah, to us it's zeros and ones. >> Yeah. >> And, we really are data agnostic from the standpoint of, we're fortunate enough from the design of our algorithm standpoint, it doesn't get caught up on data structure issues, as long as you can capture it and make it available through one of a series of inputs, one would be load or performance tests, could be telemetry, could be observability, if we have access to it. Honestly, the messier the better from time to time from a machine learning standpoint, it's pretty powerful to see. We've never had a deployment where we saved less than 30%, while also improving performance by at least 10%. But the typical results for us are 40 to 60% savings and 30 to 40% improvement in performance. >> And what happens if the application is, I mean, yes Kubernetes is the best thing of the world but sometimes we have to, external data sources or, we have to connect with external services anyway. >> Yeah. >> So, can you provide an indication also on this particular application, like, where the problem could be? >> Yeah. >> Yeah, and that's absolutely one of the things that we look at too, 'cause it's, especially when you talk about resource consumption it's never a flat line, right? Like depending on your application, depending on the workloads that you're running it varies from sometimes minute to minute, day to day, or it could be week to week even. And so, especially with some of the products that we have coming out with what we want to do, integrating heavily with the HPA and being able to handle some of those bumps and not necessarily bumps, but bursts and being able to do it in a way that's intelligent so that we can make sure that, like I said, it's the perfect configuration for the application regardless of the time of day that you're operating in or what your traffic patterns look like, or, what your disc looks like, right. Like 'cause with our low environment testing, any metric you throw at us, we can optimize for. >> So Matt and Patrick, thank you for stopping by. >> Yeah. >> Yes. >> We can go all day because day two is I think the biggest challenge right now, not just in Kubernetes but application re-platforming and transformation, very, very difficult. Most CTOs and EASs that I talked to, this is the challenge space. From Valencia, Spain, I'm Keith Townsend, along with my host Enrico Signoretti and you're watching "theCube" the leader in high-tech coverage. (whimsical music)

(bright upbeat music) >> Hello everyone, and welcome to the special exclusive CUBE Conversation, where we continue our coverage of the trends of the database market. With me is Nipun Agarwal, who's the vice president, MySQL HeatWave in advanced development at Oracle. Nipun, welcome. >> Thank you Dave. >> I love to have technical people on the Cube to educate, debate, inform, and we've extensively covered this market. We were all over the Snowflake IPO and at that time I remember, I challenged organizations bring your best people. Because I want to better understand what's happening at Database. After Oracle kind of won the Database wars 20 years ago, Database kind of got boring. And then it got really exciting with the big data movement, and all the, not only SQL stuff coming out, and Hadoop and blah, blah, blah. And now it's just exploding. You're seeing huge investments from many of your competitors, VCs are trying to get into the action. Meanwhile, as I've said many, many times, your chairman and head of technology, CTO, Larry Ellison, continues to invest to keep Oracle relevant. So it's really been fun to watch and I really appreciate you coming on. >> Sure thing. >> We have written extensively, we talked to a lot of Oracle customers. You get the leading mission critical database in the world. Everybody from Fortune 100, we evaluated what Gardner said about the operational databases. I think there's not a lot of question there. And we've written about that on WikiBound about you're converged databases, and the strategy there, and we're going to get into that. We've covered Autonomous Data Warehouse Exadata Cloud at Customer, and then we just want to really try to get into your area, which has been, kind of caught our attention recently. And I'm talking about the MySQL Database Service with HeatWave. I love the name, I laugh. It was an unveiled, I don't know, a few months ago. So Nipun, let's start the discussion today. Maybe you can update our viewers on what is HeatWave? What's the overall focus with Oracle? And how does it fit into the Cloud Database Service? >> Sure Dave. So HeatWave is a in-memory query accelerator for the MySQL Database Service for speeding up analytic queries as well as long running complex OLTP queries. And this is all done in the context of a single database which is the MySQL Database Service. Also, all existing MySQL applications or MySQL compatible tools and applications continue to work as is. So there is no change. And with this HeatWave, Oracle is delivering the only MySQL service which provides customers with a single unified platform for both analytic as well as transaction processing workloads. >> Okay, so, we've seen open source databases in the cloud growing very rapidly. I mentioned Snowflake, I think Google's BigQuery, get some mention, we'll talk, we'll maybe talk more about Redshift later on, but what I'm wondering, well let's talk about now, how does MySQL HeatWave service, how does that compare to MySQL-based services from other cloud vendors? I can get MySQL from others. In fact, I think we do. I think we run WikiBound on the LAMP stack. I think it's running on Amazon, but so how does your service compare? >> No other vendor, like, no other vendor offers this differentiated solution with an open source database namely, having a single database, which is optimized both for transactional processing and analytics, right? So the example is like MySQL. A lot of other cloud vendors provide MySQL service but MySQL has been optimized for transaction processing so when customs need to run analytics they need to move the data out of MySQL into some other database for any analytics, right? So we are the only vendor which is now offering this unified solution for both transactional processing analytics. That's the first point. Second thing is, most of the vendors out there have taken open source databases and they're basically hosting it in the cloud. Whereas HeatWave, has been designed from the ground up for the cloud, and it is a 100% compatible with MySQL applications. And the fact that we have designed it from the ground up for the cloud, maybe I'll spend 100s of person years of research and engineering means that we have a solution, which is very, very scalable, it's very optimized in terms of performance, and it is very inexpensive in terms of the cost. >> Are you saying, well, wait, are you saying that you essentially rewrote MySQL to create HeatWave but at the same time maintained compatibility with existing applications? >> Right. So we enhanced MySQL significantly and we wrote a whole bunch of new code which is brand new code optimized for the cloud in such a manner that yes, it is 100% compatible with all existing MySQL applications. >> What does it mean? And if I'm to optimize for the cloud, I mean, I hear that and I say, okay, it's taking advantage of cloud-native. I hear kind of the buzzwords, cloud-first, cloud-native. What does it specifically mean from a technical standpoint? >> Right. So first, let's talk about performance. What we have done is that we have looked at two aspects. We have worked with shapes like for instance, like, the compute shapes which provide the best performance for dollar, per dollar. So I'll give you a couple of examples. We have optimized for certain shifts. So, HeatWave is in-memory query accelerator. So the cost of the system is dominated by the cost. So we are working with chips which provide the cheapest cost per terabyte of memory. Secondly, we are using commodity cloud services in such a manner that it's in-optimized for both performance as well as performance per dollar. So, example is, we are not using any locally-attached SSDs. We use ObjectStore because it's very inexpensive. And then I guess at some point I will get into the details of the architecture. The system has been really, really designed for massive scalability. So as you add more compute, as you add more service, the system continues to scale almost perfectly linearly. So this is what I mean in terms of being optimized for the cloud. >> All right, great. >> And furthermore, (indistinct). >> Thank you. No, carry on. >> Over the next few months, you will see a bunch of other announcements where we're adding a whole bunch of machine learning and data driven-based automation which we believe is critical for the cloud. So optimized for performance, optimized for the cloud, and machine learning-based automation which we believe is critical for any good cloud-based service. >> All right, I want to come back and ask you more about the architecture, but you mentioned some of the others taking open source databases and shoving them into the cloud. Let's take the example of AWS. They have a series of specialized data stores and, for different workloads, Aurora is for OLTP I actually think it's based on MySQL Redshift which is based on ParAccel. And so, and I've asked Amazon about this, and their response is, actually kind of made sense to me. Look, we want the right tool for the right job, we want access to the primitives because when the market changes we can change faster as opposed to, if we put, if we start building bigger and bigger databases with more functionality, it's, we're not as agile. So that kind of made sense to me. I know we, again, we use a lot, we use, I think I said MySQL in Amazon we're using DynamoDB, works, that's cool. We're not huge. And I, we fully admit and we've researched this, when you start to get big that starts to get maybe expensive. But what do you think about that approach and why is your approach better? >> Right, we believe that there are multiple drawbacks of having different databases or different services, one, optimized for transactional processing and one for analytics and having to ETL between these different services. First of all, it's expensive because you have to manage different databases. Secondly, it's complex. From an application standpoint, applications need, now need to understand the semantics of two different databases. It's inefficient because you have to transfer data at some PRPC from one database to the other one. It's not secure because there is security aspects involved when your transferring data and also the identity of users in the two different databases is different. So it's, the approach which has been taken by Amazons and such, we believe, is more costly, complex, inefficient and not secure. Whereas with HeatWave, all the data resides in one database which is MySQL and it can run both transaction processing and analytics. So in addition to all the benefits I talked about, customers can also make their decisions in real time because there is no need to move the data. All the data resides in a single database. So as soon as you make any changes, those changes are visible to customers for queries right away, which is not the case when you have different siloed specialized databases. >> Okay, that, a lot of ways to skin a cat and that what you just said makes sense. By the way, we were saying before, companies have taken off the shelf or open source database has shoved them in the cloud. I have to give Amazon some props. They actually have done engineering to Aurora and Redshift. And they've got the engineering capabilities to do that. But you can see, for example, in Redshift the way they handle separating compute from storage it's maybe not as elegant as some of the other players like a Snowflake, for example, but they get there and they, maybe it's a little bit more brute force but so I don't want to just make it sound like they're just hosting off the shelf in the cloud. But is it fair to say that there's like a crossover point? So in other words, if I'm smaller and I'm not, like doing a bunch of big, like us, I mean, it's fine. It's easy, I spin it up. It's cheaper than having to host my own servers. So there's, presumably there's a sweet spot for that approach and a sweet spot for your approach. Is that fair or do you feel like you can cover a wider spectrum? >> We feel we can cover the entire spectrum, not wider, the entire spectrum. And we have benchmarks published which are actually available on GitHub for anyone to try. You will see that this approach you have taken with the MySQL Database Service in HeatWave, we are faster, we are cheaper without having to move the data. And the mileage or the amount of improvement you will get, surely vary. So if you have less data the amount of improvement you will get, maybe like say 100 times, right, or 500 times, but smaller data sizes. If you get to lots of data sizes this improvement amplifies to 1000 times or 10,000 times. And similarly for the cost, if the data size is smaller, the cost advantage you will have is less, maybe MySQL HeatWave is one third the cost. If the data size is larger, the cost advantage amplifies. So to your point, MySQL Database Service in HeatWave is going to be better for all sizes but the amount of mileage or the amount of benefit you will get increases as the size of the data increases. >> Okay, so you're saying you got better performance, better cost, better price performance. Let me just push back a little bit on this because I, having been around for awhile, I often see these performance and price comparisons. And what often happens is a vendor will take the latest and greatest, the one they just announced and they'll compare it to an N-1 or an N-2, running on old hardware. So, is, you're normalizing for that, is that the game you're playing here? I mean, how can you, give us confidence that this is easier kind of legitimate benchmarks in your GitHub repo. >> Absolutely. I'll give you a bunch of like, information. But let me preface this by saying that all of our scripts are available in the open source in the GitHub repo for anyone to try and we would welcome feedback otherwise. So we have taken, yes, the latest version of MySQL Database Service in HeatWave, we have optimized it, and we have run multiple benchmarks. For instance, TBC-H, TPC-DS, right? Because the amount of improvement a query will get depends upon the specific query, depends upon the predicates, it depends on the selectivity so we just wanted to use standard benchmarks. So it's not the case that if you're using certain classes of query, excuse me, benefit, get them more. So, standard benchmarks. Similarly, for the other vendors or other services like Redshift, we have run benchmarks on the latest shapes of Redshift the most optimized configuration which they recommend, running their scripts. So this is not something that, hey, we're just running out of the box. We have optimized Aurora, we have optimized (indistinct) to the best and possible extent we can based on their guidelines, based on their latest release, and that's what you're talking about in terms of the numbers. >> All right. Please continue. >> Now, for some other vendors, if we get to the benchmark section, we'll talk about, we are comparing with other services, let's say Snowflake. Well there, there are issues in terms of you can't legally run Snowflake numbers, right? So there, we have looked at some reports published by Gigaom report. and we are taking the numbers published by the Gigaom report for Snowflake, Google BigQuery and as you'll see maps numbers, right? So those, we have not won ourselves. But for AWS Redshift, as well as AWS Aurora, we have run the numbers and I believe these are the best numbers anyone can get. >> I saw that Gigaom report and I got to say, Gigaom, sometimes I'm like, eh, but I got to say that, I forget the guy's name, he knew what he was talking about. He did a good job, I thought. I was curious as to the workload. I always say, well, what's the workload. And, but I thought that report was pretty detailed. And Snowflake did not look great in that report. Oftentimes, and they've been marketing the heck out of it. I forget who sponsored it. It is, it was sponsored content. But, I did, I remember seeing that and thinking, hmm. So, I think maybe for Snowflake that sweet spot is not, maybe not that performance, maybe it's the simplicity and I think that's where they're making their mark. And most of their databases are small and a lot of read-only stuff. And so they've found a market there. But I want to come back to the architecture and really sort of understand how you've able, you've been able to get this range of both performance and cost you talked about. I thought I heard that you're optimizing the chips, you're using ObjectStore. You're, you've got an architecture that's not using SSD, it's using ObjectStore. So this, is their cashing there? I wonder if you could just give us some details of the architecture and tell us how you got to where you are. >> Right, so let me start off saying like, what are the kind of numbers we are talking about just to kind of be clear, like what the improvements are. So if you take the MySQL Database Service in HeatWave in Oracle Cloud and compare it with MySQL service in any other cloud, and if you look at smaller data sizes, say data sizes which are about half a terabyte or so, HeatWave is 400 times faster, 400 times faster. And as you get to... >> Sorry. Sorry to interrupt. What are you measuring there? Faster in terms of what? >> Latency. So we take TCP-H 22 queries, we run them on HeatWave, and we run the same queries on MySQL service on any other cloud, half a terabyte and the performance in terms of latency is 400 times faster in HeatWave. >> Thank you. Okay. >> If you go to a lot of other data sites, then the other data point of view, we're looking at say something like, 4 TB, there, we did two comparisons. One is with AWS Aurora, which is, as you said, they have taken MySQL. They have done a bunch of innovations over there and we are offering it as a premier service. So on 4 TB TPC-H, MySQL Database Service with HeatWave is 1100 times faster than Aurora. It is three times faster than the fastest shape of Redshift. So Redshift comes in different flavors some talking about dense compute too, right? And again, looking at the most recommended configuration from Redshift. So 1100 times faster that Aurora, three times faster than Redshift and at one third, the cost. So this where I just really want to point out that it is much faster and much cheaper. One third the cost. And then going back to the Gigaom report, there was a comparison done with Snowflake, Google BigQuery, Redshift, Azure Synapse. I wouldn't go into the numbers here but HeatWave was faster on both TPC-H as well as TPC-DS across all these products and cheaper compared to any of these products. So faster, cheaper on both the benchmarks across all these products. Now let's come to, like, what is the technology underneath? >> Great. >> So, basically there are three parts which you're going to see. One is, improve performance, very good scale, and improve a lower cost. So the first thing is that HeatWave has been optimized and, for the cloud. And when I say that, we talked about this a bit earlier. One is we are using the cheapest shapes which are available. We're using the cheapest services which are available without having to compromise the performance and then there is this machine learning-based automation. Now, underneath, in terms of the architecture of HeatWave there are basically, I would say, four key things. First is, HeatWave is an in-memory engine that a presentation which we have in memory is a hybrid columnar representation which is optimized for vector process. That's the first thing. And that's pretty table stakes these days for anyone who wants to do in-memory analytics except that it's hybrid columnar which is optimized for vector processing. So that's the first thing. The second thing which starts getting to be novel is that HeatWave has a massively parallel architecture which is enabled by a massively partitioned architecture. So we take the data, we read the data from MySQL into the memory of the HeatWave and we massively partition this data. So as we're reading the data, we're partitioning the data based on the workload, the sizes of these partitions is such that it fits in the cache of the underlying processor and then we're able to consume these partitions really, really fast. So that's the second bit which is like, massively parallel architecture enabled by massively partitioned architecture. Then the third thing is, that we have developed new state-of-art algorithms for distributed query processing. So for many of the workloads, we find that joints are the long pole in terms of the amount of time it takes. So we at Oracle have developed new algorithms for distributed joint processing and similarly for many other operators. And this is how we're being able to consume this data or process this data, which is in-memory really, really fast. And finally, and what we have, is that we have an eye for scalability and we have designed algorithms such that there's a lot of overlap between compute and communication, which means that as you're sending data across various nodes and there could be like, dozens of of nodes or 100s of nodes that they're able to overlap the computation time with the communication time and this is what gives us massive scalability in the cloud. >> Yeah, so, some hard core database techniques that you've brought to HeatWave, that's impressive. Thank you for that description. Let me ask you, just to go to quicker side. So, MySQL is open source, HeatWave is what? Is it like, open core? Is it open source? >> No, so, HeatWave is something which has been designed and optimized for the cloud. So it can't be open source. So any, it's not open service. >> It is a service. >> It is a service. That's correct. >> So it's a managed service that I pay Oracle to host for me. Okay. Got it. >> That's right. >> Okay, I wonder if you could talk about some of the use cases that you're seeing for HeatWave, any patterns that you're seeing with customers? >> Sure, so we've had the service, we had the HeatWave service in limited availability for almost 15 months and it's been about five months since we have gone G. And there's a very interesting trend of our customers we're seeing. The first one is, we are seeing many migrations from AWS specifically from Aurora. Similarly, we are seeing many migrations from Azure MySQL we're migrations from Google. And the number one reason customers are coming is because of ease of use. Because they have their databases currently siloed. As you were talking about some for optimized for transactional processing, some for analytics. Here, what customers find is that in a single database, they're able to get very good performance, they don't need to move the data around, they don't need to manage multiple databaes. So we are seeing many migrations from these services. And the number one reason is reduce complexity of ease of use. And the second one is, much better performance and reduced costs, right? So that's the first thing. We are very excited and delighted to see the number of migrations we're getting. The second thing which we're seeing is, initially, when we had the service announced, we were like, targeting really towards analytics. But now what are finding is, many of these customers, for instance, who have be running on Aurora, when they are moving from MySQL in HeatWave, they are finding that many of the OLTP queries as well, are seeing significant acceleration with the HeatWave. So now customers are moving their entire applications or, to HeatWave. So that's the second trend we're seeing. The third thing, and I think I kind of missed mentioning this earlier, one of the very key and unique value propositions we provide with the MySQL Database Service in HeatWave, is that we provide a mechanism where if customers have their data stored on premise they can still leverage the HeatWave service by enabling MySQL replication. So they can have their data on premise, they can replicate this data in the Oracle Cloud and then they can run analytics. So this deployment which we are calling the hybrid deployment is turning out to be very, very popular because there are customers, there are some customers who for various reasons, compliance or regulatory reasons cannot move the entire data to the cloud or migrate the data to the cloud completely. So this provides them a very good setup where they can continue to run their existing database and when it comes to getting benefits of HeatWave for query acceleration, they can set up this replication. >> And I can run that on anyone, any available server capacity or is there an appliance to facilitate that? >> No, this is just standard MySQL replication. So if a customer is running MySQL on premise they can just turn off this application. We have obviously enhanced it to support this inbound replication between on-premise and Oracle Cloud with something which can be enabled as long as the source and destination are both MySQL. >> Okay, so I want to come back to this sort of idea of the architecture a little bit. I mean, it's hard for me to go toe to toe with the, I'm not an engineer, but I'm going to try anyway. So you've talked about OLTP queries. I thought, I always thought HeatWave was optimized for analytics. But so, I want to push on this notion because people think of this the converged database, and what you're talking about here with HeatWave is sort of the Swiss army knife which is great 'cause you got a screwdriver and you got Phillips and a flathead and some scissors, maybe they're not as good. They're not as good necessarily as the purpose-built tool. But you're arguing that this is best of breed for OLTP and best of breed for analytics, both in terms of performance and cost. Am I getting that right or is this really a Swiss army knife where that flathead is really not as good as the big, long screwdriver that I have in my bag? >> Yes, so, you're getting it right but I did want to make a clarification. That HeatWave is definitely the accelerator for all your queries, all analytic queries and also for the long running complex transaction processing inquiries. So yes, HeatWave the uber query accelerator engine. However, when it comes to transaction processing in terms of your insert statements, delete statements, those are still all done and served by the MySQL database. So all, the transactions are still sent to the MySQL database and they're persistent there, it's the queries for which HeatWave is the accelerator. So what you said is correct. For all query acceleration, HeatWave is the engine. >> Makes sense. Okay, so if I'm a MySQL customer and I want to use HeatWave, what do I have to do? Do I have to make changes to my existing applications? You applied earlier that, no, it's just sort of plugs right in. But can you clarify that. >> Yes, there are absolutely no changes, which any MySQL or MySQL compatible application needs to make to take advantage of HeatWave. HeatWave is an in-memory accelerator and it's completely transparent to the application. So we have like, dozens and dozens of like, applications which have migrated to HeatWave, and they are seeing the same thing, similarly tools. So if you look at various tools which work for analytics like, Tableau, Looker, Oracle Analytics Cloud, all of them will work just seamlessly. And this is one of the reasons we had to do a lot of heavy lifting in the MySQL database itself. So the MySQL database engineering team was, has been very actively working on this. And one of the reasons is because we did the heavy lifting and we meet enhancements to the MySQL optimizer in the MySQL storage layer to do the integration of HeatWave in such a seamless manner. So there is absolutely no change which an application needs to make in order to leverage or benefit from HeatWave. >> You said earlier, Nipun, that you're seeing migrations from, I think you said Aurora and Google BigQuery, you might've said Redshift as well. Do you, what kind of tooling do you have to facilitate migrations? >> Right, now, there are multiple ways in which customers may want to do this, right? So the first tooling which we have is that customers, as I was talking about the replication or the inbound replication mechanism, customers can set up heat HeatWave in the Oracle Cloud and they can send the data, they can set up replication within their instances in their cloud and HeatWave. Second thing is we have various kinds of tools to like, facilitate the data migration in terms of like, fast ingestion sites. So there are a lot of such customers we are seeing who are kind of migrating and we have a plethora of like, tools and applications, in addition to like, setting up this inbound application, which is the most seamless way of getting customers started with HeatWave. >> So, I think you mentioned before, I have my notes, machine intelligence and machine learning. We've seen that with autonomous database it's a big, big deal obviously. How does HeatWave take advantage of machine intelligence and machine learning? >> Yeah, and I'm probably going to be talking more about this in the future, but what we have already is that HeatWave uses machine learning to intelligently automate many operations. So we know that when there's a service being offered in the cloud, our customers expect automation. And there're a lot of vendors and a lot of services which do a good job in automation. One of the places where we're going to be very unique is that HeatWave uses machine learning to automate many of these operations. And I'll give you one such example which is provisioning. Right now with HeatWave, when a customer wants to determine how many nodes are needed for running their workload, they don't need to make a guess. They invoke a provisioning advisor and this advisor uses machine learning to sample a very small percentage of the data. We're talking about, like, 0.1% sampling and it's able to predict the amount of memory with 95% accuracy, which this data is going to take. And based on that, it's able to make a prediction of how many servers are needed. So just a simple operation, the first step of provisioning, this is something which is done manually across, on any of the service, whereas at HeatWave, we have machine learning-based advisor. So this is an example of what we're doing. And in the future, we'll be offering many such innovations as a part of the MySQL Database and the HeatWave service. >> Well, I've got to say I was skeptic but I really appreciate it, you're, answering my questions. And, a lot of people when you made the acquisition and inherited MySQL, thought you were going to kill it because they thought it would be competitive to Oracle Database. I'm happy to see that you've invested and figured out a way to, hey, we can serve our community and continue to be the steward of MySQL. So Nipun, thanks very much for coming to the CUBE. Appreciate your time. >> Sure. Thank you so much for the time, Dave. I appreciate it. >> And thank you for watching everybody. This is Dave Vellante with another CUBE Conversation. We'll see you next time. (bright upbeat music)

>> Live from the Julia Morgan Ballroom in San Francisco. Extracting the signal from the noise, it's TheCUBE. Covering Structure 2015. Now your host, George Gilbert. >> This is George Gilbert, we're at Structure 2015. Reborn and really healthy from the old GigaOM, and we're pleased to welcome Alex Polvi from CoreOS, everyone seems to want to talk to Alex these days. So we've got first dibs. Alex why don't you tell us a little bit about CoreOS and why it's of such relevance right now. >> Sure, so we started CoreOS a little over two years ago, about two and a half years ago now. And our mission is to fundamentally improve the security of the internet. And our approach in doing that is to help companies run infrastructure in this way that allows it to be much more serviceable and have much better security and so on. This way that we're modeling looks a lot like what we've seen from the hyperscale companies. Folks like Google. So we often call it Google's Infrastructure For Everyone Else, GIFFY for short 'cause that's kind of a mouthful. And that involves distributed systems, containers, and running on standard hardware which in 2015 can be a bare-metal server, or could be an instance in AWS. >> Okay. So help us understand though that, if CoreOS, it sounds like there's an operating system at the core. >> Yeah. >> Is this like a cut down version of Linux that gives it a small attack surface and a sort of easier deployment and patching? >> Exactly, so in our quest to run the world servers to secure the internet we start at the lowest level component possible. There's the OS, then there's the distributed system side. So CoreOS is our company name, but it's also the name of the first product that we released, CoreOS Linux. CoreOS Linux is a lightweight container-based OS that automatically updates itself, 'cause we think that updates are the key to good security. So it's a combination of the updates, the container weight, the lightweight container-based application model. As well as just stripping everything else out. I mean the last 20 years of Linux distributions have created lots of cruft so it was time to kind of rebirth a new lightweight Linux OS. >> Sticking to CoreOS >> Yeah. >> For a moment, in an earlier era, might we have called this like an embedded OS where you just sort of chopped out everything that was not necessary for the application? >> Yeah, it's very much inspired by embedded OSes. On servers you know, you really want to get everything out of the way of the resources like the memory and CPU and so on so you get as much as you want out of it. So while it's a little bit counterintuitive, you have this really monster server, you still want as light and thin of an OS on there as you possibly can like an embedded OS so you can really maximize the performance. >> So something that abstracts the hardware but gets out of the way. >> Exactly. Just focus, get on the things that matter which is running your applications and managing the actual hardware and really nothing else. >> Okay, so, presumably to provide Google's infrastructure for everyone else, and I don't remember the acronym, >> GIFFY. >> Okay. What other products did you have to fill out to make that possible? >> Sure, great question. So the next major piece that we released is a tool called ETCD. It's meant for doing shared configuration amongst servers. Whenever you have a group of servers, the first thing you need to do is they all need to know about each other, and tell each other about the configuration. This is load balancers knowing where the app servers are, the app servers knowing where the databases are and so on. And to do this in the most robust distributed systems way, you have to do this thing in computer science that's very difficult called "consensus". Consensus algorithms is an area of computing, actually speaking about here in a little bit with Eric Brewer, who is a huge academic, a very well respected engineer in the area of consenus and distributed systems. And so we built ETCD, which solves this really hard distributed systems problem in a way that's usable by many operations teams. >> So let me just interrupt you for a second, >> Yeah. >> I mean I've got this sound going off in my head that says "Zookeeper, Zookeeper". >> Exactly. It's Zookeeper for everyone else. >> It's simplified. >> It's a simplified Zookeeper and make it accessible. Areas that a lot of people wanted to use distributed systems but Zookeeper is a little bit too difficult to use as well as really oriented toward the Java and Hadoop community, and there's a whole wide array of other folks out there. >> So it couldn't make as many constraining assumptions as yours, which would simplify. >> It just couldn't be as widely adopted. And so we released ETCD around the same time we released CoreOS Linux and this point, there's been over a thousand projects if you go on GitHub that have built things around ETCD, so our bet was right. Even things like Kubernetes, itself has a hard dependency on ETCD. Without ETCD, Kupernetes will not run. So our hypothesis there was let's make the hardest part of distributed systems easier, and then we will see distributed systems overall accelerate. And that is definitely what's happened with ETCD. >> Okay so help us understand, how you've built up the rest of the infrastructure and then where you'd like to see it go. >> Sure, so the thing that we're targeting is this distributed systems approach. And again we care about this a lot because we think that the ability to manage and service your applications, is what is the key to the security. Keeping things up to date, and when we mean up to date, we don't just mean like patch a vulnerability. Of which we've fixed many of those. But it's also about company's comfort rolling out a new version of their application that they won't break something. If you run your infrastructure in a distributed system, you can roll out a version, if it breaks a little bit of the application that's okay, but you didn't take the whole thing down. And that's kind of the safety net that distributed systems give you. >> Does this require the sort of micro-service approach where there's a clean separation between this new set of bits and the rest of the app? >> It really does. And that's why we've invested so heavily in containers. It requires a container, it also requires the distributed systems components of it. So we first built CoreOS Linux, then we built ETCD, then we started building some distributed systems work very early in the market. And then things like Kubernetes came along, and we were like, "Hey, instead of us reinventing all of this stuff let's partner up with the guys from the Google" if we're monitoring Google's infrastructure for everyone else, let's partner up with the team at Google that built that and get their solution more widely adopted out in the world as well. So the whole platform comes together as this combination of Kubernetes, ETCD, CoreOS Linux, we have our own container runtime called Rocket, which we built primarily to address some security issues in Docker. And so all of these pieces come together and what we call that piece when they're all together is Tectonic. Tectonic is our product that is that Google's infrastructure in a box. >> Okay let me just drop down in the weed for a sec. Derek Collison calls, I'm sorry I'm having a senior moment. And I hope it's not early onset Alzheimer's. The Docker, he calls sort of this generation's Tarball, you know, like to distribute you know, just a sort of I guess equivalent of an executable. Are you providing something that's compatible or does what's inside the container have to change to take advantage of the additional services that's sort of Google-centric. >> Sure. So the packaging, that Tarball piece, we're compatible with. And will always remain compatible with. To even further the compatibility, we've put together standards around what that container should be so many vendors can inter-operate more widely. We've done that first through the app container project and then more recently through the open container initiative which is a joint effort between Docker and us, and the rest of the ecosystem. And so we always, we always want the user to be able to package their application once and then choose whatever system they want to run it in, and the container is what really unlocks that portability. >> Okay. So then let me ask you, does the Google compute engine folks, or the passgroup, do they view you as a way of priming the pump outside the Google ecosystem to get others using their sort of application model or their infrastructure model? Because I'm trying to understand, you know Azure sort of has its own way of looking at the world, Amazon has its own way of looking at the world, are they looking at you as a way of sort of disseminating an approach to building applications? Or managing applications. >> Sure. So the Google team and their motivations behind Kubernetes, you'd have to talk to them about it. My understanding is that they see that as a way to have a very consistent environment between different cloud providers and so on. It is a next-generation way of running infrastructure as well, and its just better than the previous way of running infrastructure. >> That's sort of the answer I was looking for which is, they don't have to either give away their stuff or manage their infrastructure elsewhere. But you're sort of the channel to deliver Google style infrastructure in other environments. >> Sure, I mean Google Cloud's motivation at the end of the day is selling cores of memory. They put all these other services on top of it to make it, to make it more attractive to use, but the end of the day anything that drives more usage of these products is how they run their business. At least that's my perception of it. I'm obviously not speaking on behalf of Google. >> So where are you in attracting showcase customers? Guys who've sort of said "okay we'll bet", if not the entire business, "we'll bet the success of this application or these set of applications on this". >> Right, so first the technology's been very, very exciting. I mean the past two years we've seen this whole space explode in interest, but the discussion around "how does this solve business problems, how does this actually get adopted to these companies and what motivates them to actually do this" outside of the tech being very cool. That's a discussion that is just getting started and in fact in about two weeks here in early December in New York we're hosting that discussion at an event called the Tectonic Summit. The Tectonic Summit is where we're bringing together all the enterprise early adopters that are using containers, using distributed systems, and talking about why did their management and their leadership decide to make investments in these technologies. And what we're seeing are use cases about multi-data center between your physical data center and your cloud environments. We're seeing folks build their next-generation web services. Many businesses that weren't traditionally in the web services businesses need to be now because of mobile, just modern product offerings. And so we're hearing from these large guys and how they're using our technologies and other companies' technologies today to do this, and it's just two weeks at our event. >> Would it be fair to say, I'm listening to this and what seems to becoming across is that your technology makes it easier to abstract not just the machine, which would be CoreOS, but hybrid infrastructure. And it doesn't even have to be hybrid, it could be this data center and that data center. >> Right. >> Or your own data center and a public cloud. >> Exactly. One of the biggest value props of all this is the consistency between environments. We just give this compute, CPUs, memory, storage, we don't care if it's on cloud or if it's a physical data center, we can allow you to manage that in an extremely consistent way. Not just between your data centers but also between development and production, and that's a really important part of all of this. >> Do you need a point of view built into the infrastructure to make it palatable to developers who want a platform? As opposed to just infrastructure. >> Sure. So one of the things that's most exciting about this space is we're splitting the difference of platform and infrastructure. Platform is typically, platform is a service, this very prescriptive way of running your server infrastructure. And there's raw infrastructure which is a like, "here is a canvas, go to town but you need to bring all your own tools". What's happening right now in this distributed systems container space is a middle category. It's still infrastructure, but it's application focused. And at the end of the day that's what a developer is trying to do, is deploy their application out into the server infrastructure. >> So it doesn't have an opinion that tells the developer "we think you should build it this way", but it does hide all the sort of, the different types of hardware and their location pretty much. >> Right, it gives you a prescriptive way to how you package and deploy that, but doesn't put on any constraints of what you can package or deploy. >> Okay. Very interesting. It's sort of like a, if platform as a service was constraining because developers didn't want a straightjacket for how they should build the app, and infrastructures, our service was too raw. You're giving them a middle ground. >> Exactly. It's still infrastructure, but it's a consistent way of running that infrastructure. And that's why companies like Google and Facebook and Twitter do this, they have millions of servers and data centers all over the world. >> And they can't prescribe. >> Well they need to be able to have a consistent way of doing it so that they don't have to have an infinitely growing operations team as they scale their infrastructure. You need to have consistency, but at the same time you need to be able to have a wide array of tools and things to deploy and interact with that infrastructure. So it's that middle ground, and that's why the hyperscale guys have adopted it because they're forced to, because they have to have that consistency to have that scale. >> Okay let me ask you then, not on the, separate from the hyperscale guys, the sort of newest distributed system that mainstream enterprises are struggling with and sort of off the record, maybe choking on, you know is Hadoop. Because they haven't had to do elastic infrastructure before and like you said the Zookeeper is not that easy, and there's 22 other projects by the way that also have to get stood up. Can you help someone who is perhaps flailing in that or if not flailing, finding the skills overhead really, really tough? >> So, Hadoop. Let's remember Hadoop's roots. Where did that come from? >> Well Yahoo!. >> Well but where did Yahoo! get the idea? >> Oh yeah, Google, sorry. >> Exactly. Yahoo! gets all the credit for it. Even though it was a Google paper that was modeled after. And so again, if Kubernetes and containers and everything is the equivalent of Google's borg, which is that raw application infrastructure, Hadoop is a certain application that consumes the spare resources on that cluster in order to do these map reducing computational jobs. >> So the next question is, how much can you simplify what mainstream enterprises do that don't have the Google infrastructure yet? >> Right, so they have to manage that as its own whole separate thing. It's its own set of infrastructure, it's its own set of servers to manage their Hadoop cluster. If you combine it with this application infrastructure, we just treat Hadoop as another application that runs on the platform. It's not its own distinct, special thing. It's just another application running out there along with your web servers and your databases, and everything else, you have your Hadoop workload in the mix. So you have this consistent pool of infrastructure and Hadoop is just another application that's monitored or managed the exact same way as everything else. >> So, for folks who are a little more familiar with Mesos, which is the opposite of a virtual machine, it makes many machines look like a single one, I assume. >> Well this is a very similar message to Mesos. Mesos is also building Google-like infrastructure for everyone else. The difference with what we're doing is really we just partnered up with the team that built that at Google, and focusing our solution around Kubernetes which is what the Google efforts are behind. So we're all modeling Google's infrastructure. >> Okay. >> Mesos took their own spin on it with Kubernetes, and CoreOS and ETCD, we're taking a different spin on it. >> So and what other products have you built out that we haven't touched on, and what do you see the roadmap looking like? >> Sure, so really, all these things we've talked about are open source projects. They're all components for building this Google-like infrastructure. Tectonic is our platform for companies that want this style of infrastructure but they don't want to have to figure out all the different pieces themselves. And we think once companies adopt Tectonic, just this general style of infrastructure, that we can give them all the benefits of this, better utilization, that consistency, easier management of lots and lots of servers and so on. But we also think we can dramatically improve the security of their infrastructure as well. And that's what we're investing in our roadmap is to leverage this kind of change, and then with that change we can do some things to the infrastructure that was never possible before. >> Okay. >> And that's the things that we're investing in as a company. >> Okay, great. We're going to break at that, this is George Gilbert, at Structure '15, with Alex Polvi of CoreOS. And we'll be back in just a few minutes. (light music)