>> Live from Seattle, Washington, it's theCUBE covering KubeCon and CloudNativeCon, North America 2018, brought to you by Redhat, the Cloud Native Computing Foundation and it's ecosystem partners. >> Okay, welcome back, everyone, this is theCUBE's live coverage here in Seattle for KubeCon and CloudNativeCon 2018. I'm John Furrier with Stu Miniman breaking down all the action, talking to all the top people, influencers, executives, start-ups, vendors, the foundation itself. We're here with two co-leads of Kubernetes at Google, legends in the Kubernetes industry. Tim Hockin and Brian Grant, both with Google, both co-leads at GKE. Thanks for joining us, legends in the industry. Kubernetes is still a short life, but still, being there from the beginning, you guys were instrumental at Google building out and contributing to this massive tsunami of 8000 people here. Who would have thought? >> It's amazing! >> It's a little overwhelming. >> It's almost like you guys are celebrity-status here inside this crowd. How's that feel? >> It's a little weird. I don't buy into the celebrity culture for technologists. I don't think it works well. >> We agree, but it's great to have you on. Let's get down to it. Kubernetes, certainly the rise of Kubernetes has grown. It's now pretty mainstream, people look at that as a key linchpin for the center of Cloud Native. And we see the growth of Cloud, you guys are living it with Google. What is the importance of Kubernetes? Why is it so important? Fundamentally at it's core, has a lot of impact, what's the fundamental reason why it's so successful? >> I think fundamentally Kubernetes provides a framework for driving migration towards Cloud Native patterns across your entire operational infrastructure. The basic design of Kubernetes is pretty simple and can be applied to automating pretty much anything. We're seeing that here, there are at least more than half a dozen talks about how people are using the Kubernetes to control plane to manage their applications or workflows or functions or things other than just core Kubernetes, containers, for example. Cloud Native is about... One of the things I'm involved with is I'm on the Technical Oversight Committee of the Cloud Native Computing Foundation. I drove the update of the Cloud Native definition. If you're trying to operate with high velocity, deploying many times a day, if you're trying to operate at scale, especially with containers and functions, scale is increasing and compounding as people break their applications into more and more micro services. Kubernetes really provides the framework for managing that scale and for integrating other infrastructure that needs to accommodate that scale and that pace of change. >> I think Kubernetes speaks to the pain points that users are really having today. Everybody's a software company now, right? And they have to deploy their software, they have to build their software, they have to run their software, and these things, they build up pain. When it was just a little thing, you didn't have to worry about scale, internet-scale and web-scale, you could tolerate it within your organization. But more and more, you need to deploy faster, you need to automate things. You can't afford to have giant staffs of people who are running your applications. These things are all part of Kubernetes purvey. I think it just spoke to people in a way, they said I suffer from that every day and you just made it go away. >> And what's the core impact now? Because then now people are seeing it, what is the impact to the organizations that are rethinking their entire operation from all parts of the staff, from how they buy infrastructure, which is also Cloud, you see some Cloud there, and then that deploying applicant, what's the real impact? >> I think the most obvious, the most important part here is the way it changes how people operate and how they think about how they manage systems. It no longer becomes scary to update your application. It's just a thing you do. If you can do it with high confidence, you're going to do it more often, which means you get features and bugs fixed and you get your roll-outs done quicker. It's amazing, the result that it can have on the user experience. A user reports a bug in the morning, and you fix it in the afternoon, and you don't worry about that. >> You bring up some really interesting points. I think back 10 years ago, from a research standpoint, we were looking at how can the enterprise do some of the things that the hyperscale vendors were doing. I feel over the last 10 years, every time Google released one of the great scientific papers, we'd all get a peer inside and say like, oh hey. When I went to the first DockerCon and heard how Google was using containers, when Kubernetes first came out, it's like, oh wow, maybe the rest of us will get to do something that Google's been doing for the last 10 years. Maybe bring us back a little bit to Borg and how that led to Kubernetes. Are we still all the rest of us just doing whatever Google did 10 years ago? >> Yeah, Tim and I both worked on Borg previously, Tim on the node-agent side and I worked on the control-point side in Borg One lesson we really took from Borg is that really you can run all types of applications. People started with stateless applications and we started with that because it's simpler in Kubernetes. But really it's just a general management control plane for managing applications. With the model of one application per container, then you can manage the applications in a much more first-class way and unlock a lot of opportunities for automation in the management control plane. At Google, several years ago when we started, Google had already gone through the transition of moving most of its applications to Borg. It was after that phase that Google started its Cloud effort and the rest of the world was doing VMs. When Docker emerged, we were... In the early phases, Tim mentioned this in our keynote yesterday of open-sourcing our container runtime. When Docker emerged, it is clear it had a much better user experience for the way folks were managing applications outside of Google and we just pivoted to that immediately. >> When Docker first came out, we took a look at it, we, my node-agent team in Borg, and we went, yeah, it's kind of like poor man's version of Borglet. We sort of ignored it for awhile because we were already working on our open-source effort. We were open-sourcing it, not really to change the world and make everybody use it, but more so that we can have conversations with people like the Linux kernel community. When we said we need this feature, and they'd say well why, why do you need this, we could actually demonstrate for them why we needed it. When Docker landed, we saw the community building, and building, and building. That was a snowball of its own, right? As it caught on, we realized we know what this is going to. We know once you embrace the Docker mindset that you very quickly need something to manage all of your Docker nodes once you get beyond two or three of them. We know how to build that. We got a ton of experience here. We went to our leadership and said, please, this is going to happen with us or without us and I think the world would be better if we helped. >> I think that's an interesting point. You guys had to open-source to do collaboration with Linux to get that flywheel going for you guys out of necessity. Then when Docker validated the community acceptance of hey, we can just use containers, a lot of magic will happen, it hit the second trigger point. What happened after that? You guys just had a debate internally? Is this another MapReduce? What's happening? Like, we should get behind this. I knew there was a big argument or debate, I should say, within Google. At that time there were a lot of conversations, how do we handle this? >> That was around the time that Google Compute Engine, our infrastructures and service platform, was going GA and really starting to get usage. So then we had an opportunity to enable our customers to benefit from the kinds of techniques we had been using internally. So I don't think the debate was whether we should participate, it was more how. For example, should we have a fully managed product, should we have to do open-source, should we do managed open-source, so those were really the three alternatives that we were discussing. >> Well, congratulations, you guys done great work and certainly a huge impact to the industry. I think it's clear that the motivation to have some sort of standardization, de facto standard, whatever word can be used to kind of let people be enabled on top or below Kubernetes is great. I guess the next question is how do you guys envision this going forward as a core? If we're going to go to decomposition with low levels of granularity tying together through the network and cloud-scale and the new operating law, we'll have comments in this, how does the industry maintain the greatness of what Kubernetes is delivering and bring new things to market faster? What's your vision on this? >> I talked a little bit about this this week. We put a ton of work into extension points, extensibility of the system trying to stay very true to the original vision of Kubernetes. It is a box, and Kubernetes fits inside a box, and anything that's outside the box has to stay outside the box. This gives us the opportunity to build new ecosystems. You can see it in networking space, you can see it in storage space where whole sort of cottage industries are now springing up around doing networking for Kubernetes and doing storage for Kubernetes. And that's fantastic! You see projects like Istio, which I'm a big fan of, it's outside of Kubernetes. It works really well with Kubernetes, it's designed on top of Kubernetes infrastructure, but it's not Kubernetes. It's totally removable and you don't need it. There's systems like Knative which are taking the serverless idea and upleveling Kubernetes into serverless space. It's happening all over the place. We're trying to sort of pray fanatically, say, no, we're staying this big and no bigger. >> It's a really... From an engineering standpoint, it's much simpler if I just build a product and build everything into it. All those connection points, I go back to my engineering training. It's like every connection point is going to be another place where it could fail. Now it's got all these APIs, there's all the security issues, and things like that. But what I love what I heard right here is some of the learnings that we've had in open-source is these are all of these individual components that most of them can stand on their own. They don't even have to be with Kubernetes, but altogether you can build lots of different offerings. How do you balance that? How do you look at that from kind of a design and architecture standpoint? >> So one thing I've been looking at is how do we ensure compatibility of workloads across Kubernetes in all different environments and different configurations. How do we ensure that the tools and other systems building an ecosystem work with Kubernetes everywhere? So this is why we created the Conformance Program to certify that the critical APIs that everybody depends on behave the same way. As we try to improve the test coverage of the conformance, people are focusing on these areas of the system that are highly pluggable and extensible. So for example, the kubelet in the node has a pluggable container runtime, pluggable networks, pluggable storage systems now with CSI. So we're really focusing on ensuring we have good coverage of the Pod API, for example. And other parts of the system, people have swapped out an ecosystem, whether it's kube-proxy for our Kubernetes services or the scheduler. So we'll be working through those areas to make sure that they have really good coverage so users can deploy, say, a Helm Chart or their takes on a configuration or whatever, however they manage their applications and have that behave the same way on Kubernetes everywhere. >> I think you guys have done a great job of identifying this enabling concept. What is good enabling technology? Allowing others to do innovation around it. I think that's a nice positioning. What are the new problem areas that you guys see to work on next? Now I see things are developing in the ecosystem. You mentioned the Istio service mesh and people see value in that. Security is certainly a big conversation we've been having this week. What new problem areas or problem sets you guys see emerging that are needed to just tackle and just knock down right away? >> The most obvious, the thing that comes up sort of in every conversation of users now is multi-cluster, multi-cloud, hybrid, whether that's two clouds or on-prem plus cloud or even across different data centers on your premises. It's a hard topic. For a long time Kubernetes was able to sort of put a finger in our ears and pretend it didn't exist while we built out the Kubernetes model. Now we're at a place where we've crossed the adoption chasm. We're into the real adoption now. It's a real problem. It actually exists and we have to deal with it, and so we're now looking at how's it supposed to work. Philosophically, what do we think is supposed to happen here? Technologically, how do we make it happen? How do these pieces fit together? What primitives can we bring into Kubernetes to make these higher level systems possible? >> Would you consider 2019 to be the year of multi-cloud, in terms of the evolution of trying to tackle some of these things from latency? >> Yeah, I'm always reluctant to say the year of something because... >> Someone has to get killed, and someone dies, and someone's winning. >> It's the year of the last desktop. >> It's the year of something. (laughs) EDI, I'm just saying. >> I think multi-cluster is definitely the hot topic right now. It's certainly almost every customer that we talk to through Google and tons of community chatter about how to make this work. >> You've seen companies like NetApp and Cisco, for instance, and how they're been getting a tail-wind from the Kubernetes. It's been interesting. You need networks. They have a lot of networks. They can play a role in it. So it's interesting how it's designed to allow people to put their hands in there without kind of mucking up the main... >> Yeah, I think that really contributes to the success of Kubernetes, the more people that can help add value to Kubernetes, more people have a stake in the success of Kubernetes, both users and vendors, and developers, and contributors. We're all stakeholders in this endeavor now and we all share common goals, I think. >> Well guys, final question for you. I know we got to break on time. Thanks for coming. I really appreciate the time. Talk about an area of Kubernetes that most people should know about that might not know about. In other words, there was a lot of hype around Kubernetes, and it's warranted, it's a lot of buzz, what's an important area that's not talked about much that people should know more about it and pay attention to within the Kubernetes realms of that world? Is there any area that you think is not talked about enough that should be focused on in the conversations, the press, or just in general? >> Wow, that's a challenging question. I spent a lot of my time in the infrastructure side of Kubernetes, the lower end of the stack, so my brain immediately goes to networking and storage and all the lower level pieces there. I think there's a lot of policy knobs that Kubernetes has that not everybody's aware of, whether those are security policies or network policies. There's a whole family of these things and I think we're going to continue to acree more and more policy as more people come up with real-use cases for doing stuff. It's hard to keep that all in your mind, but it's really valuable stuff down there. >> For programmability, it's like a Holy Grail, really. Thoughts on the things that (chuckles) put you on the spot there? >> I think this question of how people should change what they were doing before if they're going to migrate to Kubernetes. To operate any workload, you need at least monitoring and you need really CI/CD if you want to operate with any amount of velocity. When you bring those practices to Kubernetes, should you just lift and shift those into Kubernetes or do you really need to change your mindset? I think Kubernetes really provides some capabilities that create opportunities for changing the way some things happen. I'm a big fan of GitOps, for example, in managing the resources to declaritively using version control as a source of truth and keeping that in sync with the state in your for live clusters. I think that enables a lot of interesting capabilities like instant disaster recovery, for example, migrations, new locations. There are some key folks here who are talking about that, giving that message, but we're really at the early stages there. >> All right, well great to have you guys on. Thanks for the insight. We've got to wrap up. Thanks Brian, thanks Tim, appreciate it. Live coverage here, theCUBE is at KubeCon, Cloud Native, Cloud 2018. I'm John Furrier with Stu Miniman, we'll be back after this short break.

>> From theCUBE studios in Palo Alto, in Boston, bringing you data driven insights from theCUBE and ETR. This is Breaking Analysis with Dave Vollante. >> The rise of Kubernetes came about through a combination of forces that were, in hindsight, quite a long shot. Amazon's dominance created momentum for Cloud native application development, and the need for newer and simpler experiences, beyond just easily spinning up computer as a service. This wave crashed into innovations from a startup named Docker, and a reluctant competitor in Google, that needed a way to change the game on Amazon and the Cloud. Now, add in the effort of Red Hat, which needed a new path beyond Enterprise Linux, and oh, by the way, it was just about to commit to a path of a Kubernetes alternative for OpenShift and figure out a governance structure to hurt all the cats and the ecosystem and you get the remarkable ascendancy of Kubernetes. Hello and welcome to this week's Wikibon CUBE Insights powered by ETR. In this breaking analysis, we tapped the back stories of a new documentary that explains the improbable events that led to the creation of Kubernetes. We'll share some new survey data from ETR and commentary from the many early the innovators who came on theCUBE during the exciting period since the founding of Docker in 2013, which marked a new era in computing, because we're talking about Kubernetes and developers today, the hoodie is on. And there's a new two part documentary that I just referenced, it's out and it was produced by Honeypot on Kubernetes, part one and part two, tells a story of how Kubernetes came to prominence and many of the players that made it happen. Now, a lot of these players, including Tim Hawkin Kelsey Hightower, Craig McLuckie, Joe Beda, Brian Grant Solomon Hykes, Jerry Chen and others came on theCUBE during formative years of containers going mainstream and the rise of Kubernetes. John Furrier and Stu Miniman were at the many shows we covered back then and they unpacked what was happening at the time. We'll share the commentary from the guests that they interviewed and try to add some context. Now let's start with the concept of developer defined structure, DDI. Jerry Chen was at VMware and he could see the trends that were evolving. He left VMware to become a venture capitalist at Greylock. Docker was his first investment. And he saw the future this way. >> What happens is when you define infrastructure software you can program it. You make it portable. And that the beauty of this cloud wave what I call DDI's. Now, to your point is every piece of infrastructure from storage, networking, to compute has an API, right? And, and AWS there was an early trend where S3, EBS, EC2 had API. >> As building blocks too. >> As building blocks, exactly. >> Not monolithic. >> Monolithic building blocks every little building bone block has it own API and just like Docker really is the API for this unit of the cloud enables developers to define how they want to build their applications, how to network them know as Wills talked about, and how you want to secure them and how you want to store them. And so the beauty of this generation is now developers are determining how apps are built, not just at the, you know, end user, you know, iPhone app layer the data layer, the storage layer, the networking layer. So every single level is being disrupted by this concept of a DDI and where, how you build use and actually purchase IT has changed. And you're seeing the incumbent vendors like Oracle, VMware Microsoft try to react but you're seeing a whole new generation startup. >> Now what Jerry was explaining is that this new abstraction layer that was being built here's some ETR data that quantifies that and shows where we are today. The chart shows net score or spending momentum on the vertical axis and market share which represents the pervasiveness in the survey set. So as Jerry and the innovators who created Docker saw the cloud was becoming prominent and you can see it still has spending velocity that's elevated above that 40% red line which is kind of a magic mark of momentum. And of course, it's very prominent on the X axis as well. And you see the low level infrastructure virtualization and that even floats above servers and storage and networking right. Back in 2013 the conversation with VMware. And by the way, I remember having this conversation deeply at the time with Chad Sakac was we're going to make this low level infrastructure invisible, and we intend to make virtualization invisible, IE simplified. And so, you see above the two arrows there related to containers, container orchestration and container platforms, which are abstraction layers and services above the underlying VMs and hardware. And you can see the momentum that they have right there with the cloud and AI and RPA. So you had these forces that Jerry described that were taking shape, and this picture kind of summarizes how they came together to form Kubernetes. And the upper left, Of course you see AWS and we inserted a picture from a post we did, right after the first reinvent in 2012, it was obvious to us at the time that the cloud gorilla was AWS and had all this momentum. Now, Solomon Hykes, the founder of Docker, you see there in the upper right. He saw the need to simplify the packaging of applications for cloud developers. Here's how he described it. Back in 2014 in theCUBE with John Furrier >> Container is a unit of deployment, right? It's the format in which you package your application all the files, all the executables libraries all the dependencies in one thing that you can move to any server and deploy in a repeatable way. So it's similar to how you would run an iOS app on an iPhone, for example. >> A Docker at the time was a 30% company and it just changed its name from .cloud. And back to the diagram you have Google with a red question mark. So why would you need more than what Docker had created. Craig McLuckie, who was a product manager at Google back then explains the need for yet another abstraction. >> We created the strong separation between infrastructure operations and application operations. And so, Docker has created a portable framework to take it, basically a binary and run it anywhere which is an amazing capability, but that's not enough. You also need to be able to manage that with a framework that can run anywhere. And so, the union of Docker and Kubernetes provides this framework where you're completely abstracted from the underlying infrastructure. You could use VMware, you could use Red Hat open stack deployment. You could run on another major cloud provider like rec. >> Now Google had this huge cloud infrastructure but no commercial cloud business compete with AWS. At least not one that was taken seriously at the time. So it needed a way to change the game. And it had this thing called Google Borg, which is a container management system and scheduler and Google looked at what was happening with virtualization and said, you know, we obviously could do better Joe Beda, who was with Google at the time explains their mindset going back to the beginning. >> Craig and I started up Google compute engine VM as a service. And the odd thing to recognize is that, nobody who had been in Google for a long time thought that there was anything to this VM stuff, right? Cause Google had been on containers for so long. That was their mindset board was the way that stuff was actually deployed. So, you know, my boss at the time, who's now at Cloudera booted up a VM for the first time, and anybody in the outside world be like, Hey, that's really cool. And his response was like, well now what? Right. You're sitting at a prompt. Like that's not super interesting. How do I run my app? Right. Which is, that's what everybody's been struggling with, with cloud is not how do I get a VM up? How do I actually run my code? >> Okay. So Google never really did virtualization. They were looking at the market and said, okay what can we do to make Google relevant in cloud. Here's Eric Brewer from Google. Talking on theCUBE about Google's thought process at the time. >> One interest things about Google is it essentially makes no use of virtual machines internally. And that's because Google started in 1998 which is the same year that VMware started was kind of brought the modern virtual machine to bear. And so Google infrastructure tends to be built really on kind of classic Unix processes and communication. And so scaling that up, you get a system that works a lot with just processes and containers. So kind of when I saw containers come along with Docker, we said, well, that's a good model for us. And we can take what we know internally which was called Borg a big scheduler. And we can turn that into Kubernetes and we'll open source it. And suddenly we have kind of a cloud version of Google that works the way we would like it to work. >> Now, Eric Brewer gave us the bumper sticker version of the story there. What he reveals in the documentary that I referenced earlier is that initially Google was like, why would we open source our secret sauce to help competitors? So folks like Tim Hockin and Brian Grant who were on the original Kubernetes team, went to management and pressed hard to convince them to bless open sourcing Kubernetes. Here's Hockin's explanation. >> When Docker landed, we saw the community building and building and building. I mean, that was a snowball of its own, right? And as it caught on we realized we know what this is going to we know once you embrace the Docker mindset that you very quickly need something to manage all of your Docker nodes, once you get beyond two or three of them, and we know how to build that, right? We got a ton of experience here. Like we went to our leadership and said, you know, please this is going to happen with us or without us. And I think it, the world would be better if we helped. >> So the open source strategy became more compelling as they studied the problem because it gave Google a way to neutralize AWS's advantage because with containers you could develop on AWS for example, and then run the application anywhere like Google's cloud. So it not only gave developers a path off of AWS. If Google could develop a strong service on GCP they could monetize that play. Now, focus your attention back to the diagram which shows this smiling, Alex Polvi from Core OS which was acquired by Red Hat in 2018. And he saw the need to bring Linux into the cloud. I mean, after all Linux was powering the internet it was the OS for enterprise apps. And he saw the need to extend its path into the cloud. Now here's how he described it at an OpenStack event in 2015. >> Similar to what happened with Linux. Like yes, there is still need for Linux and Windows and other OSs out there. But by and large on production, web infrastructure it's all Linux now. And you were able to get onto one stack. And how were you able to do that? It was, it was by having a truly open consistent API and a commitment into not breaking APIs and, so on. That allowed Linux to really become ubiquitous in the data center. Yes, there are other OSs, but Linux buy in large for production infrastructure, what is being used. And I think you'll see a similar phenomenon happen for this next level up cause we're treating the whole data center as a computer instead of trading one in visual instance is just the computer. And that's the stuff that Kubernetes to me and someone is doing. And I think there will be one that shakes out over time and we believe that'll be Kubernetes. >> So Alex saw the need for a dominant container orchestration platform. And you heard him, they made the right bet. It would be Kubernetes. Now Red Hat, Red Hat is been around since 1993. So it has a lot of on-prem. So it needed a future path to the cloud. So they rang up Google and said, hey. What do you guys have going on in this space? So Google, was kind of non-committal, but it did expose that they were thinking about doing something that was you know, pre Kubernetes. It was before it was called Kubernetes. But hey, we have this thing and we're thinking about open sourcing it, but Google's internal debates, and you know, some of the arm twisting from the engine engineers, it was taking too long. So Red Hat said, well, screw it. We got to move forward with OpenShift. So we'll do what Apple and Airbnb and Heroku are doing and we'll build on an alternative. And so they were ready to go with Mesos which was very much more sophisticated than Kubernetes at the time and much more mature, but then Google the last minute said, hey, let's do this. So Clayton Coleman with Red Hat, he was an architect. And he leaned in right away. He was one of the first outside committers outside of Google. But you still led these competing forces in the market. And internally there were debates. Do we go with simplicity or do we go with system scale? And Hen Goldberg from Google explains why they focus first on simplicity in getting that right. >> We had to defend of why we are only supporting 100 nodes in the first release of Kubernetes. And they explained that they know how to build for scale. They've done that. They know how to do it, but realistically most of users don't need large clusters. So why create this complexity? >> So Goldberg explains that rather than competing right away with say Mesos or Docker swarm, which were far more baked they made the bet to keep it simple and go for adoption and ubiquity, which obviously turned out to be the right choice. But the last piece of the puzzle was governance. Now Google promised to open source Kubernetes but when it started to open up to contributors outside of Google, the code was still controlled by Google and developers had to sign Google paper that said Google could still do whatever it wanted. It could sub license, et cetera. So Google had to pass the Baton to an independent entity and that's how CNCF was started. Kubernetes was its first project. And let's listen to Chris Aniszczyk of the CNCF explain >> CNCF is all about providing a neutral home for cloud native technology. And, you know, it's been about almost two years since our first board meeting. And the idea was, you know there's a certain set of technology out there, you know that are essentially microservice based that like live in containers that are essentially orchestrated by some process, right? That's essentially what we mean when we say cloud native right. And CNCF was seated with Kubernetes as its first project. And you know, as, as we've seen over the last couple years Kubernetes has grown, you know, quite well they have a large community a diverse con you know, contributor base and have done, you know, kind of extremely well. They're one of actually the fastest, you know highest velocity, open source projects out there, maybe. >> Okay. So this is how we got to where we are today. This ETR data shows container orchestration offerings. It's the same X Y graph that we showed earlier. And you can see where Kubernetes lands not we're standing that Kubernetes not a company but respondents, you know, they doing Kubernetes. They maybe don't know, you know, whose platform and it's hard with the ETR taxon economy as a fuzzy and survey data because Kubernetes is increasingly becoming embedded into cloud platforms. And IT pros, they may not even know which one specifically. And so the reason we've linked these two platforms Kubernetes and Red Hat OpenShift is because OpenShift right now is a dominant revenue player in the space and is increasingly popular PaaS layer. Yeah. You could download Kubernetes and do what you want with it. But if you're really building enterprise apps you're going to need support. And that's where OpenShift comes in. And there's not much data on this but we did find this chart from AMDA which show was the container software market, whatever that really is. And Red Hat has got 50% of it. This is revenue. And, you know, we know the muscle of IBM is behind OpenShift. So there's really not hard to believe. Now we've got some other data points that show how Kubernetes is becoming less visible and more embedded under of the hood. If you will, as this chart shows this is data from CNCF's annual survey they had 1800 respondents here, and the data showed that 79% of respondents use certified Kubernetes hosted platforms. Amazon elastic container service for Kubernetes was the most prominent 39% followed by Azure Kubernetes service at 23% in Azure AKS engine at 17%. With Google's GKE, Google Kubernetes engine behind those three. Now. You have to ask, okay, Google. Google's management Initially they had concerns. You know, why are we open sourcing such a key technology? And the premise was, it would level the playing field. And for sure it has, but you have to ask has it driven the monetization Google was after? And I would've to say no, it probably didn't. But think about where Google would've been. If it hadn't open source Kubernetes how relevant would it be in the cloud discussion. Despite its distant third position behind AWS and Microsoft or even fourth, if you include Alibaba without Kubernetes Google probably would be much less prominent or possibly even irrelevant in cloud, enterprise cloud. Okay. Let's wrap up with some comments on the state of Kubernetes and maybe a thought or two about, you know, where we're headed. So look, no shocker Kubernetes for all its improbable beginning has gone mainstream in the past year or so. We're seeing much more maturity and support for state full workloads and big ecosystem support with respect to better security and continued simplification. But you know, it's still pretty complex. It's getting better, but it's not VMware level of maturity. For example, of course. Now adoption has always been strong for Kubernetes, for cloud native companies who start with containers on day one, but we're seeing many more. IT organizations adopting Kubernetes as it matures. It's interesting, you know, Docker set out to be the system of the cloud and Kubernetes has really kind of become that. Docker desktop is where Docker's action really is. That's where Docker is thriving. It sold off Docker swarm to Mirantis has made some tweaks. Docker has made some tweaks to its licensing model to be able to continue to evolve its its business. To hear more about that at DockerCon. And as we said, years ago we expected Kubernetes to become less visible Stu Miniman and I talked about this in one of our predictions post and really become more embedded into other platforms. And that's exactly what's happening here but it's still complicated. Remember, remember the... Go back to the early and mid cycle of VMware understanding things like application performance you needed folks in lab coats to really remediate problems and dig in and peel the onion and scale the system you know, and in some ways you're seeing that dynamic repeated with Kubernetes, security performance scale recovery, when something goes wrong all are made more difficult by the rapid pace at which the ecosystem is evolving Kubernetes. But it's definitely headed in the right direction. So what's next for Kubernetes we would expect further simplification and you're going to see more abstractions. We live in this world of almost perpetual abstractions. Now, as Kubernetes improves support from multi cluster it will be begin to treat those clusters as a unified group. So kind of abstracting multiple clusters and treating them as, as one to be managed together. And this is going to create a lot of ecosystem focus on scaling globally. Okay, once you do that, you're going to have to worry about latency and then you're going to have to keep pace with security as you expand the, the threat area. And then of course recovery what happens when something goes wrong, more complexity, the harder it is to recover and that's going to require new services to share resources across clusters. So look for that. You also should expect more automation. It's going to be driven by the host cloud providers as Kubernetes supports more state full applications and begins to extend its cluster management. Cloud providers will inject as much automation as possible into the system. Now and finally, as these capabilities mature we would expect to see better support for data intensive workloads like, AI and Machine learning and inference. Schedule with these workloads becomes harder because they're so resource intensive and performance management becomes more complex. So that's going to have to evolve. I mean, frankly, many of the things that Kubernetes team way back when, you know they back burn it early on, for example, you saw in Docker swarm or Mesos they're going to start to enter the scene now with Kubernetes as they start to sort of prioritize some of those more complex functions. Now, the last thing I'll ask you to think about is what's next beyond Kubernetes, you know this isn't it right with serverless and IOT in the edge and new data, heavy workloads there's something that's going to disrupt Kubernetes. So in that, by the way, in that CNCF survey nearly 40% of respondents were using serverless and that's going to keep growing. So how is that going to change the development model? You know, Andy Jassy once famously said that if they had to start over with Amazon retail, they'd start with serverless. So let's keep an eye on the horizon to see what's coming next. All right, that's it for now. I want to thank my colleagues, Stephanie Chan who helped research this week's topics and Alex Myerson on the production team, who also manages the breaking analysis podcast, Kristin Martin and Cheryl Knight help get the word out on socials, so thanks to all of you. Remember these episodes, they're all available as podcasts wherever you listen, just search breaking analysis podcast. Don't forget to check out ETR website @etr.ai. We'll also publish. We publish a full report every week on wikibon.com and Silicon angle.com. You can get in touch with me, email me directly david.villane@Siliconangle.com or DM me at D Vollante. You can comment on our LinkedIn post. This is Dave Vollante for theCUBE insights powered by ETR. Have a great week, everybody. Thanks for watching. Stay safe, be well. And we'll see you next time. (upbeat music)

>> From Seattle, Washington, it's theCUBE. Covering KubeCon and CloudNativeCon North America 2018. Brought to you by Red Hat. The Cloud Native Computing Foundation and it's ecosystem partners. [techno Music] >> Okay, welcome back everyone. It's theCUBE's live coverage in Seattle for KubeCon and CloudNativeCon 2018. I'm John Furrier with theCUBE. Stu Miniman. Breaking down all the action. Talking to all the thought leaders, all the experts, all the people making it happen. We're here with Aparna Sinha who's the group product manager, Kubernetes, Google Cloud. Also one of the power women of the Cloud at Google, according the Forbes. I wrote the story. Great to see you again. >> Thank you, great to be here with you. >> Thanks for coming on. >> CUBE alumni. Great to have you on. I want to get your prospective. One when you've seen a lot of action, certainly overseeing the group engineering team at Google and all the Kubernetes action. A lot of contribution, a lot of activity, that you guys are leading. >> Yes. >> And quite frankly enabling and contributing to the community. So, congratulations and thanks for that work. Kubernetes certainly looking good. People are pumped up. >> Very much. >> 8,000 people. A lot of activity. A lot of new things around that you guys are always kind of bringing into, the Geo, knative, a lot things. You gave a key note. What's your focus here this year? What's the message from Google? >> Yeah, well as you pointed out, this is the largest KubeCon ever. 8,000 people, 2,000 on the wait list. And people are telling me here that this is the... This is here to stay, right? It's in the early majority going to the mainstream very much like you kind of think about virtualization was 10 years ago. So that's the momentum that I'm seeing here, that I'm hearing here. My keynote was about the community. Thanking the community first of all. So I talked about how open-source really, success in contingent on contribution. And so, I started by showing the contribution over the last one year, the companies that are contributing. And 80% of contributions are by at least 10 entities. One of them is individual contributors. 40% percent I think was Google, which is still staggeringly high. And then the next highest was Red Hat. And so I think in many of the keynotes, we've been calling out the contributors because it's really important. 1.13, the 13th release of Kubernetes shipped last week. A lot of stability, a lot of GA features, and the uptake in the enterprise. The other thing I called out was just the amount of job opportunity in Kubernetes >> Yeah >> 230% growth in the last year. You see here so many customers that are here to talk about their experience. But also they're here to hire. >> Yeah. And there recruiters on the floor, so it's been I think a huge economic value add. And we feel very proud of that. >> Yeah, Aparna, great point. We've been talking about the end users. I always loved... There's a job board right outside the hall here and it's just covered. Big giant white board there. Bring us inside a little bit. I mean Google's always fascinating people. What's the hiring situation there? What's your team lookin' like? Is anybody smart enough to actually go work there? >> Google, I think we've been very, very fortunate in that we've had the original board team that started the Kubernetes project. And so we have a really, really deep bench because we've been running containers since the beginning. So now 15 years of experience with that, which many people tell me, I think that the reason that Kubernetes is so successful is because it's not new actually, right? >> Yeah >> It's been tried and true at scale. So, we have quite a bit of that, but we've been building this community and a lot of folks have been hired in through the community-- >> Yeah >> into Google. And really amazing, amazing people. So yeah. >> The thing about we had Brian Grant on yesterday and Tim Hockin -- Yes. >> Who was talking about some of those early board days. >> Yes. I want to ask you your point of about the hiring because I think this is a interesting dynamic. Open-source is key to your strategy. We've talked many times about how you guys are committed to open source, but what's interesting is not just net new jobs are available, we're seeing a revitalization around traditional roles like the network engineer under Kubernetes. Looking at the policy knobs that your folks pointed out that's... They think it's underutilized. And then on top of Kubernetes, new things are going on that's getting the app kind of server guy-- >> Yeah. >> Kind of energized. >> Yeah. >> It's kind of enabling a lot of thing, actions that's transforming existing jobs. >> That's right. >> And bringing new ones. >> Talk about that dynamic because you see it from both sides. >> Yes >> You've got SREs, site reliable engineers. >> Yes >> You've got developers. But, Now enterprises are now trying to adopt... >> That's right >> You guys are hitting that note. Talk about that dynamic. >> That's right, so I've been talking to a lot of customers here, it's been non-stop. I've not been able to attend any talks or keynotes. And I'm seeing two things. One there's the kind of operations now called platform teams. And they're under tremendous pressure. They're doing incredible work. Incredible. And they're energized. They're really... So one of the customers I was talking to was moving from VMs on EC2 to containers on GCE on Kubernetes. Google Cloud. And in the last one year, they looked... Honestly, they looked miserable because they have worked so hard in doing that transfomation. Turning their application from a VM-based application into containers. But you could also see that they were so happy and so successful because of the impact that it's had. And so and then I asked them so like, "What is driving that?" This is different customer. What is driving that? And it's really... As soon they get that environment up and running, and this is a large enterprise bank that I was talking to, this other one, their developers are just all over it. And they have, they have hundreds of services running within six months. And they're like, "Well we just got this platform up. "We still have to figure how we're going to upgrade it." But it's... So those are the two constituents. The developers are happy. >> The integration and delivery changes the makeup of how teams work. So that's one thing we're seeing here. And the other one is just scale. >> Yeah. >> So that seems to be the area. Now I got to ask you, as you guys look at... As you guys are doing the work on the enterprise side, you guys, I know you're working hard, I talk to Jennifer a lot, Jennifer Lynn, as well and we've talked before, are used to doing the work. But there's still a lot more work done. Where do you guys see the work that this community value opportunities for participants in the eco-system to fill white spaces? Where are the value lines starting to be drawn? Can you comment? >> Yeah, so I see two or three different areas. One of the areas is of course hardening. And that's why Janet Quill gave the keynote about "Kubernetes is boring and that's a good thing". And that's been something we've been working on for the last year at least. Adding a lot more security capabilities. Adding a lot more just moving everything to GA, right? Adding a lot more hooks in the enterprise storage and into enterprise networking. Building up the training and building up the partners that'll do the implementations. All of those things I think are very, very healthy. >> Yeah. >> Cause I see them. You probably talked to the CNCF. They're helping a lot with the certification and the training. So that's one piece of enterprise adoption. I think the other piece is the developer experience. And that's where a lot of the talks here, my key note as well, I demoed Istio and Knative on top of GKE. The developer experience is ultimately this whole thing. My perspective, this whole thing is about making your developers more productive. And developers have been driving this transition. Again going back to those customer examples. So that's getting a lot easier. >> Yeah, Aparna, I'd love you to talk a little about Knative. So, I know the excitement is there. Products only been around for five months. I remember at your show last summer it was announce and roll. Trying to understand exactly what it is. It's like, wait, wait is serverless going to kill Kubernetes? And how does this fit? How does this work with all the various services in the Cloud? Maybe just understand where we are. >> Right. >> What it is, what it isn't. >> Right. >> Again, so the heritage of serverless, I'm going to go back to Google, right? We have the first serverless offering in the world like 10 years ago. And so that's based on containers. Underneath it's based on containers. That's why we knew that with Kubernetes that's the right foundation for building serverless. And it actually, I think, we sort of held back for the longest time. And a couple of years ago there were one, two, and then 15, and then 17 serverless frameworks that just kind of all popped up around Kubernetes, on top of Kubernetes. I remember the first demo in the community. Here's this serverless piece. And at some point, a little bit over a year ago we decided that actually serverless is really important to our customers, to our users. The majority of Kubernetes tends to be on-prem, actually. And so it's important to them to have serverless capabilities on-prem. So then we need to make sure it's stable and it's something that's standard. >> I think it's a really important point... I talked to some people that are in the serverless ecosystem that is living on a AWS and they say, "You can't build serverless on-prem "because then you're racking "and stacking and dealing with it." And it's not... We know there's servers underneath of it and it's just system calls and how we consume that. But maybe explain the nuances to how this is important and we understand it. >> Yeah. >> There's not like a solution out there. >> Yeah. >> Server meshes, there's a lot of options out there right now. >> Yeah. >> So. >> A lot of things, because this is an open-source community, a lot of things come from the users. So when the user says, "You know what, actually need "the serverless capability on-prem. "Why? "Because I've got this developer group and I don't want "them to have to muck with the infrastructure. "I don't want them to have access to the infrastructure. "I want to just give them a simple interface "where they're going to write their applications "and the rest is taken care of for them." Right? And then I want to be able to bill them on a per-use basis. So, it's... Yeah there's someone managing the server. Someone building actually the severless capability and that's the platform team. That's the guys that I talked about that are working very hard these days happily. But, working very hard. >> And these are the new personas, by the way-- >> Yeah. >> In the enterprise. This is new kind of new re-architecting of how enterprises are creating value. These new platform teams. >> Right. >> This is the opportunity. Well I got to ask you, you know everyone that watches theCUBE knows I'm a big fan of scale. Love Amazon scale. I love Google scale. I love the enterprise market. And I want to get your thoughts... I want you to take a minute to explain the culture at Google Cloud. Because it's a separate building. Give you an opportunity to share. But you guys are working hard to go after the enterprise. It's not like a new thing. But the enterprise is interesting. It's not so much the best technology that wins. It's grit. It's almost like a street fight. You got to go out. You got to win those battles. Get all the work done. Hit those features. You can't just roll into town and say we've got great technology. We're Google. You guys recognize this. And I want you to share the culture you guys are building and how you guys are attacking the enterprise. What's the guiding principles? What are some of the core tenants? >> Yeah, yeah. So you know my entire life has been spent in enterprise software. >> Yeah. >> I do think that enterprises respect Google Cloud. I work very closely with them. And they respect certainly the engineering prowess. Like, "Wow. I need that." >> Yeah. Right? Especially you see all these enterprises that are being transformed by technology. Their industry is being transformed by technology. Whether that's in transportation, or it's in retail, or it's in media. And they want the best. They want the latest. Right? And they also don't necessarily have the skills, like you said, right? So they're looking for a partner that'll both help them scale up but also provide them all of that guidance. And the one thing you asked about culture at Google. I think we are a revolutionary company. We are willing to do lots of things. Lots of things that you wouldn't expect. And that's why you saw GK on-prem from my team, right? The first, kind of, Kubernetes on-prem offering from a cloud provider. Managed by a cloud provider. And that's really... I mean we've seen tremendous, tremendous interest in that. Tremendous feedback from our users and new customers. People that hadn't thought about it. Hadn't thought about Google, necessarily before that have said, "Wow. If you are going to come and help me on-prem "with this, I'm ready. "Give it to me now. "Because I trust you and I know I want to go to the Cloud. "So it's the right step for me. "You have the right incentives." Right? "And you're the open cloud, which is important to me "because I may want to be multi cloud." So that's the piece that is... >> You got the enterprise chops. You've spent your whole career there. I know Jennifer as well. >> Yes. >> A lot of people you guys have hired. >> Right. >> The good news is you've got a market that's changing. So you don't have to come in and replicate the old IT. So that's an opportunity at Google. How are you guys attacking that, that beachhead? Because you have the check. What's the vibe? What's the grit? What's it like... How you guys attacking the enterprise? What do you see as opportunities knowing the enterprise of old-- >> Yeah >> As it shifts to new kind of method? >> Yeah. >> What's the core? >> I think about the problems the users are having. I think about what is the problem the customer is facing. And so... And then breaking that down and solving that for them. I mean that's what's important, right? And so some of the problems I see is one they need a developer platform. And the developer platform sometimes cannot be in the Cloud. When I talk to large financial institutions, there's so much compliance and regulation and things that have to be on-prem. That it has to be on-prem. And they try to move to the Cloud and some things will do it. But the majority, like 90% is on-prem. And so they need an agile development environment and there's no holding it back. Because, like I said, there's all this transformation. Their developers need that environment today. So you have to provide that. That's one use case. We provide an on-prem development and agile development environment. Best in class. Your developers are super happy. Your business is going to do well. The other thing I see, and I see this a lot in retail, but also in hospitality at some of these very kind of brick and mortar enterprises is the edge. They need a solution at their edge location. Thousands, these are thousands of branch locations. We've even got this use case with Chick-fil-A, right? And a lot of times this is... A lot of different use cases, but a lot of time the common thing is that they're collecting data. They're doing some processing at that site and then they're doing further processing in the Cloud. And so it's a connected, but an intimately, it's not always connected.... Intimately connected environment. So that's the second big use case. Edge retail or just edge. There's so many... For me, it's one of the most exciting. There's so many examples of that. >> Awesome. >> Aparna, first of all, just so many goodness I want to say thank you to Google because everything from I heard at the show Google wasn't giving out swag because it actually went to charitable givings instead of spending that money. One of the things we always look is open-source is, how much more value is being created for the eco-system not just the vendor that started it. And it is a really tough balance. We've seen it fail many times. Do you step too far back? And how much do you engage? How do you strike that bound? For the last five to 10 years, we've been saying, "Where is the independent place where we can have that "conversation about cloud?" We think found it at this show. I mean we've been here for three years now. Google Cloud, phenomenal event. Our teams loves to be there, but this feels like overnight has turned into oh wait, here's the show we were looking at to have that conversation. To have that commons where we can come together and there's so many diversity of people, diversity of projects in here. Many which have very disconnected from original Kubernetes and everything, so. It's been fascinating to watch and have to imagine your team is... When you watch that first piece go and everything that's built around it. It's got to be amazing. >> My team loves this event. We have literally I think 300 people here. And a lot of them are core maintainers. Everybody is a contributor, but they are core maintainers of the Kubernetes project. The Istio project. The Knative project. And I think the best thing here is just interacting with our users. Because this is a developer, this is a developer conference, primarily. There's a lot of businesses here. >> Yeah >> With their kind of director level executives. But primarily it's an action-oriented hands-on audience. And you just... These customer meetings that I have, we review their architecture and we're like... It's an engineer to engineer conversation. >> Yep. >> And so how can we make that better? And sometimes they're contributing back and it makes the whole project better. >> Yeah. The thing, too, is it's an engineering, it's a developer conference, true. But what's interesting about that evolution as it modernizes, those end users are developers. >> That's right. >> And so the end user aspect of this show. >> That's right. >> Is the developer piece. >> That's right. >> It never used to be like that. Used to be COMDEX or some big event. >> Yeah. >> And then people just selling their stuff. >> Yeah. >> Doing business. The end user participation... >> Yes. >> Is not a consumption conversation, it's a contribution. >> Right. And end users are all over the spectrum of sort of really, really hands-on. Very, very smart to just give me something that works and I respect all of that, right? And we were actually very far here in terms of GKE. Giving you something that you really don't need to get in, that's fully managed, right? But then on the other hand we had Uber on stage earlier today in their keynote talking about how they've built all of this advanced capability on GKE. And that's a power user. That's using all their capabilities. Like custom additions and an operator. And it's just really gratifying I think for us to work with them and for us to see the user base as well as the community. So the ecosystem. Google. I thinks it's very important for us to have and create economic opportunity for our partners. And you'll see that with GKE on-prem. We're partnering heavily on that one. And you'll see that also in our marketplace. Our Kubernetes marketplace. So many of the companies that have come out of this ecosystem are now part of selling through Google Cloud. >> Aparna, thank you for your time. I know you've had to move some things around to come here. Great to have you on. I love your leadership at Google, it's phenominal. You've got the enterprise chops building out heavily over there. Congratulations. And for more CUBE interviews check out theCUBE dot net. You can check out Aparna's other good news. Of course search her name on Forbes. I wrote a story about her featuring her. Talking about her background and her passion. Always great to have her on theCUBE and get some commentary from Google. Of course, theCUBE is breaking down live coverage. Been there from the beginning of KubeCon and now CloudNativeCon, the Linux Foundation. Bringing you all the analysis and insight. Be back with more coverage after this short break. [Techno Music]