>> Live from Barcelona, Spain. It's theCUBE! Covering KubeCon, CloudNativeCon, Europe 2019. Brought to you by Red Hat, the Cloud Native Computing Foundation, and Ecosystem Partners. >> Hi, and welcome back. This is KubeCon, CloudNativeCon 2019 here in Barcelona. 7700 in attendance according to the CNCF foundation. I'm Stu Miniman and my co-host for this week is Corey Quinn. And happy to welcome back to the program, a cube-i-lom Rob Szumski, who's the Product Manager for Red Hat OpenShift. Rob, thanks so much for joining us >> Happy to be here. >> All right, so a couple of weeks ago, we had theCUBE in Boston. You know, short drive for me, didn't have to take a flight as opposed to... I'm doing okay with the jet lag here, but Red Hat Summit was there. And it was a big crowd there, and the topic we're going to talk about with you is operators. And it was something we talked about a lot, something about the ecosystem. But let's start there. For our audience that doesn't know, What is an operator? How does it fit into this whole cloud-native space in this ecosystem? >> (Corey) And where can you hire one? >> (laughs) So there's software programs first of all. And the idea of an operator is everything it takes to orchestrate one of these complex distributor applications, databases, messaging queues, machine learning services. They all are distinct components that all need to be life-cycled. And so there's operational expertise around that, and this is something that might have been in a bash script before, you have a Wiki page. It's just in your head, and so it's putting that into software so that you can stamp out mini copies of that. So the operational expertise from the experts, so you want to go to the folks that make MongoDB for Mongo, for Reddits, for CouchBase, for TensorFlow, whatever it is. Those organizations can embed that expertise, and then take your user configuration and turn that into Kubernetes. >> Okay, and is there automation in that? When I hear the description, it reminds me a little bit of robotic process automation, or RPA, which you talk about, How can I harem them? RPA is, well there's certain jobs that are rather repetitive and we can allow software to do that, so maybe that's not where it is. But help me to put it into the >> No, I think it is. >> Okay, awesome. >> When you think about it, there's a certain amount of toil involved in operating anything and then there's just mistakes that are made by humans when you're doing this. And so you would rather just automate away that toil so you can spend you human capitol on higher level tasks. So that's what operator's all about. >> (Stu) All right. Great. >> Do you find that operator's are a decent approach to taking things that historically would not have been well-suited for autoscaling, for example, because there's manual work that has to happen whenever a no-joinser leaves a swarm. Is that something operators tend to address more effectively? Or am I thinking about this slightly in the wrong direction? >> Yeah, so you can do kind of any Kubernetes event you can hook into, so if your application cares about nodes coming and leaving, for example, this is helpful for operators that are operating the infrastructure itself, which OpenShift has under the hood. But you might care about when new name spaces are created or this pod goes away or whatever it is. You can kind of hook into everything there. >> So, effectively it becomes a story around running stateful things in what was originally designed for stateless containers. >> Yeah, that can help you because you care about nodes going away because your storage was on it, for example. Or, now I need to re-balance that. Whatever that type of thing is it's really critical for running stateful workloads. >> Okay, maybe give us a little bit of context as to the scope of operators and any customer examples you have that could help us add a little bit of concreteness to it. >> Yeah, they're designed to run almost anything. Every common workload that you can think about on an OpenShift cluster, you've got your messaging queues. We have a product that uses an operator, AMQ Streams. It's Kafka. And we've got folks that heavily use a Prometheus operator. I think there's a quote that's been shared around about one of our customer's Ticketmaster. Everybody needed some container native monitoring and everybody could figure out Prometheus on their own. Or they could use operator. So, they were running, I think 300-some instances of Prometheus and dev and staging and this team, that team, this person just screwing around with something over here. So, instead of being experts in Prometheus, they just use the operator then they can scale out very quickly. >> That's great because one of the challenges in this ecosystem, there's so many pieces of it. We always ask, how many companies need to be expert on not just Kubernetes, but any of these pieces. How does this tie into the CNCF, all the various projects that are available? >> I think you nailed it. You have to integrate all this stuff all together and that's where the value of something like OpenShift comes at the infrastructure layer. You got to pick all your networking and storage and your DNS that you're going to use and wire all that together and upgrade that. Lifecycle it. The same thing happens at a higher level, too. You've got all these components, getting your Fluentd pods down to operating things like Istio on Service Mesh's, serviceless workloads. All this stuff needs to be configured and it's all pretty complex. It's moving so fast, nobody can be an expert. The operator's actually the expert, embedded from those teams which is really awesome. >> You said something before we got started. A little bit about a certification program for operators. What is that about? >> We think of it as the super set of our community operators. We've got the TensorFlow community, for example, curates an operator. But, for companies that want to go to market jointly with Red Hat, we have a certification program that takes any of their community content, or some of their enterprise distributions and makes sure that it's well-tested on OpenShift and can be jointly supported by OpenShift in that partner. If you come to Red Hat with a problem with a MongoDB operator, for example, we can jointly solve that problem with MongoDB and ultimately keep your workload up and keep it running. We've got that times a bunch of databases and all kinds of servers like that. You can access those directly from OpenShift which is really exciting. One-click install of a production-ready Mongo cluster. You don't need to dig through a bunch of documentation for how that works. >> All right, so Rob, are all of these specific only to OpenShift, or will they work with flavors of Kubernetes? >> Most of the operators work just against the generic Kubernetes cluster. Some of them also do hook into OpenShift to use some of our specialized security primitives and things like that. That's where you get a little bit more value on OpenShift, but you're just targeting Kubernetes at the end of the day. >> What do you seeing customers doing with this specifically? I guess, what user stories are you seeing that is validating that this is the right direction to go in? >> It's a number of different buckets. The first one is seeing folks running services internally. You traditionally have a DBA team that maybe runs the shared database tier and folks are bringing that the container native world from their VM's that they're used to. Using operators to help with that and so now it's self-service. You have a dedicated cluster infrastructure team that runs clusters and gives out quota. Then, you're just eating into that quota to run whatever workloads that you want in an operator format. That's kind of one bucket of it. Then, you see folks that are building operators for internal operation. They've got deep expertise on one team, but if you're running any enterprise today especially like a large scale Ecommerce shop, there's a number of different services. You've got caching tier, and load balancing tiers. You've got front-ends, you've got back-ends, you've got queues. You can build operators around each one of those, so that those teams even when they're sharing internally, you know, hey where's the latest version of your stack? Here's the operator, go to town. Run it in staging QA, all that type of stuff. Then, lastly, you see these open source communities building operators which is really cool. Something like TensorFlow, that community curates an operator to get you one consistent install, so everyone's not innovating on 30 different ways to install it and you're actually using it. You're using high level stuff with TensorFlow. >> It's interesting to lay it out. Some of these okay, well, a company is doing that because it's behind something. Others you're saying it's a community. Remind me, just Red Hat's long history of helping to give if you will, adult supervision for all of these changes that are happening in the world out there. >> It's a fast moving landscape and some tools that we have are our operator SDK are helping to tame some of that. So, you can get quickly up and running, building an operator whether you are one of those communities, you are a commercial vendor, you're one of our partners, you're one of our customers. We've got tools for everybody. >> Anything specific in the database world that's something we're seeing, that Cambrian explosion in the database world? >> Yeah, I think that folks are finally wrapping their heads around that Kubernetes is for all workloads. And, to make people feel really good about that, you need something like an operator that's got this extremely well-tested code path for what happens when these databases do fail, how do I fail it over? It wasn't just some person that went in and made this. It's the expert, the folks that are committing to MongoDB, to CouchBase, to MySQL, to Postgres. That's the really exciting thing. You're getting that expertise kind of as extension of your operations team. >> For people here at the show, are there sessions about operators? What's the general discussion here at the show for your team? >> There's a ton. Even too many to mention. There's from a bunch of different partners and communities that are curating operators, talking about best practices for managing upgrades of them. Users, all that kind of stuff. I'm going to be giving a keynote, kind of an update about some of stuff we've been talking about here later on this evening. It's all over the place. >> What do you think right now in the ecosystem is being most misunderstood about operators, if anything? >> I think that nothing is quite misunderstood, it's just wrapping your head around what it means to operate applications in this manner. Just like Kubernetes components, there's this desired state loop that's in there and you need to wrap your head around exactly what needs to be in that. You're declarative state is just the Kubernetes API, so you can look at desired and actual and make that happen, just like all the Kub components. So, just looking at a different way of thinking. We had a panel yesterday at the OpenShift Commons about operators and one of the questions that had some really interesting answers was, What did you understand about your software by building an operator? Cause sometimes you need to tease apart some of these things. Oh, I had hard coded configuration here, one group shared that their leader election was not actually working correctly in every single incidences and their operator forced them to dig into that and figure out why. So, I think it's a give and take that's pretty interesting when you're building one of these things. >> Do you find that customers are starting to rely on operators to effectively run their own? For example, MongoDB inside of their Kubernetes clusters, rather than depending upon a managed service offering provided by their public cloud vendor, for example. Are you starting to see people effectively reducing public cloud to baseline primitives at a place to run containers, rather than the higher level services that are starting to move up the stack? >> A number of different reasons for that too. You see this for services if you find a bug in that service, for example, you're just out of luck. You can't go introspect the versions, you can't see how those components are interacting. With an operator you have an open source stack, it's running on your cluster in your infrastructure. You can go introspect exactly what's going on. The operator has that expertise built in, so it's not like you can screw around with everything. But, you have much more insight into what's going on. Another thing you can't get with a cloud service is you can't run it locally. So, if you've got developers that are doing development on an airplane, or just want to have something local so it's running fast, you can put your whole operator stack right on your laptop. Not something you can do with a hosted service which is really cool. Most of these are opens source too, so you can go see exactly how the operator's built. It's very transparent, especially if you're going to trust this for a core part of the infrastructure. You really want to know what's going on under the hood. >> Just to double check, all this can run on OpenShift? It is agnostic to where it lives, whether public cloud or data center? >> Exactly. These are truly hybrid services, so if you're migrating your database to here, for example, over now you have a truly hybrid just targeting Kubernetes environment. You can move that in any infrastructure that you like. This is one of the things that we see OpenShift customers do. Some of them want to be cloud-to-cloud, cloud-to-on-prem, different environments on prem only, because you've got database workloads that might not be leaving or a mainframe you need to tie into, a lot of our FSI customers. Operators can help you there where you can't move some of those workloads. >> Cloud-on-prem makes a fair bit of sense to me. One thing I'm not seeing as much of in the ecosystem is cloud-to-cloud. What are you seeing that's driving that? >> I think everybody has their own cloud that they prefer for whatever reasons. I think it's typically not even cost. It's tooling and cultural change. And, so you kind of invest in one of those. I think people are investing in technologies that might allow them to leave in the future, and operators and Kubernetes being one of those important things. But, that doesn't meant that they're not perfectly happy running on one cloud versus the other, running Kubernetes on top of that. >> Rob, really appreciate all the updates on operators. Thanks so much for joining us again. >> Absolutely. It's been fun. >> Good luck on the keynote. >> Thank you. >> For Corey Quinn, I'm Stu Miniman, back with more coverage two days live from wall to wall here at KubeCon CloudNativeCon 2019 in Barcelona, Spain. Thanks for watching.

>> Live from Seattle, Washington, it's theCUBE, covering KubeCon, and CloudNativeCon North America 2018. Brought to you by Red Hat, the CloudNative Computing Foundation, and the Antigo System Partners. >> Hey, welcome back everyone live here in Seattle for the theCUBE's coverage of KubeCon and CloundNativeCon 2018. I'm John Furrier, theCUBE with Stu Miniman, breaking down all the action. Three days of coverage, we're in day two. A lot of action at Open-source. 8,000 attendees, up from 4,000 North America, they were in China, they were all over Europe. The community's growing in a massive way. We had two great guests from Red Hat, all making it happen, part of the community. We've got Diane Mueller, whose theCUBE alumni director of community development, many times on theCUBE, good to see you, and Rob Szumski, principal product manager, both at Red Hat. Guys, thanks for coming on. Great to see you again. >> Yeah, glad to be here. - Great to be here. >> So the world's changing a lot, and there was some news recently around Red Hat. I can't remember what it was. Recently, something big news, but you guys have been big players in Open-source for years. We always cover it, we always wax on about the origination of it and how the evolution, but the CloudNative piece has gotten so real, and your role in it particularly, we've had many conversations, going maybe back to the OpenStack days of how OpenShift was developing, then the bet on Kubernetes that you made, Core OS acquisition, those two things I think, to me, at least from my perspective, really catalyzed a lot of things at the right time, right? So, from there, just a lot of things has just been happening really in a good way. Big tail wind for you guys, CloudNative app developers are using Open-source, CI/CD pipeline, and then also policy based up under the hood, completely big shift in moving the game down the field. So big congratulations first of all. But what's new? What's the update? >> The update is Operators. I think the next big thing that we are really focusing on, and that's a game changer for all the second day operations type things, and we'll make Rob talk about it in detail, is the rise of Kubernetes' Operators. It's not a scary thing, it's not like terminator day, or anything like that, but it is really the thing that helps us make the service catalogs, the Kubernetes marketplaces really accessible to all of the data bases as a service, and all of the other things, and takes out some of the complexity of delivering applications and database  as a Service to anybody running Kubernetes anywhere. >> Take a minute to explain Operator, real quick, and then we can jump into it, because I think this is a fundamental trend, that we're seeing. Developer trend is pretty obvious, it's been that word for awhile, CloudScale, ML, machine learning, and all the goodness around application development, but the Operator side of it has been an IT thing. But now you guys have a different, a new approach that's winning. What is it? What is Operator? >> Well, it's Kubernetes that has the approach, and I'll let you-- >> Yeah, so it's basically like the rise of containers was great, because you could take a single container and package an application and give to somebody, and know that they can run it successfully. And Operator does that for a distributed system in the exact same way. So you're using all the Kubernetes primitives, so you're not reinventing service discovery, and seeker management, and all that. And you can give somebody an entire Kafka stack, or a machine learning stack, or whatever it is, these very complex distributed systems, and have them run it without having to be an expert. They need to know Kafka at a high level, but not exactly all the underpinnings of it, because that's all baked in the software. >> And the benefit and the impact of the organization is what? >> And just to clarify, so this was added in, I believe Kubernetes is like 1.7, it's something that's in there, it's not something Red Hat specific- >> Yeah, it's like-- >> So you're extending Kubernetes so that you have a custom resource definition, which is an extensible mechanism for saying, hey, I've got a deployment or a staple set, but what if I want to have a new object called a MongoDB? That knows how to deploy, and manage, and upgrade MongoDB. So that's the extension mechanism that we're using. >> Yeah, so you got to think, there's certain applications that this is going to make, just a lot easier how I manage them, deploy them, things like that. Any specific examples you want to share as to-- >> All the clustered data bases. >> There's a lot of the application side in this model have been very excited about this. >> So its all the vendors and partners that want a hybrid Cloud story, just targeting Kubernetes, and we're using Kubernetes under the hood, and then everybody wants to run like a staple data base tier, whether that's Mongo and Couchbase, and Cassandra, whatever. And these are all distributed systems. >> Alright, so I want you to just perch, you said a hybrid Cloud. Explain that model, because there's just something in general discussion that is hybrid or multi means I'm running multiple places, I'm not necessarily stretching an application, but I have instances there, just want to make sure we're on the same page. >> So this would be more the compatibility that you're programming against when you're building an operator, is Kubernetes. It's not a Cloud offering, it's not OpenShift, so you're just targeting Kubernetes, and so you can run MongoDB on prem, in the Cloud, and have it function the exact same, by standing up one of these Operators. And then if that Operator has higher level constructs for how to do multi-cluster aware data rebalancing, you can take advantage of that too. >> And the Open-source status of this product is what? >> It's all Open-source, it's all in the github repos, there's a Google group for Operator framework, that anyone can come and participate in. We hold SIG meetings on the third Friday of every month, 9 a.m. Pacific Time, and it's a completely Open-source project. There's a whole framework around it, so there's the Operator SDK, the Operator Lifecycle Management, and Operator metering, all the tooling there to help people build and manage these Operators, and it's all being built out there in the open with the community's support and feedback loops. >> What's the feedback? What's the top feedback you guys are getting right now? Seeing right now? >> I have to say, this is really, like I've been hanging out with you guys like for the past three, four months on this topic, trying to get my head around it and everything, and we came here and we had two sessions, an intro session and a deep dive session, intro yesterday, deep dive today. Today's deep dive, the room was about 250 people, and they're were people outside of it-- >> Security guards blocking people from coming in. >> Nobody could come in and it's like, it's insane. It's like, everybody needs these things, and everybody wants to figure out that, and when you ask people in the room whose building one, half the room raises their hands. It's just crazy. This thing crept up on us really, maybe not on Core OS, okay, it crept up on me very quickly, and it's very rapid adoption. We have a Kubernetes Operators workshop on Friday, so not only do we have pre-conference days of like OpenShift Cons that are huge now, but now we're starting to book end, CNCF events and put on other things, just because, and that, we had 100 seats that we were hoping we would fill, and it sold out in like minutes once it got in there, and there's a waiting list of like 300 people. It is like one of, aside from Knative, and all the other wonderful hot things too, it is one of the most interesting developments I think right now. >> Thirst for the content. Would it impact? >> Yeah, and you can get all of the documentation is out there now, and people are already building them. We have a list of 50 community Operators. It's just, it's phenomenal how quickly it's growing. >> You know, Diane and Rob, it's funny because you know, we do so many of these theCUBE interviews, and this is our 10th year doing theCUBE coming up, and I remember the conversations going back in the OpenStack days, we would ask questions like, if you had a magic wand, what would you like, hope to have happened, right? And you know, those are parts of the evolution, where it's like, it's aspirational, things are being built. It seems now with Kubernetes, it's almost like, wait a minute, it's actually, this is like the goodness is so compelling, above and below Kubernetes that it's almost like uncomprehendible. You think about, oh this is actually happening. Finally the kinds of steady state kind of operational things that have been a pain in the butt for years-- >> Yeah, the toil, it's gone, for the most part. >> Yeah. >> So Rob, I've been having a lot of just thinking back to, you're employee number two at Core OS, when I first talked to Core OS, it was, we're going to build all of these individual tools, and we're going to Open-source them, and it's going to be good. We watched this just rising ecosystem and the CNCF, and it feels like what's nice and what's different that I see, compared to some previous things, is it's not one product or even a small group of companies. It's, I have this tool kit, and some of them work together, but many of them are independently used. We've talked to your peers earlier about it, etCD. etCD is totally stand alone, doesn't need to be Kubernetes. What have you seen, if you go back to that original vision, would Core OS just been, part of this whole ecosystem, and done it, if this was available, and has this delivering on a promise that your team had hoped to work on? >> Yeah, so we've always filled in where we see gaps, and so something like etCD, the concept is not new, and it comes from Google, and they have a system internally, and as Brandon got up on stage and said, we needed that coordinate, reboot, to grow out, to cluster of machines. It didn't exist so we had to build it. Same thing with how we wanted to manage Linux. There was no distro that even resembled what we were doing. Wanted to do automatic upgrades, people thought that was crazy, so we had to go build it. And so, but we always adopted the best of breed technology, when it existed. In our early bet Kubernetes, we just saw, this is the thing, and went for it. I don't even remember what version, but it was months and months before it was zero point oh, or one point oh, so it was, we've been doing it forever. And you just see the right thing, and it's the little nugget that you need, and if you don't see it, then you build it. >> What are you surprised about Rob, in terms of the ecosystem now, you mentioned some goodness is happening, still a lot more to do, visibility around value creation, you're starting to see spots where value can be created in the ecosystem, which is great. Still more work areas, but what's surprising you? What do you see as opportunities, challenges? Your thoughts, because this vision of ease of use and programmability, is happening, right? So there's still more work to do. What's your vision there? What's your thoughts? >> I mean, I think self service is key, so this is like the rise of the Cloud comes from self service for developers, and Kubernetes gives you the right abstraction, where self service for VM's, like OpenStack, which is not quite at the level of what you want. You don't want a VM, you actually wanted a place to deploy an application, you wanted load balancing, you wanted service discovery, you didn't want like a bare Ubuntu VM, and so Kubernetes raises you up to where you're productive, and then it's about building stuff on top. But what's interesting, in the space is, we're still kind of competing on Kubernetes installers, and stuff like that, so we're not even really into like the phase where people are being super productive on the platform, other than these leading companies. So I think we'll democratize that, and we'll have a whole new landscape. >> And so 2019 you see as what being a key theme for Kubernetes? >> I think it'll be Core stuff built on top, like all the serverless frameworks, a bunch of container natives storage solutions, solving some of these problems that folks are reaching out to external machine learning, but bringing that onto the cluster, GPU support, that type of stuff. It's all about the workloads. >> And tradition end users, you have a huge install base, with Red Hat, well documented, as the end users start coming in and looking at CloudNative, and doing a reimagine of their environment, whether it's IT span, IT investments, to have a run their coding and the deployments. It's going to change. 2019's going to have an impact on what I call mainstream enterprise, for lack of a better description. What's the impact of those guys, 'cause now, they now have head room, they can do more, what's the main stream enterprise look like right now with the impact of Kubernetes? >> I think they're going to start deploying applications and get like lower the time to business value, much, much lower. And I was just talking to a customer, and they ordered bare metal machines like a year ago, and they're still not racked and in the data center. And so people are still getting over that type of stuff, but once you have like a shared Kubernetes layer, you can onboard teams like crazy. I mean, name spaces are free, quote, unquote, and you can get 35 engineering teams on a Kubernetes cluster super easy. >> So they can ramp up in development teams basically, as they bring value in-house, versus outsourcing everything. They start getting development teams, this is where the action is. >> I think you're also going to see the rise of those end users contributing back things, to the Kubernetes community and as Lyft, and Uber, and everybody are great examples of that. Uber with Jaeger, and Lyft is, we were just in the Operators thing, and they raised their hand that they are about to Open-source it, a few Operators that they're building and stuff, and you're just going to see people that you didn't normally see. Often these large foundation driven things are vendor driven, but I think what you see here, is the end user community is now embracing the Open-source, is getting the legal teams there, allowing them to share their things, because one, they get more people to maintain them, and more people working on them, but it's really I think the rise of the end user we'll see, as they start participating more and more in here. And that's the promise of Open-source. >> And that's where CNCF really made it's bones. It wasn't really vendor led per se, it was really end users, the guys building out their stuff for the first time. You see Lyft for instance, great example, you guys did a Core OS, this is like the new generational model. Final question before we break. I want to get this out there. Get a plug in for Red Hat. What are you guys, what's the focus for the show? What's the news? What's the big story for Red Hat here at KubeCon this year? >> I think it's Operators, that's what we're here talking about. It's a really big push to once again get smarter workloads onto the cluster. We've got a really great hybrid story, we've got a really great over the air upgrade story that we're bringing from some of the Core OS technology, and then the next thing is, once it's easy to run 35 clusters, we need a bunch of workloads to put on there. And so we want to save folks from the toil of running all those workloads as well, just like we did at the cluster level. >> Awesome. >> Well put. I couldn't add more. One of the things that Core OS did, you hit the nail on the head earlier, is when there was something missing, they helped us build it, and with the Operator SDK, and the Lifecycle Management, and the metering, and whatever else the tooling is, they have really been inspirational inside of Red Hat. And so they filled a number of gaps, and it's just been all Operators all the time right now. >> It's great when a plan comes together. You guys got a great tail wind. Congratulations on all the success, and it's just the beginning of the wave. It's theCUBE, covering the wave of innovation here at KubeCon CloudNativeCon 2018, we'll be back with more live coverage. Day two of Three days of Kube Coverage. We'll be right back. (upbeat music)