>>from >>around the globe. It's the Cube with digital coverage of AWS reinvent 2020 sponsored by Intel, AWS and our community partners. >>Everyone welcome back to the cubes Coverage of AWS reinvent 2020 virtual. This is the Cube virtual. I'm John for your host this year. We're not in person. We're doing remote interviews because of the pandemic. The whole events virtual over three weeks for this week would be having a lot of coverage in and out of what's going on with the news. All that stuff here happening on the Cube Our next guest is a featured segment. Brown Venkatesh, VP of Engineering at Cloudera. Welcome back to the Cube Cube Alumni. Last time you were on was 2018 when we had physical events. Great to see you, >>like good to be here. Thank you. >>S O. You know, Cloudera obviously modernized up with Horton works. That comedy has been for a while, always pioneering this abstraction layer originally with a dupe. Now, with data, all those right calls were made. Data is hot is a big part of reinvent. That's a big part of the theme, you know, machine learning ai ai edge edge edge data lakes on steroids, higher level services in the cloud. This is the focus of reinvents. The big conversations Give us an update on cloud eras. Data platform. What's that? What's new? >>Absolutely. You are really speaking of languages. Read with the whole, uh, data lake architecture that you alluded to. It's uploaded. This mission has always been about, you know, we want to manage how the world's data that what this means for our customers is being ableto aggregate data from lots of different sources into central places that we call data lakes on. Then apply lots of different types of passing to it to direct business value that would cdp with Florida data platform. What we have essentially done is take those same three core tenants around data legs multifunctional takes on data stewardship of management to add on a bunch off cloud native capabilities to it. So this was fundamentally I'm talking about things like disaggregated storage and compute by being able to now not only take advantage of H d efs, but also had a pretty deep, fundamental level club storage. But this is the form factor that's really, really good for our customers. Toe or to operate that from a TCO perspective, if you're going to manage hundreds of terabytes of data like like a lot of a lot of customers do it. The second key piece that we've done with CDP has to do with us embracing containers and communities in a big way on primer heritages around which machines and clusters and things of that nature. But in the cloud context, especially in the context, off managed community services like Amazon CKs, this Lexus spin apart traditional workloads, Sequels, park machine learning and so on. In the context of these Cuban exiles containerized environments which lets customers spin these up in seconds. They're supposed to, you know, tens of minutes on as they're passing, needs grow and shrink. They can actually scale much, much faster up and down to, you know, to make sure that they have the right cost effective footprint for their compute e >>go ahead third piece. >>But the turkey piece of all of this right is to say, along with like cloud native orchestration and cloud NATO storage is that we've embraced this notion of making sure that you actually have a robust data discovery story around it. so increasingly the data sets that you create on top off a platform like CDP. There themselves have value in other use cases that you want to make sure that these data sets are properly replicated. They're probably secure the public government. So you can go and analyze where the data set came from. Capabilities of security and provenance are increasingly more important to our customers. So with CDP, we have a really good story around that data stewardship aspect, which is increasingly important as you as you get into the cloud. And you have these sophisticated sharing scenarios. The >>you know, Clotaire has always had and Horton works. Both companies had strong technical chops. It's well document. Certainly the queues been toe all the events and covered both companies since the inception of 10 years ago. A big data. But now we're in cloud. Big data, fast data, little data, all data. This is what the cloud brings. So I want to get your thoughts on the number one focus of problem solving around cloud. I gotta migrate. Or do I move to the cloud immediately and be born there? Now we know the hyper scale is born in the cloud companies like the Dropbox in the world. They were born in the cloud and all the benefits and goodness came with that. But I'm gonna be pivoting. I'm a company at a co vid with a growth strategy. Lift and shift. Okay, that was It's over. Now that's the low hanging fruit that's use cases kind of done. Been there, done that. Is it migration or born in the cloud? Take us through your thoughts on what does the company do right now? >>E thinks it's a really good question. If you think off, you know where our customers are in their own data journey, right? So increasingly. You know, a few years ago, I would say it was about operating infrastructure. That's where their head was at, right? Increasingly, I think for them it's about deriving value from the data assets that they already have on. This typically means in a combining data from different sources the structure data, some restructure data, transactional data, non transactional, data event oriented data messaging data. They wanna bring all of that and analyze that to make sure that they can actually identify ways toe monetize it in ways that they had not thought about when they actually stored the data originally, right? So I think it's this drive towards increasing monetization of data assets that's driving the new use cases on the platform. Traditionally, it used to be about, you know, sequel analysts who are, if you are like a data scientist using a party's park. So it was sort of this one function that you would focus on with the data. But increasingly, we're seeing these air about, you know, these air collaborative use cases where you wanna have a little bit of sequel, a little bit of machine learning, a little bit off, you know, potentially real time streaming or even things like Apache fling that you're gonna use to actually analyze the data eso when this kind of an environment. But we see that the data that's being generated on Prem is extremely relevant to the use case, but the speed at which they want to deploy the use case. They really want to make sure that they can take advantage of the clouds, agility and infinite capacity to go do that. So it's it's really the answer is it's complicated. It's not so much about you know I'm gonna move my data platform that I used to run the old way from here to there. But it's about I got this use case and I got to stand this up in six weeks, right in the middle of the pandemic on how do I go do that on the data that has to come from my existing line of business systems. I'm not gonna move those over, but I want to make sure that I can analyze the data from their in some cohesive Does that make sense? >>Totally makes sense. And I think just to kind of bring that back for the folks watching. And I remember when CDP was launching the thes data platforms, it really was to replace the data warehouse is the old antiquated way of doing things. But it was interesting. It wasn't just about competing at that old category. It was a new category. So, yeah, you had to have some tooling some sequel, you know, to wrangle data and have some prefabricated, you know, data fenced out somewhere in some warehouse. But the value was the new use cases of data where you never know. You don't know where it's going to come until it comes right, because if you make it addressable, that was the idea of the data platform and data Lakes and then having higher level services. So s so to me. That's, I think, one distinction kind of new category coexisting and disrupting an old category data warehousing. Always bought into that. You know, there's some technical things spark Do all these elements on mechanisms underneath. That's just evolution. But income in incomes cloud on. I want to get your thoughts on this because one of the things that's coming out of all my interviews is speed, speed, speed, deploying high, high, large scale at very large speed. This is the modern application thinking okay to make that work, you gotta have the data fabric underneath. This has always been kind of the dream scenario, So it's kind of playing out. So one Do you believe in that? And to what is the relationship between Cloudera and AWS? Because I think that kind of interestingly points to this one piece. >>Absolutely. So I think that yeah, from my perspective, this is what we call the shared data experience that's central to see PP like the idea is that, you know, data that is generated by the business in one use case is relevant and valid in another use case that is central to how we see companies leveraging data or the second order monetization that they're after, Right? So I think this is where getting out off a traditional data warehouse like data side of context, being able to analyze all of the data that you have, I think is really, really important for many of our customers. For example, many of them increasingly hold what they call this like data hackathons right where they're looking at can be answered. This new question from all the data that we have that is, that is a type of use case that's really hard to enable unless you have a very cohesive, very homogeneous view off all of your data. When it comes to the cloud partners, right, Increasingly, we see that the cloud native services, especially for the core storage, compute and security services are extremely robust that they give us, you know, the scale and that's really truly unparalled in terms of how much data we can address, how quickly we can actually get access to compute on demand when we need it. And we can do all of this with, like, a very, very mature security and governance fabric that you can fit into. So we see that, you know, technologies like s three, for example, have come a long way on along the journey with Amazon on this over the last 78 years. But we both learned how to operate our work clothes. When you're running a terabytes scale, right, you really have to pay attention to matters like scale out and consistency and parallelism and all of these things. These matters significantly right? And it's taken a certain maturity curve that you have to go through to get there. The last part of that is that because the TCO is so optimized with the customer to operate this without any ops on their side, they could just start consuming data, even if it's a terabyte of data. So this means that now we have to have the smarts in the processing engines to think about things like cashing, for example very, very differently because the way you cash data that Zinn hedge defense is very different from how you would do that in the context of his three are similarly, the way you think about consistency and metadata is very, very different at that layer. But we made sure that we can abstract these differences out at the platform layer so that as an as it is an application consumer, you really get the same experience, whether you're running these analytics on clam or whether you're running them in the cloud. And that's really central to how I see this space evolving is that we want to meet the customer where they are, rather than forcing them to change the way they work because off the platform that they're simple. >>So could you take them in to explain some of the integrations with AWS and some customer examples? Because, um, you know, first of all, cost is a big concern on everyone's mind because, you know, it's still lower costs and higher value with the cloud anyway. But it could get away from you. So you know, you're constantly petabytes of scale. There's a lot of data moving around. That's one thing to integration with higher level services. Can you give where does explain how Claudia integration with Amazon? What's the relation of customer wants to know. Hey, you guys, you know, partnering, explain the partnership. And what does it mean for me? >>Absolutely. So the way we look at the partnership hit that one person and ghetto. It's really a four layer cake because the lowest layer is the core infrastructure services. We talked about storage and computing on security, and I am so on and so forth. So that layer is a very robust integration that goes back a few years. The next layer up from that has to do with increasingly, you know, as our customers use analytic experiences from Florida on, they want to combine that with data that's actually in the AWS compute experiences like the red Ship, for example. That's what the analytics layer uploaded the data warehouse offering and how that interrupts would be other services in Amazon that could be relevant. This is common file formats that open source well form it really help us in this context to make sure that they have a very strong level of interest at the analytics there. The third layer up from that has to do with consumption. Like if you're gonna bring an analyst on board. You want to make sure that all of their sequel, like analyst experiences, notebooks, things of that nature that's really strong. And club out of the third layer on the highest layer is really around. Data sharing. That's as aws new and technologies like that become more prevalent. Now. Customers want to make sure that they can have these data states that they have in the different clouds, actually in a robbery. So we provide ways for them, toe browse and search data, regardless of whether that data is on AWS or on traffic. And so that's how the fourth layer in the stack, the vertical slice running through all of these, that we have a really strong business relationship with them both on the on the on the commercial market side as well as in AWS marketplace. Right? So we can actually by having cdp be a part of it of the US marketplace. This means that if you have an enterprise agreement with with Amazon, you can actually pay for CDP toe the credit sexuality purchased. This is a very, very tight relationship that's designed again for these large scale speeds and feeds. Can the customer >>so just to get this right. So if I love the four layer cake icings the success of CDP love that birthday candles can be on top to when you're successful. But you're saying that you're going to mark with Amazon two ways marketplace listing and then also jointly with their enterprise field programs. That right? You say because they have this program you can bundle into the blanket pos or Pio processes That right can explain that again. >>S so if you think this'll states, if you're talking about are significant. So we want to make sure that, you know, we're really aligned with them in terms off our cloud migration strategy in terms of how the customer actually execute to what is a fairly you know, it's a complex deployment to deploy a large multiple functions did and existed takes time, right, So we're gonna make sure that we navigate this together jointly with the U. S. To make sure that from a best practices standpoint, for example, were very well aligned from a cost standpoint, you know what we're telling the customer architecturally is very rather nine. That's that's where I think really the heart of the engineering relationship between the two companies without. >>So if you want Cloudera on Amazon, you just go in. You can click to buy. Or if you got to deal with Amazon in terms of global marketplace deal, which they have been rolling out, I could buy there too, Right? All right, well, run. Thanks for the update and insight. Um, love the four layer cake love gets. See the modernization of the data platform from Cloudera. And congratulations on all the hard work you guys been doing with AWS. >>Thank you so much. Appreciate. >>Okay, good to see you. Okay, I'm John for your hearing. The Cube for Cube virtual for eight of us. Reinvent 2020 virtual. Thanks for watching.

>> Live from San Jose, in the heart of Silicon Valley, it's theCUBE, covering DataWorks Summit 2018. Brought to you by HortonWorks. >> We are wrapping up Day One of coverage of Dataworks here in San Jose, California on theCUBE. I'm your host, Rebecca Knight, along with my co-host, James Kobielus. We have two guests for this last segment of the day. We have Sudhir Hasbe, who is the director of product management at Google and Ram Venkatesh, who is VP of Engineering at Hortonworks. Ram, Sudhir, thanks so much for coming on the show. >> Thank you very much. >> Thank you. >> So, I want to start out by asking you about a joint announcement that was made earlier this morning about using some Hortonworks technology deployed onto Google Cloud. Tell our viewers more. >> Sure, so basically what we announced was support for the Hortonworks DataPlatform and Hortonworks DataFlow, HDP and HDF, running on top of the Google Cloud Platform. So this includes deep integration with Google's cloud storage connector layer as well as it's a certified distribution of HDP to run on the Google Cloud Platform. >> I think the key thing is a lot of our customers have been telling us they like the familiar environment of Hortonworks distribution that they've been using on-premises and as they look at moving to cloud, like in GCP, Google Cloud, they want the similar, familiar environment. So, they want the choice to deploy on-premises or Google Cloud, but they want the familiarity of what they've already been using with Hortonworks products. So this announcement actually helps customers pick and choose like whether they want to run Hortonworks distribution on-premises, they want to do it in cloud, or they wat to build this hybrid solution where the data can reside on-premises, can move to cloud and build these common, hybrid architecture. So, that's what this does. >> So, HDP customers can store data in the Google Cloud. They can execute ephemeral workloads, analytic workloads, machine learning in the Google Cloud. And there's some tie-in between Hortonworks's real-time or low latency or streaming capabilities from HDF in the Google Cloud. So, could you describe, at a full sort of detail level, the degrees of technical integration between your two offerings here. >> You want to take that? >> Sure, I'll handle that. So, essentially, deep in the heart of HDP, there's the HDFS layer that includes Hadoop compatible file system which is a plug-able file system layer. So, what Google has done is they have provided an implementation of this API for the Google Cloud Storage Connector. So this is the GCS Connector. We've taken the connector and we've actually continued to refine it to work with our workloads and now Hortonworks has actually bundling, packaging, and making this connector be available as part of HDP. >> So bilateral data movement between them? Bilateral workload movement? >> No, think of this as being very efficient when our workloads are running on top of GCP. When they need to get at data, they can get at data that is in the Google Cloud Storage buckets in a very, very efficient manner. So, since we have fairly deep expertise on workloads like Apache Hive and Apache Spark, we've actually done work in these workloads to make sure that they can run efficiently, not just on HDFS, but also in the cloud storage connector. This is a critical part of making sure that the architecture is actually optimized for the cloud. So, at our skill and our customers are moving their workloads from on-premise to the cloud, it's not just functional parity, but they also need sort of the operational and the cost efficiency that they're looking for as they move to the cloud. So, to do that, we need to enable these fundamental disaggregated storage pattern. See, on-prem, the big win with Hadoop was we could bring the processing to where the data was. In the cloud, we need to make sure that we work well when storage and compute are disaggregated and they're scaled elastically, independent of each other. So this is a fairly fundamental architectural change. We want to make sure that we enable this in a first-class manner. >> I think that's a key point, right. I think what cloud allows you to do is scale the storage and compute independently. And so, with storing data in Google Cloud Storage, you can like scale that horizontally and then just leverage that as your storage layer. And the compute can independently scale by itself. And what this is allowing customers of HDP and HDF is store the data on GCP, on the cloud storage, and then just use the scale, the compute side of it with HDP and HDF. >> So, if you'll indulge me to a name, another Hortonworks partner for just a hypothetical. Let's say one of your customers is using IBM Data Science Experience to do TensorFlow modeling and training, can they then inside of HDP on GCP, can they use the compute infrastructure inside of GCP to do the actual modeling which is more compute intensive and then the separate decoupled storage infrastructure to do the training which is more storage intensive? Is that a capability that would available to your customers? With this integration with Google? >> Yeah, so where we are going with this is we are saying, IBM DSX and other solutions that are built on top of HDP, they can transparently take advantage of the fact that they have HDP compute infrastructure to run against. So, you can run your machine learning training jobs, you can run your scoring jobs and you can have the same unmodified DSX experience whether you're running against an on-premise HDP environment or an in-cloud HDP environment. Further, that's sort of the benefit for partners and partner solutions. From a customer standpoint, the big value prop here is that customers, they're used to securing and governing their data on-prem in their particular way with HDP, with Apache Ranger, Atlas, and so forth. So, when they move to the cloud, we want this experience to be seamless from a management standpoint. So, from a data management standpoint, we want all of their learning from a security and governance perspective to apply when they are running in Google Cloud as well. So, we've had this capability on Azure and on AWS, so with this partnership, we are announcing the same type of deep integration with GCP as well. >> So Hortonworks is that one pane of glass across all your product partners for all manner of jobs. Go ahead, Rebecca. >> Well, I just wanted to ask about, we've talked about the reason, the impetus for this. With the customer, it's more familiar for customers, it offers the seamless experience, But, can you delve a little bit into the business problems that you're solving for customers here? >> A lot of times, our customers are at various points on their cloud journey, that for some of them, it's very simple, they're like there's a broom coming by and the datacenter is going away in 12 months and I need to be in the cloud. So, this is where there is a wholesale movement of infrastructure from on-premise to the cloud. Others are exploring individual business use cases. So, for example, one of our large customers, a travel partner, so they are exploring their new pricing model and they want to roll out this pricing model in the cloud. They have on-premise infrastructure, they know they have that for a while. They are spinning up new use cases in the cloud typically for reasons of agility. So, if you, typically many of our customers, they operate large, multi-tenant clusters on-prem. That's nice for, so a very scalable compute for running large jobs. But, if you want to run, for example, a new version of Spark, you have to upgrade the entire cluster before you can do that. Whereas in this sort of model, what they can say is, they can bring up a new workload and just have the specific versions and dependency that it needs, independent of all of their other infrastructure. So this gives them agility where they can move as fast as... >> Through the containerization of the Spark jobs or whatever. >> Correct, and so containerization as well as even spinning up an entire new environment. Because, in the cloud, given that you have access to elastic compute resources, they can come and go. So, your workloads are much more independent of the underlying cluster than they are on-premise. And this is where sort of the core business benefits around agility, speed of deployment, things like that come into play. >> And also, if you look at the total cost of ownership, really take an example where customers are collecting all this information through the month. And, at month end, you want to do closing of books. And so that's a great example where you want ephemeral workloads. So this is like do it once in a month, finish the books and close the books. That's a great scenario for cloud where you don't have to on-premises create an infrastructure, keep it ready. So that's one example where now, in the new partnership, you can collect all the data through the on-premises if you want throughout the month. But, move that and leverage cloud to go ahead and scale and do this workload and finish the books and all. That's one, the second example I can give is, a lot of customers collecting, like they run their e-commerce platforms and all on-premises, let's say they're running it. They can still connect all these events through HDP that may be running on-premises with Kafka and then, what you can do is, in-cloud, in GCP, you can deploy HDP, HDF, and you can use the HDF from there for real-time stream processing. So, collect all these clickstream events, use them, make decisions like, hey, which products are selling better?, should we go ahead and give?, how many people are looking at that product?, or how many people have bought it?. That kind of aggregation and real-time at scale, now you can do in-cloud and build these hybrid architectures that are there. And enable scenarios where in past, to do that kind of stuff, you would have to procure hardware, deploy hardware, all of that. Which all goes away. In-cloud, you can do that much more flexibly and just use whatever capacity you have. >> Well, you know, ephemeral workloads are at the heart of what many enterprise data scientists do. Real-world experiments, ad-hoc experiments, with certain datasets. You build a TensorFlow model or maybe a model in Caffe or whatever and you deploy it out to a cluster and so the life of a data scientist is often nothing but a stream of new tasks that are all ephemeral in their own right but are part of an ongoing experimentation program that's, you know, they're building and testing assets that may be or may not be deployed in the production applications. That's you know, so I can see a clear need for that, well, that capability of this announcement in lots of working data science shops in the business world. >> Absolutely. >> And I think coming down to, if you really look at the partnership, right. There are two or three key areas where it's going to have a huge advantage for our customers. One is analytics at-scale at a lower cost, like total cost of ownership, reducing that, running at-scale analytics. That's one of the big things. Again, as I said, the hybrid scenarios. Most customers, enterprise customers have huge deployments of infrastructure on-premises and that's not going to go away. Over a period of time, leveraging cloud is a priority for a lot of customers but they will be in these hybrid scenarios. And what this partnership allows them to do is have these scenarios that can span across cloud and on-premises infrastructure that they are building and get business value out of all of these. And then, finally, we at Google believe that the world will be more and more real-time over a period of time. Like, we already are seeing a lot of these real-time scenarios with IoT events coming in and people making real-time decisions. And this is only going to grow. And this partnership also provides the whole streaming analytics capabilities in-cloud at-scale for customers to build these hybrid plus also real-time streaming scenarios with this package. >> Well it's clear from Google what the Hortonworks partnership gives you in this competitive space, in the multi-cloud space. It gives you that ability to support hybrid cloud scenarios. You're one of the premier public cloud providers and we all know about. And clearly now that you got, you've had the Hortonworks partnership, you have that ability to support those kinds of highly hybridized deployments for your customers, many of whom I'm sure have those requirements. >> That's perfect, exactly right. >> Well a great note to end on. Thank you so much for coming on theCUBE. Sudhir, Ram, that you so much. >> Thank you, thanks a lot. >> Thank you. >> I'm Rebecca Knight for James Kobielus, we will have more tomorrow from DataWorks. We will see you tomorrow. This is theCUBE signing off. >> From sunny San Jose. >> That's right.