>> Live from Las Vegas, it's theCUBE. Covering AWS re:Invent 2018. Brought to you by Amazon Web Services, Intel. And their ecosystem partners. >> It's good to have you back here on theCUBE as we continue our day three coverage of AWS re:Invent. This is our 7th year at this show by the way, and it was just a little itty bitty thing some seven years ago. It's going to almost 40 thousand plus this year, and I think most of them are still here enjoying day three. Rebecca Knight with John Walls, and we're now joined by Dan Potter, who is the vice president of product management, and marketing at Attunity. Dan, good to see ya. >> Great to be back at theCUBE. >> You're a CUBE alumn, >> I am a CUBE alumn. >> we should say. >> Yes I am. >> And you were with Rebecca last year, two Bostonians, so again, I'll try to interject when I can, right? (Dan laughs) >> You don't speak our language fella. We can translate. >> It's alright Dan, it's okay. >> We were saying, before we got started here, you go to a lot of shows, right? >> Yes. >> And so, every one has it's own personality, it has it's own rhythm, it's own vibe. I mean, how would you characterize what you're seeing here? Especially, here we are day three, and it's still alive and thriving. >> It is absolutely overwhelming. This is my 3rd. Every year it grows. But I just seem to spend my days going from hotel to hotel, you know, to try to hit the sessions you want to, I feel like I'm always in an Uber, it's just so big, and the keynotes, there are so many new solutions that they're rolling out, it's just, the scale is so impressive. >> So what keeps you coming back? I mean, is it the chance to see so many customers in one place? Is it to hear the dizzying number of announcements from Andy Jassy? >> So I loved Andy's presentation, and the keynote this morning was great. For us, all of our customers are moving to the Cloud. I mean, Amazon really is the pioneer of people, and their transformation to the Cloud, and the success that customers are having with the Amazon platform is just astounding, and to see, over the last few years, how organizations have overcome some of the technical barriers, some of the perceptual, the regulatory barriers, they're all gone now, and this wave of movement to Amazon, and to Google and to Azure, it's real and it's happening and it's only accelerating. So it's exciting for us, you know, we're a vendor of data integration solutions. So we help customers move their data into the Cloud, and it's been great business for us, but it's been really fun connecting with our customers who we've gone through multiyear journeys with them as they're moving to the Cloud. So it's fun to see the success that they're having now with all the new technologies in the Amazon stack, it's great. >> So I want to ask you about the trends in the marketplace, what you're seeing, what you're hearing, as you've said, the security, the regulatory, the concerns are pretty much gone now. >> They are. >> They've had this aha moment. >> The Cloud is where I want to be. >> Yes. >> So what else are you seeing? >> Well, things continue to change, so if you look over the last few years, if you look at what's happening, all of those barriers are removed, but the technology stack it's still very much in motion in a positive way. New products are being introduced. Like today if you look at the announcement of a managed Kafka service. So one of the big trends we see is the move for realtime analytics, and to empower realtime analytics you need realtime data movement infrastructure, and Kafka is becoming an integral part of our customers data integration fabric. So that trend to realtime analytics, and having services like Kafka now on the Amazon platform, really important. >> And you got to Hadoop play, right? I mean you're working with Hadoop, you're working with Kafka as you point out. >> For sure. >> Yeah so. >> Well Hadoop's a great example of some of the changes that have happened over the last few years. Five years ago it was all Hadoop, and then all of a sudden the data lake strategy was Hadoop, and S3, and now it's Hadoop, S3, and it's Snowflake. There's so many different technologies that are really purposed to solve very particular pain points, this is the excitement for customers to be able to have this array of different technologies, and done right, if they have an architecture that supports them in moving that data where and when it's needed in whatever time frame, and structuring that information so it's analytics ready, that's the value, and that's some of the real innovations that you've seen over the last few years as this has all started to mature. >> Yeah, well I mean, take me through the data decision if you will. When am I going to leave on-prem? When am I going to move into the public cloud? As the volume of data grows, right? We're talking about trillions of processes within seconds. That's a big nut to crack for a lot of people. Why do I leave put on my Legacy system? Why do I move over? How reliable is it? What's the latency factor here? How do I make sure everybody gets to it, who needs to get to it, if it's over here, and over here? >> Exactly. >> So take us through that. >> So there are two big use cases that we see. One is analytics workloads. The Cloud is a perfect place for analytics. It allows you to create a very large data lake, bring in all kinds of hetero-genius data, bring it together, perform realtime transformation, and deliver analytics ready data to a wide variety of different business users and use cases. So the Cloud is really well purpose fit for analytics. If you look at all of the innovations that you've seen this week, a lot in AI and machine learning, a lot in realtime analytics. I mean this is the elasticity of the Cloud, and the storage capabilities, and the cost benefits of being able to store lots of information, and to be able to run different processes, analytic processes, when you need those, scale up and scale down, perfect fit for analytics. So that one is an absolute no brainer. We see a lot of people, this is the 1st choice for them as they're moving their analytic processes. The 2nd one we see is customers who have core transactional systems, like mainframe systems, you see this a lot in finance, big banks, insurance companies, these are 20 year old. >> I don't want to leave the mainframe, right? >> Not only are they not leaving the mainframe, but they're continuing to invest in the mainframe, and the mainframe is optimized for those transaction processing systems. But what they're not optimized for is how do I build new customer facing, web based applications, mobile applications, and the Cloud is the perfect environment to do that. So the way that we marry those two things together, and the big trend here is this is where realtime synchronization of data comes in. Every time there's an update on that mainframe system we can move that changed data to the Cloud in realtime. So if you're a bank, and you want to provide a web based interface, let me check my account balance, I need a realtime view, but you don't want to write that application against the mainframe, it's too expensive, the processing of a mainframe is too expensive. So if I can replicate that data into the Cloud, and I've got this whole modern array of tools in the Cloud, and I can take modern approaches, like microservices architectures, so I can have different optimized smaller databases that are purposed for different types of mobile apps, or web apps, that's the other trend that we're seeing. So that's kind of bridging that Legacy gap, and to your question of, what data do I leave on-prem? And what do I move to the Cloud? Those core transaction processing systems, they may never move to the Cloud, or in our lifetime we may not see those. Other databases, other applications are lift and shift moving to the Cloud. So things that are a more modern architecture we're seeing a lot of lift and shift directly to the Cloud. But it's going to be a mix for some time. >> So I understand you have a new launch of Attunity for data lakes on AWS, >> We do, yeah. >> tell our viewers a little more about that. >> So this is exciting. So I'll step back for a moment. We provide realtime data integration, and we move that changed data from on-prem into the Cloud. Moving the data is the 1st step, and it's an absolute requirement. But what really needs to happen in order to get the value from your data lake and cloud, you need to be able to not just move the data but shape that data, and make it purpose fit and analytics ready. So if our use case is analytics, and I want to be able to shape this data into a data mart, or I want to create an operational data store for realtime reporting or I'm a data scientist, I need a historic data store on a subset of information. Those are the analytic ready data sets that need to be created, and we're doing that end-to-end data pipeline. So realtime data movement, shaping that data, making it analytics ready and fully automating that process. So it's a streaming data pipeline process that is really leveraging the best of your core transactional systems, mainframe, SAP, Oracle, Legacy apps, files, and moving that to the Cloud in realtime so you can take advantage of all the wonderful capabilities on the Amazon platform. >> So you've been talking a lot about the changes in the data integration space, and sort of what we're seeing. What are your biggest challenges, and biggest opportunities as you're looking to 2019? >> So the biggest challenge is that there's a lot of moving parts, ya know? If you look at, again, you look at the last five years, and how many things have changed as an enterprise architect, they must scratch their head every morning and say what else Is going to change? I thought we had this figured out. So it's a challenge for us because there's a lot of different targets to support. Different clouds, Multicloud, multiple technologies, but that's also the opportunity. The opportunity here is that for us to play that role, and to help customers move data where and when they need it to whatever technology, we're completely agnostic. So if a new technology comes up, like a Snowflake. Great cloud data warehouse built on top of S3. We've seen a lot of customer interest in that, and that's been recent, the last two years out of nowhere. But very large enterprise customers have said, I want to jump on Snowflake. So for them to very quickly say, alright, now I'm going to point my data in addition to Hadoop on-prem, I'm going to point it into the Amazon Cloud, load it into Snowflake, automatically build out that data warehouse for me, and let's get real value. That's the opportunity and the excitement for us. It's never stale, there's always lots of work to do, and the types of impact that it's having on our customers, again it's really transformative to watch them go from the traditional monolithic, slow, traditional warehousing processes to more dynamic, realtime spinning up data marts for business users very very quickly so business users can have better insights, faster, make better decisions quicker, that has the impact that these organizations have been looking for, and that's why they're investing so much in the Cloud, so they can have that business impact, and we're really starting to see that. >> It's almost good new, bad news, right? The good news is things will always change. >> Yeah. >> The bad news is things will always change. >> Absolutely. >> But that's what makes it fun. Every year you come here and it's just, there's a buzz. (Rebecca laughs) There's always something exciting, and there's been some great announcements over the last few days, including ours, and it's been fun. >> It has been fun. >> Alright, Dan thanks for being with us. >> Happy to be here. >> Great to have you once again on theCUBE. >> Thanks for having me. >> See you soon I hope, down the road. >> I hope. >> Dan Potter joining us here on theCUBE. Back with more from AWS re:Invent after a short break.

>> Live from San Jose in the heart of Silicon Valley, it's theCUBE, covering DataWorks Summit 2018, brought to you by Hortonworks. >> Welcome back to theCUBE's live coverage of DataWorks here in sunny San Jose, California. I'm your host Rebecca Knight along with my co-host James Kobielus. We're joined by Dan Potter. He is the VP Product Management at Attunity and also Ali Bajwah, who is the principal partner solutions engineer at Hortonworks. Thanks so much for coming on theCUBE. >> Pleasure to be here. >> It's good to be here. >> So I want to start with you, Dan, and have you tell our viewers a little bit about the company based in Boston, Massachusetts, what Attunity does. >> Attunity, we're a data integration vendor. We are best known as a provider of real-time data movement from transactional systems into data lakes, into clouds, into streaming architectures, so it's a modern approach to data integration. So as these core transactional systems are being updated, we're able to take those changes and move those changes where they're needed when they're needed for analytics for new operational applications, for a variety of different tasks. >> Change data capture. >> Change data capture is the heart of our-- >> They are well known in this business. They have changed data capture. Go ahead. >> We are. >> So tell us about the announcement today that Attunity has made at the Hortonworks-- >> Yeah, thank you, it's a great announcement because it showcases the collaboration between Attunity and Hortonworks and it's all about taking the metadata that we capture in that integration process. So we're a piece of a data lake architecture. As we are capturing changes from those source systems, we are also capturing the metadata, so we understand the source systems, we understand how the data gets modified along the way. We use that metadata internally and now we're built extensions to share that metadata into Atlas and to be able to extend that out through Atlas to higher data governance initiatives, so Data Steward Studio, into the DataPlane Services, so it's really important to be able to take the metadata that we have and to add to it the metadata that's from the other sources of information. >> Sure, for more of the transactional semantics of what Hortonworks has been describing they've baked in to HDP in your overall portfolios. Is that true? I mean, that supports those kind of requirements. >> With HTP, what we're seeing is you know the EDW optimization play has become more and more important for a lot of customers as they try to optimize the data that their EDWs are working on, so it really gels well with what we've done here with Attunity and then on the Atlas side with the integration on the governance side with GDPR and other sort of regulations coming into the play now, you know, those sort of things are becoming more and more important, you know, specifically around the governance initiative. We actually have a talk just on Thursday morning where we're actually showcasing the integration as well. >> So can you talk a little bit more about that for those who aren't going to be there for Thursday. GDPR was really a big theme at the DataWorks Berlin event and now we're in this new era and it's not talked about too, too much, I mean we-- >> And global business who have businesses at EU, but also all over the world, are trying to be systematic and are consistent about how they manage PII everywhere. So GDPR are those in EU regulation, really in many ways it's having ripple effects across the world in terms of practices. >> Absolutely and at the heart of understanding how you protect yourself and comply, I need to understand my data, and that's where metadata comes in. So having a holistic understanding of all of the data that resides in your data lake or in your cloud, metadata becomes a key part of that. And also in terms of enforcing that, if I understand my customer data, where the customer data comes from, the lineage from that, then I'm able to apply the protections of the masking on top of that data. So it's really, the GDPR effect has had, you know, it's created a broad-scale need for organizations to really get a handle on metadata so the timing of our announcement just works real well. >> And one nice thing about this integration is that you know it's not just about being able to capture the data in Atlas, but now with the integration of Atlas and Ranger, you can do enforcement of policies based on classifications as well, so if you can tag data as PCI, PII, personal data, that can get enforced through Ranger to say, hey, only certain admins can access certain types of data and now all that becomes possible once we've taken the initial steps of the Atlas integration. >> So with this collaboration, and it's really deepening an existing relationship, so how do you go to market? How do you collaborate with each other and then also service clients? >> You want to? >> Yeah, so from an engineering perspective, we've got deep roots in terms of being a first-class provider into the Hortonworks platform, both HDP and HDF. Last year about this time, we announced our support for acid merge capabilities, so the leading-edge work that Hortonworks has done in bringing acid compliance capabilities into Hive, was a really important one, so our change to data capture capabilities are able to feed directly into that and be able to support those extensions. >> Yeah, we have a lot of you know really key customers together with Attunity and you know maybe a a result of that they are actually our ISV of the Year as well, which they probably showcase on their booth there. >> We're very proud of that. Yeah, no, it's a nice honor for us to get that distinction from Hortonworks and it's also a proof point to the collaboration that we have commercially. You know our sales reps work hand in hand. When we go into a large organization, we both sell to very large organizations. These are big transformative initiatives for these organizations and they're looking for solutions not technologies, so the fact that we can come in, we can show the proof points from other customers that are successfully using our joint solution, that's really, it's critical. >> And I think it helps that they're integrating with some of our key technologies because, you know, that's where our sales force and our customers really see, you know, that as well as that's where we're putting in the investment and that's where these guys are also investing, so it really, you know, helps the story together. So with Hive, we're doing a lot of investment of making it closer and closer to a sort of real-time database, where you can combine historical insights as well as your, you know, real-time insights. with the new acid merge capabilities where you can do the inserts, updates and deletes, and so that's exactly what Attunity's integrating with with Atlas. We're doing a lot of investments there and that's exactly what these guys are integrating with. So I think our customers and prospects really see that and that's where all the wins are coming from. >> Yeah, and I think together there were two main barriers that we saw in terms of customers getting the most out of their data lake investment. One of them was, as I'm moving data into my data lake, I need to be able to put some structure around this, I need to be able to handle continuously updating data from multiple sources and that's what we introduce with Attunity composed for Hive, building out the structure in an automated fashion so I've got analytics-ready data and using the acid merge capabilities just made those updates much easier. The second piece was metadata. Business users need to have confidence that the data that they're using. Where did this come from? How is it modified? And overcoming both of those is really helping organizations make the most of those investments. >> How would you describe customer attitudes right now in terms of their approach to data because I mean, as we've talked about, data is the new oil, so there's a real excitement and there's a buzz around it and yet there's also so many high-profile cases of breeches and security concerns, so what would you say, is it that customers, are they more excited or are they more trepidatious? How would you describe the CIL mindset right now? >> So I think security and governance has become top of minds right, so more and more the serveways that we've taken with our customers, right, you know, more and more customers are more concerned about security, they're more concerned about governance. The joke is that we talk to some of our customers and they keep talking to us about Atlas, which is sort of one of the newer offerings on governance that we have, but then we ask, "Hey, what about Ranger for enforcement?" And they're like, "Oh, yeah, that's a standard now." So we have Ranger, now it's a question of you know how do we get our you know hooks into the Atlas and all that kind of stuff, so yeah, definitely, as you mentioned, because of GDPR, because of all these kind of issues that have happened, it's definitely become top of minds. >> And I would say the other side of that is there's real excitement as well about the possibilities. Now bringing together all of this data, AI, machine learning, real-time analytics and real-time visualization. There's analytic capabilities now that organizations have never had, so there's great excitement, but there's also trepidation. You know, how do we solve for both of those? And together, we're doing just that. >> But as you mentioned, if you look at Europe, some of the European companies that are more hit by GDPR, they're actually excited that now they can, you know, really get to understand their data more and do better things with it as a result of you know the GDPR initiative. >> Absolutely. >> Are you using machine learning inside of Attunity in a Hortonworks context to find patterns in that data in real time? >> So we enable data scientists to build those models. So we're not only bringing the data together but again, part of the announcement last year is the way we structure that data in Hive, we provide a complete historic data store so every single transaction that has happened and we send those transactions as they happen, it's at a big append, so if you're a data scientist, I want to understand the complete history of the transactions of a customer to be able to build those models, so building those out in Hive and making those analytics ready in Hive, that's what we do, so we're a key enabler to machine learning. >> Making analytics ready rather than do the analytics in the spring, yeah. >> Absolutely. >> Yeah, the other side to that is that because they're integrated with Atlas, you know, now we have a new capability called DataPlane and Data Steward Studio so the idea there is around multi-everything, so more and more customers have multiple clusters whether it's on-prem, in the cloud, so now more and more customers are looking at how do I get a single glass pane of view across all my data whether it's on-prem, in the cloud, whether it's IOT, whether it's data at rest, right, so that's where DataPlane comes in and with the Data Steward Studio, which is our second offering on top of DataPlane, they can kind of get that view across all their clusters, so as soon as you know the data lands from Attunity into Atlas, you can get a view into that across as a part of Data Steward Studio, and one of the nice things we do in Data Steward Studio is that we also have machine learning models to do some profiling, to figure out that hey, this looks like a credit card, so maybe I should suggest this as a tag of sensitive data and now the end user, the end administration has the option of you know saying that okay, yeah, this is a credit card, I'll accept that tag, or they can reject that and pick one of their own. >> Will any of this going forward of the Attunity CDC change in the capture capability be containerized for deployment to the edges in HDP 3.0? I mean, 'cause it seems, I mean for internetive things, edge analytics and so forth, change data capture, is it absolutely necessary to make the entire, some call it the fog computing, cloud or whatever, to make it a completely transactional environment for all applications from micro endpoint to micro endpoint? Are there any plans to do that going forward? >> Yeah, so I think what HDP 3.0 as you mentioned right, one of the key factors that was coming into play was around time to value, so with containerization now being able to bring third-party apps on top of Yarn through Docker, I think that's definitely an avenue that we're looking at. >> Yes, we're excited about that with 3.0 as well, so that's definitely in the cards for us. >> Great, well, Ali and Dan, thank you so much for coming on theCUBE. It's fun to have you here. >> Nice to be here, thank you guys. >> Great to have you. >> Thank you, it was a pleasure. >> I'm Rebecca Knight, for James Kobielus, we will have more from DataWorks in San Jose just after this. (techno music)