>>Okay. We're back here, live in New York city for big data week. This is siliconangle.tvs, exclusive coverage of Hadoop world strata plus Hadoop world big event, a big data week. And we just wrote a blog post on siliconangle.com calling this the south by Southwest for data geeks and, and, um, it's my prediction that this is going to turn into a, quite the geek Fest. Uh, obviously the crowd here is enormous packed and an amazing event. And, uh, we're excited. This is siliconangle.com. I'm the founder John ferry. I'm joined by cohost update >>Volante of Wiki bond.org, where people go for free research and peers collaborate to solve problems. And we're here with Jack Norris. Who's the vice president of market marketing at map are a company that we've been tracking for quite some time. Jack, welcome back to the cube. Thank you, Dave. I'm going to hand it to you. You know, we met quite a while ago now. It was well over a year ago and we were pushing at you guys and saying, well, you know, open source and nice look, we're solving problems for customers. We got the right model. We think, you know, this is, this is our strategy. We're sticking to it. Watch what happens. And like I said, I have to hand it to you. You guys are really have some great traction in the market and you're doing what you said. And so congratulations on that. I know you've got a lot more work to do, but >>Yeah, and actually the, the topic of openness is when it's, it's pretty interesting. Um, and, uh, you know, if you look at the different options out there, all of them are combining open source with some proprietary. Uh, now in the case of some distributions, it's very small, like an ODBC driver with a proprietary, um, driver. Um, but I think it represents that that any solution combining to make it more open is, is important. So what we've done is make innovations, but what we've made those innovations we've opened up and provided API. It's like NFS for standard access, like rest, like, uh, ODBC drivers, et cetera. >>So, so it's a spectrum. I mean, actually we were at Oracle open world a few weeks ago and you listen to Larry Ellison, talk about the Oracle public cloud mix of actually a very strong case that it's open. You can move data, it's all Java. So it's all about standards. Yeah. And, uh, yeah, it from an opposite, but it was really all about the business value. That's, that's what the bottom line is. So, uh, we had your CEO, John Schroeder on yesterday. Uh, John and I both were very impressed with, um, essentially what he described as your philosophy of we, we not as a product when we have, we have customers when we announce that product and, um, you know, that's impressive, >>Is that what he was also given some good feedback that startup entrepreneurs out there who are obviously a lot of action going on with the startup community. And he's basically said the same thing, get customers. Yeah. And that's it, that's all and use your tech, but don't be so locked into the tech, get the cutters, understand the needs and then deliver that. So you guys have done great. And, uh, I want to talk about the, the show here. Okay. Because, uh, you guys are, um, have a big booth and big presence here at the show. What, what did you guys are learning? I'll say how's the positioning, how's the new news hitting. Give us a quick update. So, >>Uh, a lot of news, uh, first started, uh, on Tuesday where we announced the M seven edition. And, uh, yeah, I brought a demo here for me, uh, for you all. Uh, because the, the big thing about M seven is what we don't have. So, uh, w we're not demoing Regents servers, we're not demoing compactions, uh, we're not demoing a lot of, uh, manual administration, uh, administrative tasks. So what that really means is that we took this stack. And if you look at HBase HBase today has about half of dupe users, uh, adopting HBase. So it's a lot of momentum in the market, uh, and, you know, use for everything from real-time analytics to kind of lightweight LTP processing. But it's an infrastructure that sits on top of a JVM that stores it's data in the Hadoop distributed file system that sits on a JVM that stores its data in a Linux file system that writes to disk. >>And so a lot of the complexity is that stack. And so as an administrator, you have to worry about how data gets permit, uh, uh, you know, kind of basically written across that. And you've got region servers to keep up, uh, when you're doing kind of rights, you have things called compactions, which increased response time. So it's, uh, it's a complex environment and we've spent quite a bit of time in, in collapsing that infrastructure and with the M seven edition, you've got files and tables together in the same layer writing directly to disc. So there's no region servers, uh, there's no compactions to deal with. There's no pre splitting of tables and trying to do manual merges. It just makes it much, much simpler. >>Let's talk about some of your customers in terms of, um, the profile of these guys are, uh, I'm assuming and correct me if I'm wrong, that you're not selling to the tire kickers. You're selling to the guys who actually have some experience with, with a dupe and have run into some of the limitations and you come in and say, Hey, we can solve some of those problems. Is that, is that, is that right? Can you talk about that a little bit >>Characterization? I think part of it is when you're in the evaluation process and when you first hear about Hadoop, it's kind of like the Gartner hype curve, right. And, uh, you know, this stuff, it does everything. And of course you got data protection, cause you've got things replicated across the cluster. And, uh, of course you've got scalability because you can just add nodes and so forth. Well, once you start using it, you realize that yes, I've got data replicated across the cluster, but if I accidentally delete something or if I've got some corruption that's replicated across the cluster too. So things like snapshots are really important. So you can return to, you know, what was it, five minutes before, uh, you know, performance where you can get the most out of your hardware, um, you know, ease of administration where I can cut this up into, into logical volumes and, and have policies at that whole level instead of at an individual file. >>So there's a, there's a bunch of features that really resonate with users after they've had some experience. And those tend to be our, um, you know, our, our kind of key customers. There's a, there's another phase two, which is when you're testing Hadoop, you're looking at, what's possible with this platform. What, what type of analytics can I do when you go into production? Now, all of a sudden you're looking at how does this fit in with my SLS? How does this fit in with my data protection, uh, policies, you know, how do I integrate with my different data sources? And can I leverage existing code? You know, we had one customer, um, you know, a large kind of a systems integrator for the federal government. They have a million lines of code that they were told to rewrite, to run with other distributions that they could use just out of the box with Matt BARR. >>So, um, let's talk about some of those customers. Can you name some names and get >>Sure. So, um, actually I'll, I'll, I'll talk with, uh, we had a keynote today and, uh, we had this beautiful customer video. They've had to cut because of times it's running in our booth and it's screaming on our website. And I think we've got to, uh, actually some of the bumper here, we kind of inserted. So, um, but I want to shout out to those because they ended up in the cutting room floor running it here. Yeah. So one was Rubicon project and, um, they're, they're an interesting company. They're a real-time advertising platform at auction network. They recently passed a Google in terms of number one ad reach as mentioned by comScore, uh, and a lot of press on that. Um, I particularly liked the headline that mentioned those three companies because it was measured by comScore and comScore's customer to map our customer. And Google's a key partner. >>And, uh, yesterday we announced a world record for the Hadoop pterosaur running on, running on Google. So, um, M seven for Rubicon, it allows them to address and replace different point solutions that were running alongside of Hadoop. And, uh, you know, it simplifies their, their potentially simplifies their architecture because now they have more things done with a single platform, increases performance, simplifies administration. Um, another customer is ancestry.com who, uh, you know, maybe you've seen their ads or heard, uh, some of their radio shots. Um, they're they do a tremendous amount of, of data processing to help family services and genealogy and figure out, you know, family backgrounds. One of the things they do is, is DNA testing. Uh, so for an internet service to do that, advanced technology is pretty impressive. And, uh, you know, you send them it's $99, I believe, and they'll send you a DNA kit spit in the tube, you send it back and then they process that and match and give you insights into your family background. So for them simplifying HBase meant additional performance, so they could do matches faster and really simplified administration. Uh, so, you know, and, and Melinda Graham's words, uh, you know, it's simpler because they're just not there. Those, those components >>Jack, I want to ask you about enterprise grade had duped because, um, um, and then, uh, Ted Dunning, because he was, he was mentioned by Tim SDS on his keynote speech. So, so you have some rockstars stars in the company. I was in his management team. We had your CEO when we've interviewed MC Sri vis and Google IO, and we were on a panel together. So as to know your team solid team, uh, so let's talk about, uh, Ted in a minute, but I want to ask you about the enterprise grade Hadoop conversation. What does that mean now? I mean, obviously you guys were very successful at first. Again, we were skeptics at first, but now your traction and your performance has proven this is a market for that kind of platform. What does that mean now in this, uh, at this event today, as this is evolving as Hadoop ecosystem is not just Hadoop anymore. It's other things. Yeah, >>There's, there's, there's three dimensions to enterprise grade. Um, the first is, is ease of use and ease of use from an administrator standpoint, how easy does it integrate into an existing environment? How easy does it, does it fit into my, my it policies? You know, do you run in a lights out data center? Does the Hadoop distribution fit into that? So that's, that's one whole dimension. Um, a key to that is, is, you know, complete NFS support. So it functions like, uh, you know, like standard storage. Uh, a second dimension is undependability reliability. So it's not just, you know, do you have a checkbox ha feature it's do you have automated stateful fail over? Do you have self healing? Can you handle multiple, uh, failures and, and, you know, automated recovery. So, you know, in a lights out data center, can you actually go there once a week? Uh, and then just, you know, replace drives. And a great example of that is one of our customers had a test cluster with, with Matt BARR. It was a POC went on and did other things. They had a power field, they came back a week later and the cluster was up and running and they hadn't done any manual tasks there. And they were, they were just blown away to the recovery process for the other distributions, a long laundry list of, >>So I've got to ask you, I got to ask you this, the third >>One, what's the third one, third one is performance and performance is, is, you know, kind of Ross' speed. It's also, how do you leverage the infrastructure? Can you take advantage of, of the network infrastructure, multiple Knicks? Can you take advantage of heterogeneous hardware? Can you mix and match for different workloads? And it's really about sharing a cluster for different use cases and, and different users. And there's a lot of features there. It's not just raw >>The existing it infrastructure policies that whole, the whole, what happens when something goes wrong. Can you automate that? And then, >>And it's easy to be dependable, fast, and speed the same thing, making HBase, uh, easy, dependable, fast with themselves. >>So the talk of the show right now, he had the keynote this morning is that map. Our marketing has dropped the big data term and going with data Kozum. Is that true? Is that true? So, Joe, Hellerstein just had a tweet, Joe, um, famous, uh, Cal Berkeley professor, computer science professor now is CEO of a startup. Um, what's the industry trifecta they're doing, and he had a good couple of epic tweets this week. So shout out to Joe Hellerstein, but Joel Hellison's tweet that says map our marketing has decided to drop the term big data and go with data Kozum with a shout out to George Gilder. So I'm kind of like middle intellectual kind of humor. So w w w what's what's your response to that? Is it true? What's happening? What is your, the embargo, the VP of marketing? >>Well, if you look at the big data term, I think, you know, there's a lot of big data washing going on where, um, you know, architectures that have been out there for 30 years or, you know, all about big data. Uh, so I think there's a, uh, there's the need for a more descriptive term. Um, the, the purpose of data Kozum was not to try to coin something or try to, you know, change a big data label. It was just to get people to take a step back and think, and to realize that we are in a massive paradigm shift. And, you know, with a shout out to George Gilder, acknowledging, you know, he recognized what the impact of, of making available compute, uh, meant he recognized with Telekom what bandwidth would mean. And if you look at the combination of we've got all this, this, uh, compute efficiency and bandwidth, now data them is, is basically taking those resources and unleashing it and changing the way we do things. >>And, um, I think, I think one of the ways to look at that is the new things that will be possible. And there's been a lot of focus on, you know, SQL interfaces on top of, of Hadoop, which are important. But I think some of the more interesting use cases are taking this machine J generated data that's being produced very, very rapidly and having automated operational analytics that can respond in a very fast time to change how you do business, either, how you're communicating with customers, um, how you're responding to two different, uh, uh, risk factors in the environment for fraud, et cetera, or, uh, just increasing and improving, um, uh, your response time to kind of cost events. We met earlier called >>Actionable insight. Then he said, assigning intent, you be able to respond. It's interesting that you talk about that George Gilder, cause we like to kind of riff and get into the concept abstract concepts, but he also was very big in supply side economics. And so if you look at the business value conversation, one of things we pointed out, uh, yesterday and this morning, so opening, um, review was, you know, the, the top conversations, insight and analytics, you know, as a killer app right now, the app market has not developed. And that's why we like companies like continuity and what you guys are doing under the hood is being worked on right at many levels, performance units of those three things, but analytics is a no brainer insight, but the other one's business value. So when you look at that kind of data, Kozum, I can see where you're going with that. >>Um, and that's kind of what people want, because it's not so much like I'm Republican because he's Republican George Gilder and he bought American spectator. Everyone knows that. So, so obviously he's a Republican, but politics aside, the business side of what big data is implementing is massive. Now that I guess that's a Republican concept. Um, but not really. I mean, businesses is, is, uh, all parties. So relative to data caused them. I mean, no one talks about e-business anymore. We talking to IBM at the IBM conference and they were saying, Hey, that was a great marketing campaign, but no one says, Hey, uh, you and eat business today. So we think that big data is going to have the same effect, which is, Hey, are you, do you have big data? No, it's just assumed. Yeah. So that's what you're basically trying to establish that it's not just about big. >>Yeah. Let me give you one small example, um, from a business value standpoint and, uh, Ted Dunning, you mentioned Ted earlier, chief application architect, um, and one of the coauthors of, of, uh, the book hoot, which deals with machine learning, uh, he dealt with one of our large financial services, uh, companies, and, uh, you know, one of the techniques on Hadoop is, is clustering, uh, you know, K nearest neighbors, uh, you know, different algorithms. And they looked at a particular process and they sped up that process by 30,000 times. So there's a blog post, uh, that's on our website. You can find out additional information on that. And I, >>There's one >>Point on this one point, but I think, you know, to your point about business value and you know, what does data Kozum really mean? That's an incredible speed up, uh, in terms of, of performance and it changes how companies can react in real time. It changes how they can do pattern recognition. And Google did a really interesting paper called the unreasonable effectiveness of data. And in there they say simple algorithms on big data, on massive amounts of data, beat a complex model every time. And so I think what we'll see is a movement away from data sampling and trying to do an 80 20 to looking at all your data and identifying where are the exceptions that we want to increase because there, you know, revenue exceptions or that we want to address because it's a cost or a fraud. >>Well, that's what I, I would give a shout out to, uh, to the guys that digital reasoning Tim asked he's plugged, uh, Ted. It was idolized him in terms of his work. Obviously his work is awesome, but two, he brought up this concept of understanding gap and he showed an interesting chart in his keynote, which was the date explosion, you know, it's up and, you know, straight up, right. It's massive amount of data, 64% unstructured by his calculation. Then he showed out a flat line called attention. So as data's been exploding over time, going up attention mean user attention is flat with some uptick maybe, but so users and humans, they can't expand their mind fast enough. So machine learning technologies have to bridge that gap. That's analytics, that's insight. >>Yeah. There's a big conversation now going on about more data, better models, people trying to squint through some of the comments that Google made and say, all right, does that mean we just throw out >>The models and data trumps algorithms, data >>Trumps algorithms, but the question I have is do you think, and your customer is talking about, okay, well now they have more data. Can I actually develop better algorithms that are simpler? And is it a virtuous cycle? >>Yeah, it's I, I think, I mean, uh, there are there's, there are a lot of debate here, a lot of information, but I think one of the, one of the interesting things is given that compute cycles, given the, you know, kind of that compute efficiency that we have and given the bandwidth, you can take a model and then iterate very quickly on it and kind of arrive at, at insight. And in the past, it was just that amount of data in that amount of time to process. Okay. That could take you 40 days to get to the point where you can do now in hours. Right. >>Right. So, I mean, the great example is fraud detection, right? So we used the sample six months later, Hey, your credit card might've been hacked. And now it's, you know, you got a phone call, you know, or you can't use your credit card or whatever it is. And so, uh, but there's still a lot of use cases where, you know, whether is an example where modeling and better modeling would be very helpful. Uh, excellent. So, um, so Dana custom, are you planning other marketing initiatives around that? Or is this sort of tongue in cheek fun? Throw it out there. A little red meat into the chum in the waters is, >>You know, what really motivated us was, um, you know, the cubes here talking, you know, for the whole day, what could we possibly do to help give them a topic of conversation? >>Okay. Data cosmos. Now of course, we found that on our proprietary HBase tools, Jack Norris, thanks for coming in. We appreciate your support. You guys have been great. We've been following you and continue to follow. You've been a great support of the cube. Want to thank you personally, while we're here. Uh, Matt BARR has been generous underwriter supportive of our great independent editorial. We want to recognize you guys, thanks for your support. And we continue to look forward to watching you guys grow and kick ass. So thanks for all your support. And we'll be right back with our next guest after this short break. >>Thank you. >>10 years ago, the video news business believed the internet was a fat. The science is settled. We all know the internet is here to stay bubbles and busts come and go. But the industry deserves a news team that goes the distance coming up on social angle are some interesting new metrics for measuring the worth of a customer on the web. What zinc every morning, we're on the air to bring you the most up-to-date information on the tech industry with scrutiny on releases of the day and news of industry-wide trends. We're here daily with breaking analysis, from the best minds in the business. Join me, Kristin Filetti daily at the news desk on Silicon angle TV, your reference point for tech innovation 18 months.

>> Live from Midtown Manhattan, it's theCUBE. Covering BigData New York City 2017. Brought to you by SiliconANGLE Media and its ecosystem sponsors. (calm electronic music) >> Basil Faruqui, who's the Solutions Marketing Manger at BMC, welcome to theCUBE. >> Thank you, good to be back on theCUBE. >> So first of all, heard you guys had a tough time in Houston, so hope everything's gettin' better, and best wishes to everyone down in-- >> We're definitely in recovery mode now. >> Yeah and so hopefully that can get straightened out quick. What's going on with BMC? Give us a quick update in context to BigData NYC. What's happening, what is BMC doing in the big data space now, the AI space now, the IOT space now, the cloud space? >> So like you said that, you know, the data link space, the IOT space, the AI space, there are four components of this entire picture that literally haven't changed since the beginning of computing. If you look at those four components of a data pipeline it's ingestion, storage, processing, and analytics. What keeps changing around it, is the infrastructure, the types of data, the volume of data, and the applications that surround it. And the rate of change has picked up immensely over the last few years with Hadoop coming in to the picture, public cloud providers pushing it. It's obviously creating a number of challenges, but one of the biggest challenges that we are seeing in the market, and we're helping costumers address, is a challenge of automating this and, obviously, the benefit of automation is in scalability as well and reliability. So when you look at this rather simple data pipeline, which is now becoming more and more complex, how do you automate all of this from a single point of control? How do you continue to absorb new technologies, and not re-architect our automation strategy every time, whether it's it Hadoop, whether it's bringing in machine learning from a cloud provider? And that is the issue we've been solving for customers-- >> Alright let me jump into it. So, first of all, you mention some things that never change, ingestion, storage, and what's the third one? >> Ingestion, storage, processing and eventually analytics. >> And analytics. >> Okay so that's cool, totally buy that. Now if your move and say, hey okay, if you believe that standard, but now in the modern era that we live in, which is complex, you want breath of data, but also you want the specialization when you get down to machine limits highly bounded, that's where the automation is right now. We see the trend essentially making that automation more broader as it goes into the customer environments. >> Correct >> How do you architect that? If I'm a CXO, or I'm a CDO, what's in it for me? How do I architect this? 'Cause that's really the number one thing, as I know what the building blocks are, but they've changed in their dynamics to the market place. >> So the way I look at it, is that what defines success and failure, and particularly in big data projects, is your ability to scale. If you start a pilot, and you spend three months on it, and you deliver some results, but if you cannot roll it out worldwide, nationwide, whatever it is, essentially the project has failed. The analogy I often given is Walmart has been testing the pick-up tower, I don't know if you've seen. So this is basically a giant ATM for you to go pick up an order that you placed online. They're testing this at about a hundred stores today. Now if that's a success, and Walmart wants to roll this out nation wide, how much time do you think their IT department's going to have? Is this a five year project, a ten year project? No, and the management's going to want this done six months, ten months. So essentially, this is where automation becomes extremely crucial because it is now allowing you to deliver speed to market and without automation, you are not going to be able to get to an operational stage in a repeatable and reliable manner. >> But you're describing a very complex automation scenario. How can you automate in a hurry without sacrificing the details of what needs to be? In other words, there would seem to call for repurposing or reusing prior automation scripts and rules, so forth. How can the Walmart's of the world do that fast, but also do it well? >> Yeah so we do it, we go about it in two ways. One is that out of the box we provide a lot of pre-built integrations to some of the most commonly used systems in an enterprise. All the way from the Mainframes, Oracles, SAPs, Hadoop, Tableaus of the world, they're all available out of the box for you to quickly reuse these objects and build an automated data pipeline. The other challenge we saw, and particularly when we entered the big data space four years ago was that the automation was something that was considered close to the project becoming operational. Okay, and that's where a lot of rework happened because developers had been writing their own scripts using point solutions, so we said alright, it's time to shift automation left, and allow companies to build automations and artifact very early in the developmental life cycle. About a month ago, we released what we call Control-M Workbench, its essentially a community edition of Control-M, targeted towards developers so that instead of writing their own scripts, they can use Control-M in a completely offline manner, without having to connect to an enterprise system. As they build, and test, and iterate, they're using Control-M to do that. So as the application progresses through the development life cycle, and all of that work can then translate easily into an enterprise edition of Control-M. >> Just want to quickly define what shift left means for the folks that might not know software methodologies, they don't think >> Yeah, so. of left political, left or right. >> So, we're not shifting Control-M-- >> Alt-left, alt-right, I mean, this is software development, so quickly take a minute and explain what shift left means, and the importance of it. >> Correct, so if you think of software development as a straight line continuum, you've got, you will start with building some code, you will do some testing, then unit testing, then user acceptance testing. As it moves along this chain, there was a point right before production where all of the automation used to happen. Developers would come in and deliver the application to Ops and Ops would say, well hang on a second, all this Crontab, and these other point solutions we've been using for automation, that's not what we use in production, and we need you to now go right in-- >> So test early and often. >> Test early and often. So the challenge was the developers, the tools they used were not the tools that were being used on the production end of the site. And there was good reason for it, because developers don't need something really heavy and with all the bells and whistles early in the development lifecycle. Now Control-M Workbench is a very light version, which is targeted at developers and focuses on the needs that they have when they're building and developing it. So as the application progresses-- >> How much are you seeing waterfall-- >> But how much can they, go ahead. >> How much are you seeing waterfall, and then people shifting left becoming more prominent now? What percentage of your customers have moved to Agile, and shifting left percentage wise? >> So we survey our customers on a regular basis, and the last survey showed that eighty percent of the customers have either implemented a more continuous integration delivery type of framework, or are in the process of doing it, And that's the other-- >> And getting close to a 100 as possible, pretty much. >> Yeah, exactly. The tipping point is reached. >> And what is driving. >> What is driving all is the need from the business. The days of the five year implementation timelines are gone. This is something that you need to deliver every week, two weeks, and iteration. >> Iteration, yeah, yeah. And we have also innovated in that space, and the approach we call jobs as code, where you can build entire complex data pipelines in code format, so that you can enable the automation in a continuous integration and delivery framework. >> I have one quick question, Jim, and I'll let you take the floor and get a word in soon, but I have one final question on this BMC methodology thing. You guys have a history, obviously BMC goes way back. Remember Max Watson CEO, and Bob Beach, back in '97 we used to chat with him, dominated that landscape. But we're kind of going back to a systems mindset. The question for you is, how do you view the issue of this holy grail, the promised land of AI and machine learning, where end-to-end visibility is really the goal, right? At the same time, you want bounded experiences at root level so automation can kick in to enable more activity. So there's a trade-off between going for the end-to-end visibility out of the gate, but also having bounded visibility and data to automate. How do you guys look at that market? Because customers want the end-to-end promise, but they don't want to try to get there too fast. There's a diseconomies of scale potentially. How do you talk about that? >> Correct. >> And that's exactly the approach we've taken with Control-M Workbench, the Community Edition, because earlier on you don't need capabilities like SLA management and forecasting and automated promotion between environments. Developers want to be able to quickly build and test and show value, okay, and they don't need something that is with all the bells and whistles. We're allowing you to handle that piece, in that manner, through Control-M Workbench. As things progress and the application progresses, the needs change as well. Well now I'm closer to delivering this to the business, I need to be able to manage this within an SLA, I need to be able to manage this end-to-end and connect this to other systems of record, and streaming data, and clickstream data, all of that. So that, we believe that it doesn't have to be a trade off, that you don't have to compromise speed and quality for end-to-end visibility and enterprise grade automation. >> You mentioned trade offs, so the Control-M Workbench, the developer can use it offline, so what amount of testing can they possibly do on a complex data pipeline automation when the tool's offline? I mean it seems like the more development they do offline, the greater the risk that it simply won't work when they go into production. Give us a sense for how they mitigate, the mitigation risk in using Control-M Workbench. >> Sure, so we spend a lot of time observing how developers work, right? And very early in the development stage, all they're doing is working off of their Mac or their laptop, and they're not really connected to any. And that is where they end up writing a lot of scripts, because whatever code business logic they've written, the way they're going to make it run is by writing scripts. And that, essentially, becomes the problem, because then you have scripts managing more scripts, and as the application progresses, you have this complex web of scripts and Crontabs and maybe some opensource solutions, trying to simply make all of this run. And by doing this on an offline manner, that doesn't mean that they're losing all of the other Control-M capabilities. Simply, as the application progresses, whatever automation that the builtin Control-M can seamlessly now flow into the next stage. So when you are ready to take an application into production, there's essentially no rework required from an automation perspective. All of that, that was built, can now be translated into the enterprise-grade Control M, and that's where operations can then go in and add the other artifacts, such as SLA management and forecasting and other things that are important from an operational perspective. >> I'd like to get both your perspectives, 'cause, so you're like an analyst here, so Jim, I want you guys to comment. My question to both of you would be, lookin' at this time in history, obviously in the BMC side we mention some of the history, you guys are transforming on a new journey in extending that capability of this world. Jim, you're covering state-of-the-art AI machine learning. What's your take of this space now? Strata Data, which is now Hadoop World, which is Cloud Air went public, Hortonworks is now public, kind of the big, the Hadoop guys kind of grew up, but the world has changed around them, it's not just about Hadoop anymore. So I'd like to get your thoughts on this kind of perspective, that we're seeing a much broader picture in big data in NYC, versus the Strata Hadoop show, which seems to be losing steam, but I mean in terms of the focus. The bigger focus is much broader, horizontally scalable. And your thoughts on the ecosystem right now? >> Let the Basil answer fist, unless Basil wants me to go first. >> I think that the reason the focus is changing, is because of where the projects are in their lifecycle. Now what we're seeing is most companies are grappling with, how do I take this to the next level? How do I scale? How do I go from just proving out one or two use cases to making the entire organization data driven, and really inject data driven decision making in all facets of decision making? So that is, I believe what's driving the change that we're seeing, that now you've gone from Strata Hadoop to being Strata Data, and focus on that element. And, like I said earlier, the difference between success and failure is your ability to scale and operationalize. Take machine learning for an example. >> Good, that's where there's no, it's not a hype market, it's show me the meat on the bone, show me scale, I got operational concerns of security and what not. >> And machine learning, that's one of the hottest topics. A recent survey I read, which pulled a number of data scientists, it revealed that they spent about less than 3% of their time in training the data models, and about 80% of their time in data manipulation, data transformation and enrichment. That is obviously not the best use of a data scientist's time, and that is exactly one of the problems we're solving for our customers around the world. >> That needs to be automated to the hilt. To help them >> Correct. to be more productive, to deliver faster results. >> Ecosystem perspective, Jim, what's your thoughts? >> Yeah, everything that Basil said, and I'll just point out that many of the core uses cases for AI are automation of the data pipeline. It's driving machine learning driven predictions, classifications, abstractions and so forth, into the data pipeline, into the application pipeline to drive results in a way that is contextually and environmentally aware of what's goin' on. The history, historical data, what's goin' on in terms of current streaming data, to drive optimal outcomes, using predictive models and so forth, in line to applications. So really, fundamentally then, what's goin' on is that automation is an artifact that needs to be driven into your application architecture as a repurposable resource for a variety of-- >> Do customers even know what to automate? I mean, that's the question, what do I-- >> You're automating human judgment. You're automating effort, like the judgments that a working data engineer makes to prepare data for modeling and whatever. More and more that can be automated, 'cause those are pattern structured activities that have been mastered by smart people over many years. >> I mean we just had a customer on with a Glass'Gim CSK, with that scale, and his attitude is, we see the results from the users, then we double down and pay for it and automate it. So the automation question, it's an option question, it's a rhetorical question, but it just begs the question, which is who's writing the algorithms as machines get smarter and start throwing off their own real-time data? What are you looking at? How do you determine? You're going to need machine learning for machine learning? Are you going to need AI for AI? Who writes the algorithms >> It's actually, that's. for the algorithm? >> Automated machine learning is a hot, hot not only research focus, but we're seeing it more and more solution providers, like Microsoft and Google and others, are goin' deep down, doubling down in investments in exactly that area. That's a productivity play for data scientists. >> I think the data markets going to change radically in my opinion. I see you're startin' to some things with blockchain and some other things that are interesting. Data sovereignty, data governance are huge issues. Basil, just give your final thoughts for this segment as we wrap this up. Final thoughts on data and BMC, what should people know about BMC right now? Because people might have a historical view of BMC. What's the latest, what should they know? What's the new Instagram picture of BMC? What should they know about you guys? >> So I think what I would say people should know about BMC is that all the work that we've done over the last 25 years, in virtually every platform that came before Hadoop, we have now innovated to take this into things like big data and cloud platforms. So when you are choosing Control-M as a platform for automation, you are choosing a very, very mature solution, an example of which is Navistar. Their CIO's actually speaking at the Keno tomorrow. They've had Control-M for 15, 20 years, and they've automated virtually every business function through Control-M. And when they started their predictive maintenance project, where they're ingesting data from about 300,000 vehicles today to figure out when this vehicle might break, and to predict maintenance on it. When they started their journey, they said that they always knew that they were going to use Control-M for it, because that was the enterprise standard, and they knew that they could simply now extend that capability into this area. And when they started about three, four years ago, they were ingesting data from about 100,000 vehicles. That has now scaled to over 325,000 vehicles, and they have no had to re-architect their strategy as they grow and scale. So I would say that is one of the key messages that we are taking to market, is that we are bringing innovation that spans over 25 years, and evolving it-- >> Modernizing it, basically. >> Modernizing it, and bringing it to newer platforms. >> Well congratulations, I wouldn't call that a pivot, I'd call it an extensibility issue, kind of modernizing kind of the core things. >> Absolutely. >> Thanks for coming and sharing the BMC perspective inside theCUBE here, on BigData NYC, this is the theCUBE, I'm John Furrier. Jim Kobielus here in New York city. More live coverage, for three days we'll be here, today, tomorrow and Thursday, and BigData NYC, more coverage after this short break. (calm electronic music) (vibrant electronic music)

(upbeat music) >> Live from New York, it's theCUBE. Covering theCUBE New York City 2018. Brought to you by SiliconANGLE Media and its ecosystem partners. >> Okay, welcome back everyone to theCUBE NYC. This is theCUBE's live coverage covering CubeNYC Strata Hadoop Strata Data Conference. All things data happen here in New York this week. I'm John Furrier with Peter Burris. Our next guest is Basil Faruqui lead solutions marketing manager digital business automation within BMC returns, he was here last year with us and also Big Data SV, which has been renamed CubeNYC, Cube SV because it's not just big data anymore. We're hearing words like multi cloud, Istio, all those Kubernetes. Data now is so important, it's now up and down the stack, impacting everyone, we talked about this last year with Control M, how you guys are automating in a hurry. The four pillars of pipelining data. The setup days are over; welcome to theCUBE. >> Well thank you and it's great to be back on theCUBE. And yeah, what you said is exactly right, so you know, big data has really, I think now been distilled down to data. Everybody understands data is big, and it's important, and it is really you know, it's quite a cliche, but to a larger degree, data is the new oil, as some people say. And I think what you said earlier is important in that we've been very fortunate to be able to not only follow the journey of our customers but be a part of it. So about six years ago, some of the early adopters of Hadoop came to us and said that look, we use your products for traditional data warehousing on the ERP side for orchestration workloads. We're about to take some of these projects on Hadoop into production and really feel that the Hadoop ecosystem is lacking enterprise-grade workflow orchestration tools. So we partnered with them and some of the earliest goals they wanted to achieve was build a data lake, provide richer and wider data sets to the end users to be able to do some dashboarding, customer 360, and things of that nature. Very quickly, in about five years time, we have seen a lot of these projects mature from how do I build a data lake to now applying cutting-edge ML and AI and cloud is a major enabler of that. You know, it's really, as we were talking about earlier, it's really taking away excuses for not being able to scale quickly from an infrastructure perspective. Now you're talking about is it Hadoop or is it S3 or is it Azure Blob Storage, is it Snowflake? And from a control-end perspective, we're very platform and technology agnostic, so some of our customers who had started with Hadoop as a platform, they are now looking at other technologies like Snowflake, so one of our customers describes it as kind of the spine or a power strip of orchestration where regardless of what technology you have, you can just plug and play in and not worry about how do I rewire the orchestration workflows because control end is taking care of it. >> Well you probably always will have to worry about that to some degree. But I think where you're going, and this is where I'm going to test with you, is that as analytics, as data is increasingly recognized as a strategic asset, as analytics increasingly recognizes the way that you create value out of those data assets, and as a business becomes increasingly dependent upon the output of analytics to make decisions and ultimately through AI to act differently in markets, you are embedding these capabilities or these technologies deeper into business. They have to become capabilities. They have to become dependable. They have to become reliable, predictable, cost, performance, all these other things. That suggests that ultimately, the historical approach of focusing on the technology and trying to apply it to a periodic or series of data science problems has to become a little bit more mature so it actually becomes a strategic capability. So the business can say we're operating on this, but the technologies to take that underlying data science technology to turn into business operations that's where a lot of the net work has to happen. Is that what you guys are focused on? >> Yeah, absolutely, and I think one of the big differences that we're seeing in general in the industry is that this time around, the pull of how do you enable technology to drive the business is really coming from the line of business, versus starting on the technology side of the house and then coming to the business and saying hey we've got some cool technologies that can probably help you, it's really line of business now saying no, I need better analytics so I can drive new business models for my company, right? So the need for speed is greater than ever because the pull is from the line of business side. And this is another area where we are unique is that, you know, Control M has been designed in a way where it's not just a set of solutions or tools for the technical guys. Now, the line of business is getting closer and closer, you know, it's blending into the technical side as well. They have a very, very keen interest in understanding are the dashboards going to be refreshed on time? Are we going to be able to get all the right promotional offers at the right time? I mean, we're here at NYC Strata, there's a lot of real-time promotion happening here. The line of business has direct interest in the delivery and the timing of all of this, so we have always had multiple interfaces to Control M where a business user who has an interest in understanding are the promotional offers going to happen at the right time and is that on schedule? They have a mobile app for them to do that. A developer who's building up complex, multi-application platform, they have an API and a programmatic interface to do that. Operations that has to monitor all of this has rich dashboards to be able to do that. That's one of the areas that has been key for our success over the last couple decades, and we're seeing that translate very well into the big data place. >> So I just want to go under the hood for a minute because I love that answer. And I'd like to pivot off what Peter said, tying it back to the business, okay, that's awesome. And I want to learn a little bit more about this because we talked about this last year and I kind of am seeing it now. Kubernetes and all this orchestration is about workloads. You guys nailed the workflow issue, complex workflows. Because if you look at it, if you're adding line of business into the equation, that's just complexity in and of itself. As more workflows exist within its own line of business, whether it's recommendations and offers and workflow issues, more lines of business in there is complex for even IT to deal with, so you guys have nailed that. How does that work? Do you plug it in and the lines of businesses have their own developers, so the people who work with the workflows engage how? >> So that's a good question, with sort of orchestration and automation now becoming very, very generic, it's kind of important to classify where we play. So there's a lot of tools that do release and build automation. There's a lot of tools that'll do infrastructure automation and orchestration. All of this infrastructure and release management process is done ultimately to run applications on top of it, and the workflows of the application need orchestration and that's the layer that we play in. And if you think about how does the end user, the business and consumer interact with all of this technology is through applications, k? So the orchestration of the workflow's inside the applications, whether you start all the way from an ERP or a CRM and then you land into a data lake and then do an ML model, and then out come the recommendations analytics, that's the layer we are automating today. Obviously, all of this-- >> By the way, the technical complexity for the user's in the app. >> Correct, so the line of business obviously has a lot more control, you're seeing roles like chief digital officers emerge, you're seeing CTOs that have mandates like okay you're going to be responsible for all applications that are facing customer facing where the CIO is going to take care of everything that's inward facing. It's not a settled structure or science involved. >> It's evolving fast. >> It's evolving fast. But what's clear is that line of business has a lot more interest and influence in driving these technology projects and it's important that technologies evolve in a way where line of business can not only understand but take advantage of that. >> So I think it's a great question, John, and I want to build on that and then ask you something. So the way we look at the world is we say the first fifty years of computing were known process, unknown technology. The next fifty years are going to be unknown process, known technology. It's all going to look like a cloud. But think about what that means. Known process, unknown technology, Control M and related types of technologies tended to focus on how you put in place predictable workflows in the technology layer. And now, unknown process, known technology, driven by the line of business, now we're talking about controlling process flows that are being created, bespoke, strategic, differentiating doing business. >> Well, dynamic, too, I mean, dynamic. >> Highly dynamic, and those workflows in many respects, those technologies, piecing applications and services together, become the process that differentiates the business. Again, you're still focused on the infrastructure a bit, but you've moved it up. Is that right? >> Yeah, that's exactly right. We see our goal as abstracting the complexity of the underlying application data and infrastructure. So, I mean, it's quite amazing-- >> So it could be easily reconfigured to a business's needs. >> Exactly, so whether you're on Hadoop and now you're thinking about moving to Snowflake or tomorrow something else that comes up, the orchestration or the workflow, you know, that's as a business as a product that's our goal is to continue to evolve quickly and in a manner that we continue to abstract the complexity so from-- >> So I've got to ask you, we've been having a lot of conversations around Hadoop versus Kubernetes on multi cloud, so as cloud has certainly come in and changed the game, there's no debate on that. How it changes is debatable, but we know that multiple clouds is going to be the modus operandus for customers. >> Correct. >> So I got a lot of data and now I've got pipelining complexities and workflows are going to get even more complex, potentially. How do you see the impact of the cloud, how are you guys looking at that, and what are some customer use cases that you see for you guys? >> So the, what I mentioned earlier, that being platform and technology agnostic is actually one of the unique differentiating factors for us, so whether you are an AWS or an Azure or a Google or On-Prem or still on a mainframe, a lot of, we're in New York, a lot of the banks, insurance companies here still do some of the most critical processing on the mainframe. The ability to abstract all of that whether it's cloud or legacy solutions is one of our key enablers for our customers, and I'll give you an example. So Malwarebytes is one of our customers and they've been using Control M for several years. Primarily the entire structure is built on AWS, but they are now utilizing Google cloud for some of their recommendation analysis on sentiment analysis because their goal is to pick the best of breed technology for the problem they're looking to solve. >> Service, the best breed service is in the cloud. >> The best breed service is in the cloud to solve the business problem. So from Control M's perspective, transcending from AWS to Google cloud is completely abstracted for them, so runs Google tomorrow it's Azure, they decide to build a private cloud, they will be able to extend the same workflow orchestration. >> But you can build these workflows across whatever set of services are available. >> Correct, and you bring up an important point. It's not only being able to build the workflows across platforms but being able to define dependencies and track the dependencies across all of this, because none of this is happening in silos. If you want to use Google's API to do the recommendations, well, you've got to feed it the data, and the data's pipeline, like we talked about last time, data ingestion, data storage, data processing, and analytics have very, very intricate dependencies, and these solutions should be able to manage not only the building of the workflow but the dependencies as well. >> But you're defining those elements as fundamental building blocks through a control model >> Correct. >> That allows you to treat the higher level services as reliable, consistent, capabilities. >> Correct, and the other thing I would like to add here is not only just build complex multiplatform, multiapplication workflows, but never lose focus of the business service of the business process there, so you can tie all of this to a business service and then, these things are complex, there are problems, let's say there's an ETL job that fails somewhere upstream, Control M will immediately be able to predict the impact and be able to tell you this means the recommendation engine will not be able to make the recommendations. Now, the staff that's going to work under mediation understands the business impact versus looking at a screen where there's 500 jobs and one of them has failed. What does that really mean? >> Set priorities and focal points and everything else. >> Right. >> So I just want to wrap up by asking you how your talk went at Strata Hadoop Data Conference. What were you talking about, what was the core message? Was it Control M, was it customer presentations? What was the focus? >> So the focus of yesterday's talk was actually, you know, one of the things is academic talk is great, but it's important to, you know, show how things work in real life. The session was focused on a real-use case from a customer. Navistar, they have IOT data-driven pipelines where they are predicting failures of parts inside trucks and buses that they manufacture, you know, reducing vehicle downtime. So we wanted to simulate a demo like that, so that's exactly what we did. It was very well received. In real-time, we spun up EMR environment in AWS, automatically provision control of infrastructure there, we applied spark and machine learning algorithms to the data and out came the recommendation at the end was that, you know, here are the vehicles that are-- >> Fix their brakes. (laughing) >> Exactly, so it was very, very well received. >> I mean, there's a real-world example, there's real money to be saved, maintenance, scheduling, potential liability, accidents. >> Liability is a huge issue for a lot of manufacturers. >> And Navistar has been at the leading edge of how to apply technologies in that business. >> They really have been a poster child for visual transformation. >> They sure have. >> Here's a company that's been around for 100 plus years and when we talk to them they tell us that we have every technology under the sun that has come since the mainframe, and for them to be transforming and leading in this way, we're very fortunate to be part of their journey. >> Well we'd love to talk more about some of these customer use cases. Other people love about theCUBE, we want to do more of them, share those examples, people love to see proof in real-world examples, not just talk so appreciate it sharing. >> Absolutely. >> Thanks for sharing, thanks for the insights. We're here Cube live in New York City, part of CubeNYC, we're getting all the data, sharing that with you. I'm John Furrier with Peter Burris. Stay with us for more day two coverage after this short break. (upbeat music)

>> Live from New York, It's theCUBE! Covering theCUBE New York City 2018. Brought to you by SiliconANGLE Media and its ecosystem partners. >> Hello and welcome back to theCUBE live in New York City, here for CUBE NYC. In conjunct with Strata Conference, Strata Data, Strata Hadoop This is our ninth year covering the big data ecosystem which has evolved into machine learning, A.I., data science, cloud, a lot of great things happening all things data, impacting all businesses I'm John Furrier, your host with Dave Vellante and Peter Burris, Peter is filling in for Dave Vellante. Next guest, Stephanie McReynolds who is the CMO, VP of Marketing for Alation, thanks for joining us. >> Thanks for having me. >> Good to see you. So you guys have a pretty spectacular exhibit here in New York. I want to get to that right away, top story is Attack of the Bots. And you're showing a great demo. Explain what you guys are doing in the show. >> Yah, well it's robot fighting time in our booth, so we brought a little fun to the show floor my kids are.. >> You mean big data is not fun enough? >> Well big data is pretty fun but occasionally you got to get your geek battle on there so we're having fun with robots but I think the real story in the Alation booth is about the product and how machine learning data catalogs are helping a whole variety of users in the organization everything from improving analyst productivity and even some business user productivity of data to then really supporting data scientists in their work by helping them to distribute their data products through a data catalog. >> You guys are one of the new guard companies that are doing things that make it really easy for people who want to use data, practitioners that the average data citizen has been called, or people who want productivity. Not necessarily the hardcore, setting up clusters, really kind of like the big data user. What's that market look like right now, has it met your expectations, how's business, what's the update? >> Yah, I think we have a strong perspective that for us to close the final mile and get to real value out of the data, it's a human challenge, there's a trust gap with managers. Today on stage over at STRATA it was interesting because Google had a speaker and it wasn't their chief data officer it was their chief decision scientist and I think that reflects what that final mile is is that making decisions and it's the trust gap that managers have with data because they don't know how the insides are coming to them, what are all the details underneath. In order to be able to trust decisions you have to understand who processed the data, what decision making criteria did they use, was this data governed well, are we introducing some bias into our algorithms, and can that be controlled? And so Alation becomes a platform for supporting getting answers to those issues. And then there's plenty of other companies that are optimizing the performance of those QUERYS and the storage of that data, but we're trying to really to close that trust gap. >> It's very interesting because from a management standpoint we're trying to do more evidence based management. So there's a major trend in board rooms, and executive offices to try to find ways to acculturate the executive team to using data, evidence based management healthcare now being applied to a lot of other domains. We've also historically had a situation where the people who focused or worked with the data was a relatively small coterie of individuals that crave these crazy systems to try to bring those two together. It sounds like what you're doing, and I really like the idea of the data scientists, being able to create data products that then can be distributed. It sounds like you're trying to look at data as an asset to be created, to be distributed so they can be more easily used by more people in your organization, have we got that right? >> Absolutely. So we're now seeing we're in just over a hundred production implementations of Alation, at large enterprises, and we're now seeing those production implementations get into the thousands of users. So this is going beyond those data specialists. Beyond the unicorn data scientists that understand the systems and math and technology. >> And business. >> And business, right. In business. So what we're seeing now is that a data catalog can be a point of collaboration across those different audiences in an enterprise. So whereas three years ago some of our initial customers kept the data catalog implementations small, right. They were getting access to the specialists to this catalog and asked them to certify data assets for others, what were starting to see is a proliferation of creation of self service data assets, a certification process that now is enterprise-wide, and thousands of users in these organizations. So Ebay has over a thousand weekly logins, Munich Reinsurance was on stage yesterday, their head of data engineering said they have 2,000 users on Alation at this point on their data lake, Fiserv is going to speak on Thursday and they're getting up to those numbers as well, so we see some really solid organizations that are solving medical, pharmaceutical issues, right, the largest re insurer in the world leading tech companies, starting to adopt a data catalog as a foundation for how their going to make those data driven decisions in the organization. >> Talk about how the product works because essentially you're bringing kind of the decision scientists, for lack of a better word, and productivity worker, almost like a business office suite concept, as a SAS, so you got a SAS model that says "Hey you want to play with data, use it but you have to do some front end work." Take us through how you guys roll out the platform, how are your customers consuming the service, take us through the engagement with customers. >> I think for customers, the most interesting part of this product is that it displays itself as an application that anyone can use, right? So there's a super familiar search interface that, rather than bringing back webpages, allows you to search for data assets in your organization. If you want more information on that data asset you click on those search results and you can see all of the information of how that data has been used in the organization, as well as the technical details and the technical metadata. And I think what's even more powerful is we actually have a recommendation engine that recommends data assets to the user. And that can be plugged into Tablo and Salesworth, Einstein Analytics, and a whole variety of other data science tools like Data Haiku that you might be using in your organization. So this looks like a very easy to use application that folks are familiar with that you just need a web browser to access, but on the backend, the hard work that's happening is the automation that we do with the platform. So by going out and crawling these source systems and looking at not just the technical descriptions of data, the metadata that exists, but then being able to understand by parsing the sequel weblogs, how that data is actually being used in the organization. We call it behavior I.O. by looking at the behaviors of how that data's being used, from those logs, we can actually give you a really good sense of how that data should be used in the future or where you might have gaps in governing that data or how you might want to reorient your storage or compute infrastructure to support the type of analytics that are actually being executed by real humans in your organization. And that's eye opening to a lot of I.T. sources. >> So you're providing insights to the data usage so that the business could get optimized for whether it's I.T. footprint component, or kinds of use cases, is that kind of how it's working? >> So what's interesting is the optimization actually happens in a pretty automated way, because we can make recommendations to those consumers of data of how they want to navigate the system. Kind of like Google makes recommendations as you browse the web, right? >> If you misspell something, "Oh did you mean this", kind of thing? >> "Did you mean this, might you also be interested in this", right? It's kind of a cross between Google and Amazon. Others like you may have used these other data assets in the past to determine revenue for that particular region, have you thought about using this filter, have you thought about using this join, did you know that you're trying to do analysis that maybe the sales ops guy has already done, and here's the certified report, why don't you just start with that? We're seeing a lot of reuse in organizations, wherein the past I think as an industry when Tablo and Click and all these B.I tools that were very self service oriented started to take off it was all about democratizing visualization by letting every user do their own thing and now we're realizing to get speed and accuracy and efficiency and effectiveness maybe there's more reuse of the work we've already done in existing data assets and by recommending those and expanding the data literacy around the interpretation of those, you might actually close this trust gap with the data. >> But there's one really important point that you raised, and I want to come back to it, and that is this notion of bias. So you know, Alation knows something about the data, knows a lot about the metadata, so therefore, I don't want to say understands, but it's capable of categorizing data in that way. And you're also able to look at the usage of that data by parsing some of sequel statements and then making a determination of the data as it's identified is appropriately being used based on how people are actually applying it so you can identify potential bias or potential misuse or whatever else it might be. That is an incredibly important thing. As you know John, we had an event last night and one of the things that popped up is how do you deal with emergence in data science in A.I, etc. And what methods do you put in place to actually ensure that the governance model can be extended to understand how those things are potentially in a very soft way, corrupting the use of the data. So could you spend a little bit more time talking about that because it's something a lot of people are interested in, quite frankly we don't know about a lot of tools that are doing that kind of work right now. It's an important point. >> I think the traditional viewpoint was if we just can manage the data we will be able to have a govern system. So if we control the inputs then well have a safe environment, and that was kind of like the classic single source of truth, data warehouse type model. >> Stewards of the data. >> What we're seeing is with the proliferation of sources of data and how quickly with IOT and new modern sources, data is getting created, you're not able to manage data at that point of that entry point. And it's not just about systems, it's about individuals that go on the web and find a dataset and then load it into a corporate database, right? Or you merge an Excel file with something that in a database. And so I think what we see happening, not only when you look at bias but if you look at some of the new regulations like [Inaudible] >> Sure. Ownership, [Inaudible] >> The logic that you're using to process that data, the algorithm itself can be biased, if you have a biased training data site that you feed it into a machine learning algorithm, the algorithm itself is going to be biased. And so the control point in this world where data is proliferating and we're not sure we can control that entirely, becomes the logic embedded in the algorithm. Even if that's a simple sequel statement that's feeding a report. And so Alation is able to introspect that sequel and highlight that maybe there is bias at work and how this algorithm is composed. So with GDPR the consumer owns their own data, if they want to pull it out from a training data set, you got to rerun that algorithm without that consumer data and that's your control point then going forward for the organization on different governance issues that pop up. >> Talk about the psychology of the user base because one of the things that shifted in the data world is a few stewards of data managed everything, now you've got a model where literally thousands of people of an organization could be users, productivity users, so you get a social component in here that people know who's doing data work, which in a way, creates a new persona or class of worker. A non techy worker. >> Yeah. It's interesting if you think about moving access to the data and moving the individuals that are creating algorithms out to a broader user group, what's important, you have to make sure that you're educating and training and sharing knowledge with that democratized audience, right? And to be able to do that you kind of want to work with human psychology, right? You want to be able to give people guidance in the course of their work rather than have them memorize a set of rules and try to remember to apply those. If you had a specialist group you can kind of control and force them to memorize and then apply, the more modern approach is to say "look, with some of these machine learning techniques that we have, why don't we make a recommendation." What you're going to do is introduce bias into that calculation. >> And we're capturing that information as you use the data. >> Well were also making a recommendation to say "Hey do you know you're doing this? Maybe you don't want to do that." Most people are using the data are not bad actors. They just can't remember all the rule sets to apply. So what were trying to do is cut someone behaviorally in the act before they make that mistake and say hey just a bit of a reminder, a bit of a coaching moment, did you know what you're doing? Maybe you can think of another approach to this. And we've found that many organizations that changes the discussion around data governance. It's no longer this top down constraint to finding insight, which frustrates an audience, is trying to use that data. It's more like a coach helping you improve and then social aspect of wanting to contribute to the system comes into play and people start communicating, collaborating, the platform and curating information a little bit. >> I remember when Microsoft Excel came out, the spreadsheet, or Lotus 123, oh my God, people are going to use these amazing things with spreadsheets, they did. You're taking a similar approach with analytics, much bigger surface area of work to kind of attack from a data perspective, but in a way kind of the same kind of concept, put the hands of the users, have the data in their hands so to speak. >> Yeah, enable everyone to make data driven decisions. But make sure that they're interpreting that data in the right way, right? Give them enough guidance, don't let them just kind of attack the wild west and fair it out. >> Well looking back at the Microsoft Excel spreadsheet example, I remember when a finance department would send a formatted spreadsheet with all the rules for how to use it out of 50 different groups around the world, and everyone figured out that you can go in and manipulate the macros and deliver any results they want. And so it's that same notion, you have to know something about that, but this site, in many respects Stephanie you're describing a data governance model that really is more truly governance, that if we think about a data asset it's how do we mediate a lot of different claims against that set of data so that its used appropriately, so its not corrupted, so that it doesn't effect other people, but very importantly so that the out6comes are easier to agree upon because there's some trust and there's some valid behaviors and there's some verification in the flow of the data utilization. >> And where we give voice to a number of different constituencies. Because business opinions from different departments can run slightly counter to one another. There can be friction in how to use particular data assets in the business depending on the lens that you have in that business and so what were trying to do is surface those different perspectives, give them voice, allow those constituencies to work that out in a platform that captures that debate, captures that knowledge, makes that debate a knowledge of foundation to build upon so in many ways its kind of like the scientific method, right? As a scientist I publish a paper. >> Get peer reviewed. >> Get peer reviewed, let other people weigh in. >> And it becomes part of the canon of knowledge. >> And it becomes part of the canon. And in the scientific community over the last several years you see that folks are publishing their data sets out publicly, why can't an enterprise do the same thing internally for different business groups internally. Take the same approach. Allow others to weigh in. It gets them better insights and it gets them more trust in that foundation. >> You get collective intelligence from the user base to help come in and make the data smarter and sharper. >> Yeah and have reusable assets that you can then build upon to find the higher level insights. Don't run the same report that a hundred people in the organization have already run. >> So the final question for you. As you guys are emerging, starting to do really well, you have a unique approach, honestly we think it fits in kind of the new guard of analytics, a productivity worker with data, which is we think is going to be a huge persona, where are you guys winning, and why are you winning with your customer base? What are some things that are resonating as you go in and engage with prospects and customers and existing customers? What are they attracted to, what are they like, and why are you beating the competition in your sales and opportunities? >> I think this concept of a more agile, grassroots approach to data governance is a breath of fresh air for anyone who spend their career in the data space. Were at a turning point in industry where you're now seeing chief decision scientists, chief data officers, chief analytic officers take a leadership role in organizations. Munich Reinsurance is using their data team to actually invest and hold new arms of their business. That's how they're pushing the envelope on leadership in the insurance space and were seeing that across our install base. Alation becomes this knowledge repository for all of those mines in the organization, and encourages a community to be built around data and insightful questions of data. And in that way the whole organization raises to the next level and I think its that vision of what can be created internally, how we can move away from just claiming that were a big data organization and really starting to see the impact of how new business models can be creative in these data assets, that's exciting to our customer base. >> Well congratulations. A hot start up. Alation here on theCUBE in New York City for cubeNYC. Changing the game on analytics, bringing a breath of fresh air to hands of the users. A new persona developing. Congratulations, great to have you. Stephanie McReynolds. Its the cube. Stay with us for more live coverage, day one of two days live in New York City. We'll be right back.

>> Live from New York, it's theCUBE covering theCUBE New York City 2018. Brought to you by SiliconANGLE Media and its ecosystem partners. (techy music) >> Hello, everyone, welcome to this CUBE special presentation here in New York City for CUBENYC. I'm John Furrier with Dave Vellante. This is our ninth year covering the big data industry, starting with Hadoop World and evolved over the years. This is our ninth year, Dave. We've been covering Hadoop World, Hadoop Summit, Strata Conference, Strata Hadoop. Now it's called Strata Data, I don't know what Strata O'Reilly's going to call it next. As you all know, theCUBE has been present for the creation at the Hadoop big data ecosystem. We're here for our ninth year, certainly a lot's changed. AI's the center of the conversation, and certainly we've seen some horses come in, some haven't come in, and trends have emerged, some gone away, your thoughts. Nine years covering big data. >> Well, John, I remember fondly, vividly, the call that I got. I was in Dallas at a storage networking world show and you called and said, "Hey, we're doing "Hadoop World, get over there," and of course, Hadoop, big data, was the new, hot thing. I told everybody, "I'm leaving." Most of the people said, "What's Hadoop?" Right, so we came, we started covering, it was people like Jeff Hammerbacher, Amr Awadallah, Doug Cutting, who invented Hadoop, Mike Olson, you know, head of Cloudera at the time, and people like Abi Mehda, who at the time was at B of A, and some of the things we learned then that were profound-- >> Yeah. >> As much as Hadoop is sort of on the back burner now and people really aren't talking about it, some of the things that are profound about Hadoop, really, were the idea, the notion of bringing five megabytes of code to a petabyte of data, for example, or the notion of no schema on write. You know, put it into the database and then figure it out. >> Unstructured data. >> Right. >> Object storage. >> And so, that created a state of innovation, of funding. We were talking last night about, you know, many, many years ago at this event this time of the year, concurrent with Strata you would have VCs all over the place. There really aren't a lot of VCs here this year, not a lot of VC parties-- >> Mm-hm. >> As there used to be, so that somewhat waned, but some of the things that we talked about back then, we said that big money and big data is going to be made by the practitioners, not by the vendors, and that's proved true. I mean... >> Yeah. >> The big three Hadoop distro vendors, Cloudera, Hortonworks, and MapR, you know, Cloudera's $2.5 billion valuation, you know, not bad, but it's not a $30, $40 billion value company. The other thing we said is there will be no Red Hat of big data. You said, "Well, the only Red Hat of big data might be "Red Hat," and so, (chuckles) that's basically proved true. >> Yeah. >> And so, I think if we look back we always talked about Hadoop and big data being a reduction, the ROI was a reduction on investment. >> Yeah. >> It was a way to have a cheaper data warehouse, and that's essentially-- Well, what did we get right and wrong? I mean, let's look at some of the trends. I mean, first of all, I think we got pretty much everything right, as you know. We tend to make the calls pretty accurately with theCUBE. Got a lot of data, we look, we have the analytics in our own system, plus we have the research team digging in, so you know, we pretty much get, do a good job. I think one thing that we predicted was that Hadoop certainly would change the game, and that did. We also predicted that there wouldn't be a Red Hat for Hadoop, that was a production. The other prediction was is that we said Hadoop won't kill data warehouses, it didn't, and then data lakes came along. You know my position on data lakes. >> Yeah. >> I've always hated the term. I always liked data ocean because I think it was much more fluidity of the data, so I think we got that one right and data lakes still doesn't look like it's going to be panning out well. I mean, most people that deploy data lakes, it's really either not a core thing or as part of something else and it's turning into a data swamp, so I think the data lake piece is not panning out the way it, people thought it would be. I think one thing we did get right, also, is that data would be the center of the value proposition, and it continues and remains to be, and I think we're seeing that now, and we said data's the development kit back in 2010 when we said data's going to be part of programming. >> Some of the other things, our early data, and we went out and we talked to a lot of practitioners who are the, it was hard to find in the early days. They were just a select few, I mean, other than inside of Google and Yahoo! But what they told us is that things like SQL and the enterprise data warehouse were key components on their big data strategy, so to your point, you know, it wasn't going to kill the EDW, but it was going to surround it. The other thing we called was cloud. Four years ago our data showed clearly that much of this work, the modeling, the big data wrangling, et cetera, was being done in the cloud, and Cloudera, Hortonworks, and MapR, none of them at the time really had a cloud strategy. Today that's all they're talking about is cloud and hybrid cloud. >> Well, it's interesting, I think it was like four years ago, I think, Dave, when we actually were riffing on the notion of, you know, Cloudera's name. It's called Cloudera, you know. If you spell it out, in Cloudera we're in a cloud era, and I think we were very aggressive at that point. I think Amr Awadallah even made a comment on Twitter. He was like, "I don't understand "where you guys are coming from." We were actually saying at the time that Cloudera should actually leverage more cloud at that time, and they didn't. They stayed on their IPO track and they had to because they had everything betted on Impala and this data model that they had and being the business model, and then they went public, but I think clearly cloud is now part of Cloudera's story, and I think that's a good call, and it's not too late for them. It never was too late, but you know, Cloudera has executed. I mean, if you look at what's happened with Cloudera, they were the only game in town. When we started theCUBE we were in their office, as most people know in this industry, that we were there with Cloudera when they had like 17 employees. I thought Cloudera was going to run the table, but then what happened was Hortonworks came out of the Yahoo! That, I think, changed the game and I think in that competitive battle between Hortonworks and Cloudera, in my opinion, changed the industry, because if Hortonworks did not come out of Yahoo! Cloudera would've had an uncontested run. I think the landscape of the ecosystem would look completely different had Hortonworks not competed, because you think about, Dave, they had that competitive battle for years. The Hortonworks-Cloudera battle, and I think it changed the industry. I think it couldn't been a different outcome. If Hortonworks wasn't there, I think Cloudera probably would've taken Hadoop and making it so much more, and I think they wouldn't gotten more done. >> Yeah, and I think the other point we have to make here is complexity really hurt the Hadoop ecosystem, and it was just bespoke, new projects coming out all the time, and you had Cloudera, Hortonworks, and maybe to a lesser extent MapR, doing a lot of the heavy lifting, particularly, you know, Hortonworks and Cloudera. They had to invest a lot of their R&D in making these systems work and integrating them, and you know, complexity just really broke the back of the Hadoop ecosystem, and so then Spark came in, everybody said, "Oh, Spark's going to basically replace Hadoop." You know, yes and no, the people who got Hadoop right, you know, embraced it and they still use it. Spark definitely simplified things, but now the conversation has turned to AI, John. So, I got to ask you, I'm going to use your line on you in kind of the ask-me-anything segment here. AI, is it same wine, new bottle, or is it really substantively different in your opinion? >> I think it's substantively different. I don't think it's the same wine in a new bottle. I'll tell you... Well, it's kind of, it's like the bad wine... (laughs) Is going to be kind of blended in with the good wine, which is now AI. If you look at this industry, the big data industry, if you look at what O'Reilly did with this conference. I think O'Reilly really has not done a good job with the conference of big data. I think they blew it, I think that they made it a, you know, monetization, closed system when the big data business could've been all about AI in a much deeper way. I think AI is subordinate to cloud, and you mentioned cloud earlier. If you look at all the action within the AI segment, Diane Greene talking about it at Google Next, Amazon, AI is a software layer substrate that will be underpinned by the cloud. Cloud will drive more action, you need more compute, that drives more data, more data drives the machine learning, machine learning drives the AI, so I think AI is always going to be dependent upon cloud ends or some sort of high compute resource base, and all the cloud analytics are feeding into these AI models, so I think cloud takes over AI, no doubt, and I think this whole ecosystem of big data gets subsumed under either an AWS, VMworld, Google, and Microsoft Cloud show, and then also I think specialization around data science is going to go off on its own. So, I think you're going to see the breakup of the big data industry as we know it today. Strata Hadoop, Strata Data Conference, that thing's going to crumble into multiple, fractured ecosystems. >> It's already starting to be forked. I think the other thing I want to say about Hadoop is that it actually brought such great awareness to the notion of data, putting data at the core of your company, data and data value, the ability to understand how data at least contributes to the monetization of your company. AI would not be possible without the data. Right, and we've talked about this before. You call it the innovation sandwich. The innovation sandwich, last decade, last three decades, has been Moore's law. The innovation sandwich going forward is data, machine intelligence applied to that data, and cloud for scale, and that's the sandwich of innovation over the next 10 to 20 years. >> Yeah, and I think data is everywhere, so this idea of being a categorical industry segment is a little bit off, I mean, although I know data warehouse is kind of its own category and you're seeing that, but I don't think it's like a Magic Quadrant anymore. Every quadrant has data. >> Mm-hm. >> So, I think data's fundamental, and I think that's why it's going to become a layer within a control plane of either cloud or some other system, I think. I think that's pretty clear, there's no, like, one. You can't buy big data, you can't buy AI. I think you can have AI, you know, things like TensorFlow, but it's going to be a completely... Every layer of the stack is going to be impacted by AI and data. >> And I think the big players are going to infuse their applications and their databases with machine intelligence. You're going to see this, you're certainly, you know, seeing it with IBM, the sort of Watson heavy lift. Clearly Google, Amazon, you know, Facebook, Alibaba, and Microsoft, they're infusing AI throughout their entire set of cloud services and applications and infrastructure, and I think that's good news for the practitioners. People aren't... Most companies aren't going to build their own AI, they're going to buy AI, and that's how they close the gap between the sort of data haves and the data have-nots, and again, I want to emphasize that the fundamental difference, to me anyway, is having data at the core. If you look at the top five companies in terms of market value, US companies, Facebook maybe not so much anymore because of the fake news, though Facebook will be back with it's two billion users, but Apple, Google, Facebook, Amazon, who am I... And Microsoft, those five have put data at the core and they're the most valuable companies in the stock market from a market cap standpoint, why? Because it's a recognition that that intangible value of the data is actually quite valuable, and even though banks and financial institutions are data companies, their data lives in silos. So, these five have put data at the center, surrounded it with human expertise, as opposed to having humans at the center and having data all over the place. So, how do they, how do these companies close the gap? How do the companies in the flyover states close the gap? The way they close the gap, in my view, is they buy technologies that have AI infused in it, and I think the last thing I'll say is I see cloud as the substrate, and AI, and blockchain and other services, as the automation layer on top of it. I think that's going to be the big tailwind for innovation over the next decade. >> Yeah, and obviously the theme of machine learning drives a lot of the conversations here, and that's essentially never going to go away. Machine learning is the core of AI, and I would argue that AI truly doesn't even exist yet. It's machine learning really driving the value, but to put a validation on the fact that cloud is going to be driving AI business is some of the terms in popular conversations we're hearing here in New York around this event and topic, CUBENYC and Strata Conference, is you're hearing Kubernetes and blockchain, and you know, these automation, AI operation kind of conversations. That's an IT conversation, (chuckles) so you know, that's interesting. You've got IT, really, with storage. You've got to store the data, so you can't not talk about workloads and how the data moves with workloads, so you're starting to see data and workloads kind of be tossed in the same conversation, that's a cloud conversation. That is all about multi-cloud. That's why you're seeing Kubernetes, a term I never thought I would be saying at a big data show, but Kubernetes is going to be key for moving workloads around, of which there's data involved. (chuckles) Instrumenting the workloads, data inside the workloads, data driving data. This is where AI and machine learning's going to play, so again, cloud subsumes AI, that's the story, and I think that's going to be the big trend. >> Well, and I think you're right, now. I mean, that's why you're hearing the messaging of hybrid cloud and from the big distro vendors, and the other thing is you're hearing from a lot of the no-SQL database guys, they're bringing ACID compliance, they're bringing enterprise-grade capability, so you're seeing the world is hybrid. You're seeing those two worlds come together, so... >> Their worlds, it's getting leveled in the playing field out there. It's all about enterprise, B2B, AI, cloud, and data. That's theCUBE bringing you the data here. New York City, CUBENYC, that's the hashtag. Stay with us for more coverage live in New York after this short break. (techy music)

>> Live from midtown Manhattan, it's theCUBE covering Big Data New York City 2017. Brought to you by Silicon Angle Media and its ecosystem sponsors. (upbeat techno music) >> Welcome back, everyone. Live here in New York, Day Three coverage, winding down for three days of wall to wall coverage theCUBE covering Big Data NYC in conjunction with Strata Data, formerly Strata Hadoop and Hadoop World, all part of the Big Data ecosystem. Our next guest is Nenshad Bardoliwalla Co-Founder and Chief Product Officer of Paxata, hot start up in the space. A lot of kudos. Of course, they launched on theCUBE in 2013 three years ago when we started theCUBE as a separate event from O'Reilly. So, great to see the success. And Stephanie McReynolds, you've been on multiple times, VP of Marketing at Alation. Welcome back, good to see you guys. >> Thank you. >> Happy to be here. >> So, winding down, so great kind of wrap-up segment here in addition to the partnership that you guys have. So, let's first talk about before we get to the wrap-up of the show and kind of bring together the week here and kind of summarize everything. Tell about your partnership you guys have. Paxata, you guys have been doing extremely well. Congratulations. Prakash was talking on theCUBE. Great success. You guys worked hard for it. I'm happy for you. But partnering is everything. Ecosystem is everything. Alation, their collaboration with data. That's there ethos. They're very user-centric. >> Nenshad: Yes. >> From the founders. Seemed like a good fit. What's the deal? >> It's a very natural fit between the two companies. When we started down the path of building new information management capabilities it became very clear that the market had strong need for both finding data, right? What do I actually have? I need an inventory, especially if my data's in Amazon S3, my data is in Azure Blob storage, my data is on-premise in HDFS, my data is in databases, it's all over the place. And I need to be able to find it. And then once I find it, I want to be able to prepare it. And so, one of the things that really drove this partnership was the very common interests that both companies have. And number one, pushing user experience. I love the Alation product. It's very easy to use, it's very intuitive, really it's a delightful thing to work with. And at the same time they also share our interests in working in these hybrid multicloud environments. So, what we've done and what we announced here at Strata is actually this bi-directional integration between the products. You can start in Alation and find a data set that you want to work with, see what collaboration or notes or business metadata people have created and then say, I want to go see this in Paxata. And in a single click you can then actually open it up in Paxata and profile that data. Vice versa you can also be in Paxata and prepare data, and then with a single click push it back, and then everybody who works with Alation actually now has knowledge of where that data is. So, it's a really nice synergy. >> So, you pushed the user data back to Alation, cause that's what they care a lot about, the cataloging and making the user-centric view work. So, you provide, it's almost a flow back and forth. It's a handshake if you will to data. Am I getting that right? >> Yeah, I mean, the idea's to keep the analyst or the user of that data, data scientist, even in some cases a business user, keep them in the flow of their work as much as possible. But give them the advantage of understanding what others in the organization have done with that data prior and allow them to transform it, and then share that knowledge back with the rest of the community that might be working with that data. >> John: So, give me an example. I like your Excel spreadsheet concept cause that's obvious. People know what Excel spreadsheet is so. So, it's Excel-like. That's an easy TAM to go after. All Microsoft users might not get that Azure thing. But this one, just take me through a usecase. >> So, I've got a good example. >> Okay, take me through. >> It's very common in a data lake for your data to be compressed. And when data's compressed, to a user it looks like a black box. So, if the data is compressed in Avro or Parquet or it's even like JSON format. A business user has no idea what's in that file. >> John: Yeah. >> So, what we do is we find the file for them. It may have some comments on that file of how that data's been used in past projects that we infer from looking at how others have used that data in Alation. >> John: So, you put metadata around it. >> We put a whole bunch of metadata around it. It might be comments that people have made. It might be >> Annotations, yeah. >> actual observations, annotations. And the great thing that we can do with Paxata is open that Avro file or Parquet file, open it up so that you can actually see the data elements themselves. So, all of a sudden, the business user has access without having to use a command line utility or understand anything about compression, and how you open that file up-- >> John: So, as Paxata spitting out there nuggets of value back to you, you're kind of understanding it, translating it to the user. And they get to do their thing, you get to do your thing, right? >> It's making a Avro or a Parquet file as easy to use as Excel, basically. Which is great, right? >> It's awesome. >> Now, you've enabled >> a whole new class of people who can use that. >> Well, and people just >> Get turned off when it's anything like jargon, or like, "What is that? I'm afraid it's phishing. Click on that and oh!" >> Well, the scary thing is that in a data lake environment, in a lot of cases people don't even label the files with extensions. They're just files. (Stephanie laughs) So, what started-- >> It's like getting your pictures like DS, JPEG. It's like what? >> Exactly. >> Right. >> So, you're talking about unlabeled-- >> If you looked on your laptop, and if you didn't have JPEG or DOC or PPT. Okay, I don't know that this file is. Well, what you have in the data lake environment is that you have thousands of these files that people don't really know what they are. And so, with Alation we have the ability to get all the value around the curation of the metadata, and how people are using that data. But then somebody says, "Okay, but I understand that this file exists. What's in it?" And then with Click to Profile from Alation you're immediately taken into Paxata. And now you're actually looking at what's in that file. So, you can very quickly go from this looks interesting to let me understand what's inside of it. And that's very powerful. >> Talk about Alation. Cause I had the CEO on, also their lead investor Greg Sands from Costanoa Ventures. They're a pretty amazing team but it's kind of out there. No offense, it's kind of a compliment actually. (Stephanie laughs) >> They got a symbolic >> Stephanie: Keep going. system Stanford guy, who's like super-smart. >> Nenshad: Yeah. >> They're on something that's really unique but it's almost too simple to be. Like, wait a minute! Google for the data, it's an awesome opportunity. How do you describe Alation to people who say, "Hey, what's this Alation thing?" >> Yeah, so I think that the best way to describe it is it's the browser for all of the distributed data in the enterprise. Sorry, so it's both the catalog, and the browser that sits on top of it. It sounds very simple. Conceptually it's very simple but they have a lot of richness in what they're able to do behind the scenes in terms of introspecting what type of work people are doing with data, and then taking that knowledge and actually surfacing it to the end user. So, for example, they have very powerful scenarios where they can watch what people are doing in different data sources, and then based on that information actually bubble up how queries are being used or the different patterns that people are doing to consume data with. So, what we find really exciting is that this is something that is very complex under the covers. Which Paxata is as well being built upon Spark. But they have put in the hard engineering work so that it looks simple to the end user. And that's the exact same thing that we've tried to do. >> And that's the hard problem. Okay, Stephanie back ... That was a great example by the way. Can't wait to have our little analyst breakdown of the event. But back to Alation for you. So, how do you talk about, you've been VP of Marketing of Alation. But you've been around the block. You know B2B, tech, big data. So, you've seen a bunch of different, you've worked at Trifacta, you worked at other companies, and you've seen a lot of waves of innovation come. What's different about Alation that people might not know about? How do you describe the difference? Because it sounds easy, "Oh, it's a browser! It's a catalog!" But it's really hard. Is it the tech that's the secret? Is it the approach? How do you describe the value of Alation? I think what's interesting about Alation is that we're solving a problem that since the dawn of the data warehouse has not been solved. And that is how to help end users really find and understand the data that they need to do their jobs. A lot of our customers talk about this-- >> John: Hold on. Repeat that. Cause that's like a key thing. What problem hasn't been solved since the data warehouse? >> To be able to actually find and fully understand, understand to the point of trust the data that you want to use for your analysis. And so, in the world of-- >> John: That sounds so simple. >> Stephanie: In the world of data warehousing-- >> John: Why is it so hard? >> Well, because in the world of data warehousing business people were told what data they should use. Someone in IT decided how to model the data, came up with a KPR calculation, and told you as a business person, you as a CEO, this is how you're going to monitor you business. >> John: Yeah. >> What business person >> Wants to be told that by an IT guy, right? >> Well, it was bounded by IT. >> Right. >> Expression and discovery >> Should be unbounded. Machine learning can take care of a lot of bounded stuff. I get that. But like, when you start to get into the discovery side of it, it should be free. >> Well, no offense to the IT team, but they were doing their best to try to figure out how to make this technology work. >> Well, just look at the cost of goods sold for storage. I mean, how many EMC drives? Expensive! IT was not cheap. >> Right. >> Not even 10, 15, 20 years ago. >> So, now when we have more self-service access to data, and we can have more exploratory analysis. What data science really introduced and Hadoop introduced was this ability on-demand to be able to create these structures, you have this more iterative world of how you can discover and explore datasets to come to an insight. The only challenge is, without simplifying that process, a business person is still lost, right? >> John: Yeah. >> Still lost in the data. >> So, we simply call that a catalog. But a catalog is much more-- >> Index, catalog, anthology, there's other words for it, right? >> Yeah, but I think it's interesting because like a concept of a catalog is an inventory has been around forever in this space. But the concept of a catalog that learns from other's behavior with that data, this concept of Behavior I/O that Aaron talked about earlier today. The fact that behavior of how people query data as an input and that input then informs a recommendation as an output is very powerful. And that's where all the machine learning and A.I. comes to work. It's hidden underneath that concept of Behavior I/O but that's there real innovation that drives this rich catalog is how can we make active recommendations to a business person who doesn't have to understand the technology but they know how to apply that data to making a decision. >> Yeah, that's key. Behavior and textual information has always been the two fly wheels in analysis whether you're talking search engine or data in general. And I think what I like about the trends here at Big Data NYC this weekend. We've certainly been seeing it at the hundreds of CUBE events we've gone to over the past 12 months and more is that people are using data differently. Not only say differently, there's baselining, foundational things you got to do. But the real innovators have a twist on it that give them an advantage. They see how they can use data. And the trend is collective intelligence of the customer seems to be big. You guys are doing it. You're seeing patterns. You're automating the data. So, it seems to be this fly wheel of some data, get some collective data. What's your thoughts and reactions. Are people getting it? Is this by people doing it by accident on purpose kind of thing? Did people just fell on their head? Or you see, "Oh, I just backed into this?" >> I think that the companies that have emerged as the leaders in the last 15 or 20 years, Google being a great example, Amazon being a great example. These are companies whose entire business models were based on data. They've generated out-sized returns. They are the leaders on the stock market. And I think that many companies have awoken to the fact that data as a monetizable asset to be turned into information either for analysis, to be turned into information for generating new products that can then be resold on the market. The leading edge companies have figured that out, and our adopting technologies like Alation, like Paxata, to get a competitive advantage in the business processes where they know they can make a difference inside of the enterprise. So, I don't think it's a fluke at all. I think that most of these companies are being forced to go down that path because they have been shown the way in terms of the digital giants that are currently ruling the enterprise tech world. >> All right, what's your thoughts on the week this week so far on the big trends? What are obvious, obviously A.I., don't need to talk about A.I., but what were the big things that came out of it? And what surprised you that didn't come out from a trends standpoint buzz here at Strata Data and Big Data NYC? What were the big themes that you saw emerge and didn't emerge what was the surprise? Any surprises? >> Basically, we're seeing in general the maturation of the market finally. People are finally realizing that, hey, it's not just about cool technology. It's not about what distribution or package. It's about can you actually drive return on investment? Can you actually drive insights and results from the stack? And so, even the technologists that we were talking with today throughout the course of the show are starting to talk about it's that last mile of making the humans more intelligent about navigating this data, where all the breakthroughs are going to happen. Even in places like IOT, where you think about a lot of automation, and you think about a lot of capability to use deep learning to maybe make some decisions. There's still a lot of human training that goes into that decision-making process and having agency at the edge. And so I think this acknowledgement that there should be balance between human input and what the technology can do is a nice breakthrough that's going to help us get to the next level. >> What's missing? What do you see that people missed that is super-important, that wasn't talked much about? Is there anything that jumps out at you? I'll let you think about it. Nenshad, you have something now. >> Yeah, I would say I completely agree with what Stephanie said which we are seeing the market mature. >> John: Yeah. >> And there is a compelling force to now justify business value for all the investments people have made. The science experiment phase of the big data world is over. People now have to show a return on that investment. I think that being said though, this is my sort of way of being a little more provocative. I still think there's way too much emphasis on data science and not enough emphasis on the average business analyst who's doing work in the Fortune 500. >> It should be kind of the same thing. I mean, with data science you're just more of an advanced analyst maybe. >> Right. But the idea that every person who works with data is suddenly going to understand different types of machine learning models, and what's the right way to do hyper parameter tuning, and other words that I could throw at you to show that I'm smart. (laughter) >> You guys have a vision with the Excel thing. I could see how you see that perspective because you see a future. I just think we're not there yet because I think the data scientists are still handcuffed and hamstrung by the fact that they're doing too much provisioning work, right? >> Yeah. >> To you're point about >> surfacing the insights, it's like the data scientists, "Oh, you own it now!" They become the sysadmin, if you will, for their department. And it's like it's not their job. >> Well, we need to get them out of data preparation, right? >> Yeah, get out of that. >> You shouldn't be a data scientist-- >> Right now, you have two values. You've got the use interface value, which I love, but you guys do the automation. So, I think we're getting there. I see where you're coming from, but still those data sciences have to set the tone for the generation, right? So, it's kind of like you got to get those guys productive. >> And it's not a .. Please go ahead. >> I mean, it's somewhat interesting if you look at can the data scientist start to collaborate a little bit more with the common business person? You start to think about it as a little bit of scientific inquiry process. >> John: Yeah. >> Right? >> If you can have more innovators around the table in a common place to discuss what are the insights in this data, and people are bringing business perspective together with machine learning perspective, or the knowledge of the higher algorithms, then maybe you can bring those next leaps forward. >> Great insight. If you want my observations, I use the crazy analogy. Here's my crazy analogy. Years it's been about the engine Model T, the car, the horse and buggy, you know? Now, "We got an engine in the car!" And they got wheels, it's got a chassis. And so, it's about the apparatus of the car. And then it evolved to, "Hey, this thing actually drives. It's transportation." You can actually go from A to B faster than the other guys, and people still think there's a horse and buggy market out there. So, they got to go to that. But now people are crashing. Now, there's an art to driving the car. >> Right. >> So, whether you're a sports car or whatever, this is where the value piece I think hits home is that, people are driving the data now. They're driving the value proposition. So, I think that, to me, the big surprise here is how people aren't getting into the hype cycle. They like the hype in terms of lead gen, and A.I., but they're too busy for the hype. It's like, drive the value. This is not just B.S. either, outcomes. It's like, "I'm busy. I got security. I got app development." >> And I think they're getting smarter about how their valuing data. We're starting to see some economic models, and some ways of putting actual numbers on what impact is this data having today. We do a lot of usage analysis with our customers, and looking at they have a goal to distribute data across more of the organization, and really get people using it in a self-service manner. And from that, you're being able to calculate what actually is the impact. We're not just storing this for insurance policy reasons. >> Yeah, yeah. >> And this cheap-- >> John: It's not some POC. Don't do a POC. All right, so we're going to end the day and the segment on you guys having the last word. I want to phrase it this way. Share an anecdotal story you've heard from a customer, or a prospective customer, that looked at your product, not the joint product but your products each, that blew you away, and that would be a good thing to leave people with. What was the coolest or nicest thing you've heard someone say about Alation and Paxata? >> For me, the coolest thing they said, "This was a social network for nerds. I finally feel like I've found my home." (laughter) >> Data nerds, okay. >> Data nerds. So, if you're a data nerd, you want to network, Alation is the place you want to be. >> So, there is like profiles? And like, you guys have a profile for everybody who comes in? >> Yeah, so the interesting thing is part of our automation, when we go and we index the data sources we also index the people that are accessing those sources. So, you kind of have a leaderboard now of data users, that contract one another in system. >> John: Ooh. >> And at eBay leader was this guy, Caleb, who was their data scientist. And Caleb was famous because everyone in the organization would ask Caleb to prepare data for them. And Caleb was like well known if you were around eBay for awhile. >> John: Yeah, he was the master of the domain. >> And then when we turned on, you know, we were indexing tables on teradata as well as their Hadoop implementation. And all of a sudden, there are table structures that are Caleb underscore cussed. Caleb underscore revenue. Caleb underscore ... We're like, "Wow!" Caleb drove a lot of teradata revenue. (Laughs) >> Awesome. >> Paxata, what was the coolest thing someone said about you in terms of being the nicest or coolest most relevant thing? >> So, something that a prospect said earlier this week is that, "I've been hearing in our personal lives about self-driving cars. But seeing your product and where you're going with it I see the path towards self-driving data." And that's really what we need to aspire towards. It's not about spending hours doing prep. It's not about spending hours doing manual inventories. It's about getting to the point that you can automate the usage to get to the outcomes that people are looking for. So, I'm looking forward to self-driving information. Nenshad, thanks so much. Stephanie from Alation. Thanks so much. Congratulations both on your success. And great to see you guys partnering. Big, big community here. And just the beginning. We see the big waves coming, so thanks for sharing perspective. >> Thank you very much. >> And your color commentary on our wrap up segment here for Big Data NYC. This is theCUBE live from New York, wrapping up great three days of coverage here in Manhattan. I'm John Furrier. Thanks for watching. See you next time. (upbeat techo music)

>> Voiceover: Live from midtown Manhattan, it's theCUBE! Covering Big Data New York City 2017. Brought to you by, SiliconANGLE Media and as ecosystems sponsors. >> Okay, welcome back everyone, live in New York City this is theCUBE's coverage of, eighth year doing Hadoop World now, evolved into Strata Hadoop, now called Strata Data, it's had many incarnations but O'Reilly Media running their event in conjunction with Cloudera, mainly an O'Reilly media show. We do our own show called Big Data NYC here with our community with theCUBE bringing you the best interviews, the best people, entrepreneurs, thought leaders, experts, to get the data and try to project the future and help users find the value in data. My next guest is Rob Thomas, who is the General Manager of IBM Analytics, theCUBE Alumni, been on multiple times successfully executing in the San Francisco Bay area. Great to see you again. >> Yeah John, great to see you, thanks for having me. >> You know IBM is really been interesting through its own transformation and a lot of people will throw IBM in that category but you guys have been transforming okay and the scoreboard yet has to yet to show in my mind what's truly happening because if you still look at this industry, we're only eight years into what Hadoop evolved into now as a large data set but the analytics game just seems to be getting started with the cloud now coming over the top, you're starting to see a lot of cloud conversations in the air. Certainly there's a lot of AI washing, you know, AI this, but it's machine learning and deep learning at the heart of it as innovation but a lot more work on the analytics side is coming. You guys are at the center of that. What's the update? What's your view of this analytics market? >> Most enterprises struggle with complexity. That's the number one problem when it comes to analytics. It's not imagination, it's not willpower, in many cases, it's not even investment, it's just complexity. We are trying to make data really simple to use and the way I would describe it is we're moving from a world of products to platforms. Today, if you want to go solve a data governance problem you're typically integrating 10, 15 different products. And the burden then is on the client. So, we're trying to make analytics a platform game. And my view is an enterprise has to have three platforms if they're serious about analytics. They need a data manager platform for managing all types of data, public, private cloud. They need unified governance so governance of all types of data and they need a data science platform machine learning. If a client has those three platforms, they will be successful with data. And what I see now is really mixed. We've got 10 products that do that, five products that do this, but it has to be integrated in a platform. >> You as an IBM or the customer has these tools? >> Yeah, when I go see clients that's what I see is data... >> John: Disparate data log. >> Yeah, they have disparate tools and so we are unifying what we deliver from a product perspective to this platform concept. >> You guys announce an integrated analytic system, got to see my notes here, I want to get into that in a second but interesting you bring up the word platform because you know, platforms have always been kind of reserved for the big supplier but you're talking about customers having a platform, not a supplier delivering a platform per se 'cause this is where the integration thing becomes interesting. We were joking yesterday on theCUBE here, kind of just kind of ad hoc conceptually like the world has turned into a tool shed. I mean everyone has a tool shed or knows someone that has a tool shed where you have the tools in the back and they're rusty. And so, this brings up the tool conversation, there's too many tools out there that try to be platforms. >> Rob: Yes. >> And if you have too many tools, you're not really doing the platform game right. And complexity also turns into when you bought a hammer it turned into a lawn mower. Right so, a lot of these companies have been groping and trying to iterate what their tool was into something else it wasn't built for. So, as the industry evolves, that's natural Darwinism if you will, they will fall to the wayside. So talk about that dynamic because you still need tooling >> Rob: Yes. but tool will be a function of the work as Peter Burris would say, so talk about how does a customer really get that platform out there without sacrificing the tooling that they may have bought or want to get rid of. >> Well, so think about the, in enterprise today, what the data architecture looks like is, I've got this box that has this software on it, use your terms, has these types of tools on it, and it's isolated and if you want a different set of tooling, okay, move that data to this other box where we have the other tooling. So, it's very isolated in terms of how platforms have evolved or technology platforms today. When I talk about an integrated platform, we are big contributors to Kubernetes. We're making that foundational in terms of what we're doing on Private Cloud and Public Cloud is if you move to that model, suddenly what was a bunch of disparate tools are now microservices against a common architecture. And so it totally changes the nature of the data platform in an enterprise. It's a much more fluid data layer. The term I use sometimes is you have data as a service now, available to all your employees. That's totally different than I want to do this project, so step one, make room in the data center, step two, bring in a server. It's a much more flexible approach so that's what I mean when I say platform. >> So operationalizing it is a lot easier than just going down the linear path of provisioning. All right, so let's bring up the complexity issue because integrated and unified are two different concepts that kind of mean the same thing depending on how you look at it. When you look at the data integration problem, you've got all this complexity around governance, it's a lot of moving parts of data. How does a customer actually execute without compromising the integrity of their policies that they need to have in place? So in other words, what are the baby steps that someone can take, the customers take through with what you guys are dealing with them, how do they get into the game, how do they take steps towards the outcome? They might not have the big money to push it all at once, they might want to take a risk of risk management approach. >> I think there's a clear recipe for doing this right and we have experience of doing it well and doing it not so well, so over time we've gotten some, I'd say a pretty good perspective on that. My view is very simple, data governance has to start with a catalog. And the analogy I use is, you have to do for data what libraries do for books. And think about a library, the first thing you do with books, card catalog. You know where, you basically itemize everything, you know exactly where it sits. If you've got multiple copies of the same book, you can distinguish between which one is which. As books get older they go to archives, to microfilm or something like that. That's what you have to do with your data. >> On the front end. >> On the front end. And it starts with a catalog. And that reason I say that is, I see some organizations that start with, hey, let's go start ETL, I'll create a new warehouse, create a new Hadoop environment. That might be the right thing to do but without having a basis of what you have, which is the catalog, that's where I think clients need to start. >> Well, I would just add one more level of complexity just to kind of reinforce, first of all I agree with you but here's another example that would reinforce this step. Let's just say you write some machine learning and some algorithms and a new policy from the government comes down. Hey, you know, we're dealing with Bitcoin differently or whatever, some GPRS kind of thing happens where someone gets hacked and a new law comes out. How do you inject that policy? You got to rewrite the code, so I'm thinking that if you do this right, you don't have to do a lot of rewriting of applications to the library or the catalog will handle it. Is that right, am I getting that right? >> That's right 'cause then you have a baseline is what I would describe it as. It's codified in the form of a data model or in the form on ontology for how you're looking at unstructured data. You have a baseline so then as changes come, you can easily adjust to those changes. Where I see clients struggle is if you don't have that baseline then you're constantly trying to change things on the fly and that makes it really hard to get to this... >> Well, really hard, expensive, they have to rewrite apps. >> Exactly. >> Rewrite algorithms and machine learning things that were built probably by people that maybe left the company, who knows, right? So the consequences are pretty grave, I mean, pretty big. >> Yes. >> Okay, so let's back to something that you said yesterday. You were on theCUBE yesterday with Hortonworks CEO, Rob Bearden and you were commenting about AI or AI washing. You said quote, "You can't have AI without IA." A play on letters there, sequence of letters which was really an interesting comment, we kind of referenced it pretty much all day yesterday. Information architecture is the IA and AI is the artificial intelligence basically saying if you don't have some sort of architecture AI really can't work. Which really means models have to be understood, with the learning machine kind of approach. Expand more on that 'cause that was I think a fundamental thing that we're seeing at the show this week, this in New York is a model for the models. Who trains the machine learning? Machines got to learn somewhere too so there's learning for the learning machines. This is a real complex data problem and a half. If you don't set up the architecture it may not work, explain. >> So, there's two big problems enterprises have today. One is trying to operationalize data science and machine learning that scale, the other one is getting the cloud but let's focus on the first one for a minute. The reason clients struggle to operationalize this at scale is because they start a data science project and they build a model for one discreet data set. Problem is that only applies to that data set, it doesn't, you can't pick it up and move it somewhere else so this idea of data architecture just to kind of follow through, whether it's the catalog or how you're managing your data across multiple clouds becomes fundamental because ultimately you want to be able to provide machine learning across all your data because machine learning is about predictions and it's hard to do really good predictions on a subset. But that pre-req is the need for an information architecture that comprehends for the fact that you're going to build models and you want to train those models. As new data comes in, you want to keep the training process going. And that's the biggest challenge I see clients struggling with. So they'll have success with their first ML project but then the next one becomes progressively harder because now they're trying to use more data and they haven't prepared their architecture for that. >> Great point. Now, switching to data science. You spoke many times with us on theCUBE about data science, we know you're passionate about you guys doing a lot of work on that. We've observed and Jim Kobielus and I were talking yesterday, there's too much work still in the data science guys plate. There's still doing a lot of what I call, sys admin like work, not the right word, but like administrative building and wrangling. They're not doing enough data science and there's enough proof points now to show that data science actually impacts business in whether it's military having data intelligence to execute something, to selling something at the right time, or even for work or play or consume, or we use, all proof is out there. So why aren't we going faster, why aren't the data scientists more effective, what does it going to take for the data science to have a seamless environment that works for them? They're still doing a lot of wrangling and they're still getting down the weeds. Is that just the role they have or how does it get easier for them that's the big catch? >> That's not the role. So they're a victim of their architecture to some extent and that's why they end up spending 80% of their time on data prep, data cleansing, that type of thing. Look, I think we solved that. That's why when we introduced the integrated analytic system this week, that whole idea was get rid of all the data prep that you need because land the data in one place, machine learning and data science is built into that. So everything that the data scientist struggles with today goes away. We can federate to data on cloud, on any cloud, we can federate to data that's sitting inside Hortonworks so it looks like one system but machine learning is built into it from the start. So we've eliminated the need for all of that data movement, for all that data wrangling 'cause we organized the data, we built the catalog, and we've made it really simple. And so if you go back to the point I made, so one issue is clients can't apply machine learning at scale, the other one is they're struggling to get the cloud. I think we've nailed those problems 'cause now with a click of a button, you can scale this to part of the cloud. >> All right, so how does the customer get their hands on this? Sounds like it's a great tool, you're saying it's leading edge. We'll take a look at it, certainly I'll do a review on it with the team but how do I get it, how do I get a hold of this? What do I do, download it, you guys supply it to me, is it some open source, how do your customers and potential customers engage with this product? >> However they want to but I'll give you some examples. So, we have an analytic system built on Spark, you can bring the whole box into your data center and right away you're ready for data science. That's one way. Somebody like you, you're going to want to go get the containerized version, you go download it on the web and you'll be up and running instantly with a highly performing warehouse integrated with machine learning and data science built on Spark using Apache Jupyter. Any developer can go use that and get value out of it. You can also say I want to run it on my desktop. >> And that's free? >> Yes. >> Okay. >> There's a trial version out there. >> That's the open source, yeah, that's the free version. >> There's also a version on public cloud so if you don't want to download it, you want to run it outside your firewall, you can go run it on IBM cloud on the public cloud so... >> Just your cloud, Amazon? >> No, not today. >> John: Just IBM cloud, okay, I got it. >> So there's variety of ways that you can go use this and I think what you'll find... >> But you have a premium model that people can get started out so they'll download it to your data center, is that also free too? >> Yeah, absolutely. >> Okay, so all the base stuff is free. >> We also have a desktop version too so you can download... >> What URL can people look at this? >> Go to datascience.ibm.com, that's the best place to start a data science journey. >> Okay, multi-cloud, Common Cloud is what people are calling it, you guys have Common SQL engine. What is this product, how does it relate to the whole multi-cloud trend? Customers are looking for multiple clouds. >> Yeah, so Common SQL is the idea of integrating data wherever it is, whatever form it's in, ANSI SQL compliant so what you would expect for a SQL query and the type of response you get back, you get that back with Common SQL no matter where the data is. Now when you start thinking multi-cloud you introduce a whole other bunch of factors. Network, latency, all those types of things so what we talked about yesterday with the announcement of Hortonworks Dataplane which is kind of extending the YARN environment across multi-clouds, that's something we can plug in to. So, I think let's be honest, the multi-cloud world is still pretty early. >> John: Oh, really early. >> Our focus is delivery... >> I don't think it really exists actually. >> I think... >> It's multiple clouds but no one's actually moving workloads across all the clouds, I haven't found any. >> Yeah, I think it's hard for latency reasons today. We're trying to deliver an outstanding... >> But people are saying, I mean this is head room I got but people are saying, I'd love to have a preferred future of multi-cloud even though they're kind of getting their own shops in order, retrenching, and re-platforming it but that's not a bad ask. I mean, I'm a user, I want to move from if I don't like IBM's cloud or I got a better service, I can move around here. If Amazon is too expensive I want to move to IBM, you got product differentiation, I might want to to be in your cloud. So again, this is the customers mindset, right. If you have something really compelling on your cloud, do I have to go all in on IBM cloud to run my data? You shouldn't have to, right? >> I agree, yeah I don't think any enterprise will go all in on one cloud. I think it's delusional for people to think that so you're going to have this world. So the reason when we built IBM Cloud Private we did it on Kubernetes was we said, that can be a substrate if you will, that provides a level of standards across multiple cloud type environments. >> John: And it's got some traction too so it's a good bet there. >> Absolutely. >> Rob, final word, just talk about the personas who you now engage with from IBM's standpoint. I know you have a lot of great developers stuff going on, you've done some great work, you've got a free product out there but you still got to make money, you got to provide value to IBM, who are you selling to, what's the main thing, you've got multiple stakeholders, could you just clarify the stakeholders that you're serving in the marketplace? >> Yeah, I mean, the emerging stakeholder that we speak with more and more than we used to is chief marketing officers who have real budgets for data and data science and trying to change how they're performing their job. That's a major stakeholder, CTOs, CIOs, any C level, >> Chief data officer. >> Chief data officer. You know chief data officers, honestly, it's a mixed bag. Some organizations they're incredibly empowered and they're driving the strategy. Others, they're figure heads and so you got to know how the organizations do it. >> A puppet for the CFO or something. >> Yeah, exactly. >> Our ops. >> A puppet? (chuckles) So, you got to you know. >> Well, they're not really driving it, they're not changing it. It's not like we're mandated to go do something they're maybe governance police or something. >> Yeah, and in some cases that's true. In other cases, they drive the data architecture, the data strategy, and that's somebody that we can engage with right away and help them out so... >> Any events you got going up? Things happening in the marketplace that people might want to participate in? I know you guys do a lot of stuff out in the open, events they can connect with IBM, things going on? >> So we do, so we're doing a big event here in New York on November first and second where we're rolling out a lot of our new data products and cloud products so that's one coming up pretty soon. The biggest thing we've changed this year is there's such a craving for clients for education as we've started doing what we're calling Analytics University where we actually go to clients and we'll spend a day or two days, go really deep and open languages, open source. That's become kind of a new focus for us. >> A lot of re-skilling going on too with the transformation, right? >> Rob: Yes, absolutely. >> All right, Rob Thomas here, General Manager IBM Analytics inside theCUBE. CUBE alumni, breaking it down, giving his perspective. He's got two books out there, The Data Revolution was the first one. >> Big Data Revolution. >> Big Data Revolution and the new one is Every Company is a Tech Company. Love that title which is true, check it out on Amazon. Rob Thomas, Bid Data Revolution, first book and then second book is Every Company is a Tech Company. It's theCUBE live from New York. More coverage after the short break. (theCUBE jingle) (theCUBE jingle) (calm soothing music)

(electronic music) >> Host: Live from Midtown Manhattan it's The Cube! Covering Big Data New York City 2017, brought to you by Silicon Angle Media, and its Ecosystem sponsors. >> Okay, welcome back everyone. We are here live, The Cube in New York City for Big Data NYC, this is our fifth year, doing our own event, not with O'Reilly or Cloud Era at Strata Data, which as Hadoop World, Strata Conference, Strata Hadoop, now called Strata Data, probably called Strata AI next year, we're The Cube every year, bringing you all the great data, and what's going on. Entrepreneurs, VCs, thought leaders, we interview them and bring that to you. I'm John Furrier with our next guest, Greg Sands, who's the managing director and founder of Costa Nova ventures in Palo Alto, started out as an entrepreneur himself, then single shingle out there, now he's a big VC firm on a third fund. >> On the third fund. >> Third fund. How much in that fund? >> 175 million dollar fund. >> So now you're a big firm now, congratulations, and really great to see your success. >> Thanks very much. I mean, we're still very much an early stage boutique focused on companies that change the way the world does business, but it is the case that we have a bigger team and a bigger fund, to go do the same thing. >> Well you've been great to work with, I've been following you, we've known each other for a while, watched you left Sir Hill and start Costanova, but what's interesting is that, I can kind of joke and kid you, the VC inside joke about being a big firm, because I know you want to be small, and like to be small, help entrepreneurs, that's your thing. But it's really not a big firm, it's a few partners, but a lot of people helping companies, that's your ethos, that's what you're all about at your firm. Take a minute to just share with the folks the kinds of things you do and how you get involved in companies, you're hands on, you roll up your sleeves. You get out of the way at the right time, you help when you can, share your ethos. >> Yeah, absolutely so the way we think of it is, combining the craft of old school venture capital, with a modern operating team, and so since most founder these days are product-oriented, our job is to think like product people, not think like investors. So we think like product people, we do product level analysis, we do customer discovery, we do, we go ride along on sales calls when we're making investment decisions. And then we do the things that great venture capitalists have done for years, and so for example, at Alatian, who I know has been on the show today, we were able to incubate them in our office for a year, I had many conversations with Sathien after he'd sold the first two or three customers. Okay, who's the next person we hire? Who isn't a founder? Who's going to go out and sell? What does that person look like? Do you go straight to a VP? Or do you hire an individual contributor? Do you hire someone for domain, or do you hire someone for talent? And that's the thing that we love doing. Now we've actually built out an operating team so marketing partner, Martino Alcenco, and Jim Wilson as a sales partner, to really help turn that into a program, so that they can, we can take these founders who find product market fit, and say, how do we help you build the right sales process and marketing process, sales team and marketing team, for your company, your customer, your product? >> Well it's interesting since you mention old school venture capital, I'll get into some of the dynamics that are going on in Silicon valley, but it's important to bring that forward, because now with cloud you can get to critical mass on the fly wheel, on economics, you can see the visibility faster now. >> Greg: Absolutely. >> So the game of the old school venture capitalist is all the same, how do you get to cruising altitude, whatever metaphor you want to use, the key was getting there, and sometimes it took a couple of rounds, but now you can get these companies with five million, maybe $10 million funding, they can have unit economics visibility, scales insight, then the scale game comes in, so that seems to be the secret trick right now in venture is, don't overspend, keep the valuation in range and allows you to look for multiple exits potentially, or growth. Talk about that dynamic, because this is like, I call it the hour glass. You get through the hour glass, everyone's down here, but if you can sneak through and get the visibility on the economics, then you grow quickly. >> Absolutely. I mean, it's exactly right an I haven't heard the hour glass metaphor before but I like it. You want to basically get through the narrows of product market fit and the beginnings of scalable sales and marketing. You don't need to know all the answers, but you can do that in a capital-efficient way, building really solid foundations for future explosive growth, look, everybody loves fast growth and big markets, and being grown into. But the number of people who basically don't build those foundations and then say, go big or go home! And they take a ton of money, and they go spend all the money, doing things that just fundamentally don't work, and they blow themselves up. >> Well this is the hourglass problem. You have, once you get through that unique economics, then you have true scale, and value will increase. Everybody wins there so it's about getting through that, and you can get through it fast with good mentoring, but here's the challenge that entrepreneurs fall into the trap. I call it the, I think I made it trap. And what happens is they think they're on the other side of the hourglass, but they still haven't even gone through the straight and narrow yet, and they don't know it. And what they do is they over fund and implode. That seems to be a major trap I see a lot of entrepreneurs fall into, while I got a 50 million pre on my B round, or some monster valuation, and they get way too much cash, and they're behaving as if they're scaling, and they haven't even nailed it yet. >> Well, I think that's right. So there's certainly, there are stages of product market fit, and so I think people hit that first stage, and they say, oh I've got it. And they try to explode out of the gates. And we, in fact I know one good example of somebody saying, hey, by the way, we're doing great in field sales, and our investors want us to go really fast, so we are going to go inside and we, my job was to hire 50 inside people, without ever having tried it. And so we always preach crawl, walk, run, right? Hire a couple, see how it works. Right, in a new channel. Or a new category, or an adjacent space, and I think that it's helpful to have an investor who has seen the whole picture to say, yeah, I know it looks like light at the end of the tunnel, but see how it's a relatively small dot? You still got to go a little farther, and then the other thing I say is, look, don't build your company to feed your venture capitalist ego. Right? People do these big rounds of big valuations, and the big dog investors say, go, go, go! But, you're the CEO. Your job is analyze the data. >> John: You can find during the day (laughs). >> And say, you know, given what we know, how fast should we go? Which investments should we make? And you've got to own that. And I think sometimes our job is just to be the pulling guard and clear space for the CEO to make good decisions. >> So you know I'm a big fan, so my bias is pretty much out there, love what you guys are doing. Tim Carr is a Pivot North doing the same thing. Really adding value, getting down and dirty, but the question that entrepreneurs always ask me and talk privately, not about you, but in general, I don't want the VC to get in the way. I want them, I don't want them to preach to me, I don't want too many know-it-alls on my board, I want added value, but again, I don't want the preaching, I don't want them to get in the way, 'cause that's the fear. I'm not saying the same about VCs in general, but that's kind of the mentality of an entrepreneur. I want someone who's going to help me, be in the boat with me, but not be in my way. How do you address that concern to the founders who think, not think like that, but might have a fear. >> Well, by the way, I think it's a legitimate fear, and I think it actually is uncorrelated with added value, right? I think the idea that the board has certain responsibilities, and management has certain responsibilities, is incredibly important. And I think, I can speak for myself in saying, I'm quite conscious of not crossing that line, I think you talk. >> John: You got to build a return, that's the thing. >> But ultimately I would say to an entrepreneur, I'd just say, hey look, call references. And by the way, here are 30 names and phone numbers, and call any one of them, because I think that people who are, so a venture capital know-it-all, in the board room, telling CEOs what to do, destroys value. It's sand in the gears, and it's bad for the company. >> Absolutely, I agree 100% >> And some of my, when I talk about being a pulling guard for the CEO, that's what I'm talking about, which is blocking people who are destructive. >> And rolling the block for a touchdown, kind of use the metaphor. Adding value, that's the key, and that's why I wanted to get that out there because most guys don't get that nuance, and entrepreneurs, especially the younger ones. So it's good and important. Okay, let's talk about culture, obviously in Silicon Valley, I get, reading this morning in the Wymo guy, and they're writing it, that's the Silicon Valley, that's not crazy, there's a lot of great people in Silicon Valley, you're one of them. The culture's certainly an innovative culture, there's been some things in the press, inclusion and diversity, obviously is super important. This whole brogrammer thing that's been kind of kicked around. How are you dealing with all that? Because, you know, this is a cultural shift, but I think it's being made out more than it really is, but there's still our core issues, your thoughts on the whole inclusion and diversity, and this whole brogrammer blowback thing. >> Yeah, well so I think, so first of all, really important issues, glad we're talking about them, and we all need to get better. And to me the question for us has been, what role do we play? And because I would say it is a relatively small subset of the tech industry, and the venture capital industry. At the same time the behavior of that has become public is appalling. It's appalling and totally unacceptable, and so the question is, okay, how can we be a part of the stand-up part of the ecosystem, and some of which is calling things out when we see them. Though frankly we work with and hang out with people and we don't see them that often, and then part of which is, how do we find a couple of ways to contribute meaningfully? So for example this summer we ran what we called the Costanova Access Fellowship, intentionally, trying to provide first opportunity and venture capital for people who traditionally haven't had as much access. We created an event in the spring called, Seat at the Table, really, particularly around women in the tech industry, and it went so well that we're running it in New York on October 19th, so if you're a woman in tech in New York, we'd love to see you then. And we're just trying to figure-- >> You're doing it in an authentic way though, you're not really doing it from a promotional standpoint. It's legit. >> Yeah, we're just trying to do, you know, pick off a couple of things that we can do, so that we can be on the side of the good guys. >> So I guess what you're saying is just have high integrity, and be part of the solution not part of the problem. >> That's right, and by the way, both of these initiatives were ones that were kicked off in late 2016, so it's not a reaction to things like binary capital, and the problems at uper, both of which are appalling. >> Self-awareness is critical. Let's get back to the nuts and bolts of the real reason why I wanted you to come on, one was to find out how much money you have to spend for the entrepreneurs that are watching. Give us the update on the last fund, so you got a new fund that you just closed, the new fund, fund three. You have your other funds that are still out there, and some funds reserved, which, what's the number amount, how much are you writing checks for? Give the whole thesis. >> Absoluteley. So we're an early stage investor, so we lead series A and seed financing companies that change the way the world does business, so up and down the stack, a business-facing software, data-driven applications. Machine-learning and AI driven applications. >> John: But the filter is changing the way the world works? >> The way, yes, but in particularly the way the world does business. You can think of it as a business-facing software stack. We're not social media investors, it's not what we know, it's not what we're good at. And it includes security and management, and the data stack and-- >> Joe: Enterprise and emerging tech. >> That's right. And the-- >> And every crazy idea in between. >> That's right. (laughs) Absolutely, and so we're participate in or leave seed financings as most typically are half a million to maybe one and a quarter, and we'll lead series A financing, small ones might be two or two and a half million dollars at the outer edge is probably a six million dollar check. We were just opening up in the next couple of days, a thousand square feet of incubation space at world headquarters at Palo Alto. >> John: Nice. >> So Alation, Acme Ticketing and Zen IQ are companies that we invested in. >> Joe: What location is this going to be at? >> That's, near the Fills in downtown Palo Alto, 164 staff, and those three companies are ones where we effectively invested at formation and incubated it for a year, we love doing that. >> At the hangout at Philsmore and get the data. And so you got some funds, what else do you have going on? 175 million? >> So one was a $100 million fund, and then fund two was $135 million fund, and the last investment of fund two which we announced about three weeks ago was called Roadster, so it's ecommerce enablement for the modern dealerships. So Omnichannel and Mobile First infrastructure for auto-dealers. We have already closed, and had the first board meeting for the first new investment of fund three, which isn't yet announced, but in the land of computer vision and deep learning, so a couple of the subjects that we care deeply about, and spend a lot of time thinking about. >> And the average check size for the A round again, seed and A, what do you know about the? The lowest and highest? >> The average for the seed is half a million to one and a quarter, and probably average for a series A is four or five. >> And you'll lead As. >> And we will lead As. >> Okay great. What's the coolest thing you're working on right now that gets you excited? It doesn't have to be a portfolio company, but the research you're doing, thing, tires you're kicking, in subjects, or domains? >> You know, so honestly, one of the great benefits of the venture capital business is that I get up and my neurons are firing right away every day. And I do think that for example, one of the things that we love is is all of the adulant infrastructure and so we've got our friends at Victor Ops that are in the middle of that space, and the thinking about how the modern programmer works, how everybody-- >> Joe: Is security on your radar? >> Security is very much on our radar, in fact, someone who you should have on your show is Asheesh Guptar, and Casey Ella, so she's just joined Bug Crowd as the CEO and Casey moves over to CTO, and the word Bug Bounty was just entered into the Oxford Dictionary for the first time last week, so that to me is the ultimate in category creation. So security and dev ops tools are among the things that we really like. >> And bounties will become the norm as more and more decentralized apps hit the scene. Are you doing anything on decentralized applications? I'm not saying Blockchain in particular, but Blockchain like apps, distributing computing you're well versed on. >> That's right, well we-- >> Blockchain will have an impact in your area. >> Blockchain will have an impact, we just spent an hour talking about it in the context our off site in Decosona Lodge in Pascadero, it felt like it was important that we go there. And digging into it. I think actually the edge computing is actually more actionable for us right now, given the things that we're, given the things that we're interested in, and we're doing and they, it is just fascinating how compute centralizes and then decentralizes, centralizes and then decentralizes again, and I do think that there are a set of things that are fascinating about what your process at the edge, and what you send back to the core. >> As Pet Gelson here said in the QU, if you're not out in front of that next wave, you're driftwood, a lot of big waves coming in, you've seen a lot of waves, you were part of one that changed the world, Netscape browser, or the business plan for that first project manager, congratulations. Now you're at a whole nother generation. You ready? (laughs) >> Absolutely, I'm totally ready, I'm ready to go. >> Greg Sands here in The Cube in New York City, part of Big Data NYC, more live coverage with The Cube after this short break, thanks for watching. (electronic jingle) (inspiring electronic music)

>> Announcer: Live from Midtown Manhattan. It's the CUBE. Covering Big Data New York City 2017. Brought to you by Silicon Angle Media and its ecosystem sponsor. (electronic music) >> Hey, welcome back everyone, live here in New York. This is the CUBE here in Manhattan for Big Data NYC's three days of coverage. We're one day three, things are starting to settle in, starting to see the patterns out there. I'll say it's Big Data week here, in conjunction with Hadoop World, formerly known as Strata Conference, Strata-Hadoop, Strata-Data, soon to be Strata-AI, soon to be Strata-IOT. Big Data, Mike Maccaux who's the Global Big Data Practice Lead at Dell EMC. We've been in this world now for multiple years and, well, what a riot it's been. >> Yeah, it has. It's been really interesting as the organizations have gone from their legacy systems, they have been modernizing. And we've sort of seen Big Data 1.0 a couple years ago. Big Data 2.0 and now we're moving on sort of the what's next? >> Yeah. >> And it's interesting because the Big Data space has really lagged the application space. You talk about microservices-based applications, and deploying in the cloud and stateless things. The data technologies and the data space has not quite caught up. The technology's there, but the thinking around it, and the deployment of those, it seems to be a slower, more methodical process. And so what we're seeing in a lot of enterprises is that the ones that got in early, have built out capabilities, are now looking for that, how do we get to the next level? How do we provide self-service? How do we enable our data scientists to be more productive within the enterprise, right? If you're a startup, it's easy, right? You're somewhere in the public cloud, you're using cloud based API, it's all fine. But if you're an enterprise, with the inertia of those legacy systems and governance and controls, it's a different problem to solve for. >> Let's just face it. We'll just call a spade a spade. Total cost of ownership was out of control. Hadoop was great, but it was built for something that tried to be something else as it evolved. And that's good also, because we need to decentralize and democratize the incumbent big data warehouse stuff. But let's face it, Hadoop is not the game anymore, it's everything else. >> Right, yep. >> Around it so, we've seen that, that's a couple years old. It's about business value right now. That seems to be the big thing. The separation between the players that can deliver value for the customers. >> Matt: Yep. >> And show a little bit of headroom for future AI things, they've seen that. And have the cloud on premise play. >> Yep. >> Right now, to me, that's the call here. What do you, do you agree? >> I absolutely see it. It's funny, you talk to organizations and they say, "We're going cloud, we're doing cloud." Well what does that mean? Can you even put your data in the cloud? Are you allowed to? How are you going to manage that? How are you going to govern that? How are you going to secure that? So many organizations, once they've asked those questions, they've realized, maybe we should start with the model of cloud on premise. And figure out what works and what doesn't. How do users actually want to self serve? What do we templatize for them? And what do we give them the freedom to do themselves? >> Yeah. >> And they sort of get their sea legs with that, and then we look at sort of a hybrid cloud model. How do we be able to span on premise, off premise, whatever your public cloud is, in a seamless way? Because we don't want to end up with the same thing that we had with mainframes decades ago, where it was, IBM had the best, it was the fastest, it was the most efficient, it was the new paradigm. And then 10 years later, organizations realized they were locked in, there was different technology. The same thing's true if you go cloud native. You're sort of locked in. So how do you be cloud agnostic? >> How do you get locked in a cloud native? You mean with Amazon? >> Or any of them, right? >> Okay. >> So they all have their own APIs that are really good for doing certain things. So Google's TensorFlow happens to be very good. >> Yeah. Amazon EMR. >> But you build applications that are using those native APIS, you're sort of locked. And maybe you want to switch to something else. How do you do that? So the idea is to-- >> That's why Kubernetes is so important, right now. That's a very key workload and orchestration container-based system. >> That's right, so we believe that containerization of workloads that you can define in one place, and deploy anywhere is the path forward, right? Deploy 'em on prem, deploy 'em in a private cloud, public cloud, it doesn't matter the infrastructure. Infrastructure's irrelevant. Just like Hadoop is sort of not that important anymore. >> So let me get your reaction on this. >> Yeah. So Dell EMC, so you guys have actually been a supplier. They've been the leading supplier, and now with Dell EMC across the portfolio of everything. From Dell computers, servers and what not, to storage, EMC's run the table on that for many generations. Yeah, there's people nippin' at your heels like Pure, okay that's fine. >> Sure. It's still storage is storage. You got to store the data somewhere, so storage will always be around. Here's what I heard from a CXO. This is the pattern I hear, but I'll just summarize it in one conversation. And then you can give a reaction to it. John, my life is hell. I have application development investment plan, it's just boot up all these new developers. New dev ops guys. We're going to do open source, I got to build that out. I got that, trying to get dev ops going on. >> Yep. >> That's a huge initiative. I got the security team. I'm unbundling from my IT department, into a new, difference in a reporting to the board. And then I got all this data governance crap underneath here, and then I got IOT over the top, and I still don't know where my security holes are. >> Yep. And you want to sell me what? (Matt laughs) So that's the fear. >> That's right. >> Their plates are full. How do you guys help that scenario? You walk in, actually security's pretty much, important obviously you can see that. But how do you walk into that conversation? >> Yeah, it's sort of stop the madness, right? >> (laughs) That's right. >> And all of that matters-- >> No, but this is all critical. Every room in the house is on fire. >> It is. >> And I got to get my house in order, so your comment to me better not be hype. TensorFlow, don't give me this TensorFlow stuff. >> That's right. >> I want real deal. >> Right, I need, my guys are-- >> I love TensorFlow but, doesn't put the fire out. >> They just want spark, right? I need to speed up my-- >> John: All right, so how do you help me? >> So, what we'd do is, we want to complement and augment their existing capabilities with better ways of scaling their architecture. So let's help them containerize their big data workload so that they can deploy them anywhere. Let's help them define centralized security policies that can be defined once and enforced everywhere, so that now we have a way to automate the deployment of environments. And users can bring their own tools. They can bring their data from outside, but because we have intelligent centralized policies, we can enforce that. And so with our elastic data platform, we are doing that with partners in the industry, Blue Talent and Blue Data, they provide that capability on top of whatever the customer's infrastructure is. >> How important is it to you guys that Dell EMC are partnering. I know Michael Dell talks about it all the time, so I know it's important. But I want to hear your reaction. Down in the trenches, you're in the front lines, providing the value, pulling things together. Partnerships seem to be really important. Explain how you look at that, how you guys do your partners. You mentioned Blue Talent and Blue Data. >> That's right, well I'm in the consulting organization. So we are on the front lines. We are dealing with customers day in and day out. And they want us to help them solve their problems, not put more of our kit in their data centers, on their desktops. And so partnering is really key, and our job is to find where the problems are with our customers, and find the best tool for the best job. The right thing for the right workload. And you know what? If the customer says, "We're moving to Amazon," then Dell EMC might not sell any more compute infrastructure to that customer. They might, we might not, right? But it's our job to help them get there, and by partnering with organizations, we can help that seamless. And that strengthens the relationship, and they're going to purchase-- >> So you're saying that you will put the customer over Dell EMC? >> Well, the customer is number one to Dell EMC. Net promoter score is one of the most important metrics that we have-- >> Just want to make sure get on the record, and that's important, 'cause Amazon, and you know, we saw it in Net App. I've got to say, give Net App credit. They heard from customers early on that Amazon was important. They started building into Amazon support. So people saying, "Are you crazy?" VMware, everyone's saying, "Hey you capitulated "by going to Amazon." Turns out that that was a damn good move. >> That's right. >> For Kelsinger. >> Yep. >> Look at VM World. They're going to own the cloud service provider market as an arms dealer. >> Yep. >> I mean, you would have thought that a year ago, no way. And then when they did the deal, they said, >> We have really smart leadership in the organization. Obviously Michael is a brilliant man. And it sort of trickles on down. It's customer first, solve the customer's problems, build the relationship with them, and there will be other things that come, right? There will be other needs, other workloads. We do happen to have a private cloud solution with Virtustream. Some of these customers need that intermediary step, before they go full public, with a hosted private solution using a Virtustream. >> All right, so what's the, final question, so what's the number one thing you're working on right now with customers? What's the pattern? You got the stack rank, you're requests, your deliverables, where you spend your time. What's the top things you're working on? >> The top thing right now is scaling architectures. So getting organizations past, they've already got their first 20 use cases. They've already got lakes, they got pedabytes in there. How do we enable self service so that we can actually bring that business value back, as you mentioned. Bring that business value back by making those data scientists productive. That's number one. Number two is aligning that to overall strategy. So organizations want to monetize their data, but they don't really know what that means. And so, within a consulting practice, we help our customers define, and put a road map in place, to align that strategy to their goals, the policies, the security, the GDP, or the regulations. You have to marry the business and the technology together. You can't do either one in isolation. Or ultimately, you're not going to be efficient. >> All right, and just your take on Big Data NYC this year. What's going on in Manhattan this year? What's the big trend from your standpoint? That you could take away from this show besides it becoming a sprawl of you know, everyone just promoting their wares. I mean it's a big, hyped show that O'Reilly does, >> It is. >> But in general, what's the takeaway from the signal? >> It was good hearing from customers this year. Customer segments, I hope to see more of that in the future. Not all just vendors showing their wares. Hearing customers actually talk about the pain and the success that they've had. So the Barclay session where they went up and they talked about their entire journey. It was a packed room, standing room only. They described their journey. And I saw other banks walk up to them and say, "We're feeling the same thing." And this is a highly competitive financial services space. >> Yeah, we had Packsotta's customer on Standard Bank. They came off about their journey, and how they're wrangling automating. Automating's the big thing. Machine learning, automation, no doubt. If people aren't looking at that, they're dead in my mind. I mean, that's what I'm seeing. >> That's right. And you have to get your house in order before you can start doing the fancy gardening. >> John: Yeah. >> And organizations aspire to do the gardening, right? >> I couldn't agree more. You got to be able to drive the car, you got to know how to drive the car if you want to actually play in this game. But it's a good example, the house. Got to get the house in order. Rooms are on fire (laughs) right? Put the fires out, retrench. That's why private cloud's kicking ass right now. I'm telling you right now. Wikibon nailed it in their true private cloud survey. No other firm nailed this. They nailed it, and it went viral. And that is, private cloud is working and growing faster than some areas because the fact of the matter is, there's some bursting through the clouds, and great use cases in the cloud. But, >> Yep. >> People have to get the ops right on premise. >> Matt: That's right, yep. >> I'm not saying on premise is going to be the future. >> Not forever. >> I'm just saying that the stack and rack operational model is going cloud model. >> Yes. >> John: That's absolutely happening, that's growing. You agree? >> Absolutely, we completely, we see that pattern over and over and over again. And it's the Goldilocks problem. There's the organizations that say, "We're never going to go cloud." There's the organizations that say, "We're going to go full cloud." For big data workloads, I think there's an intermediary for the next couple years, while we figure out operating pulse. >> This evolution, what's fun about the market right now, and it's clear to me that, people who try to get a spot too early, there's too many diseconomies of scale. >> Yep. >> Let the evolution, Kubernetes looking good off the tee right now. Docker containers and containerization in general's happened. >> Yep. >> Happening, dev ops is going mainstream. >> Yep. >> So that's going to develop. While that's developing, you get your house in order, and certainly go to the cloud for bursting, and other green field opportunities. >> Sure. >> No doubt. >> But wait until everything's teed up. >> That's right, the right workload in the right place. >> I mean Amazon's got thousands of enterprises using the cloud. >> Yeah, absolutely. >> It's not like people aren't using the cloud. >> No, they're, yeah. >> It's not 100% yet. (laughs) >> And what's the workload, right? What data can you put there? Do you know what data you're putting there? How do you secure that? And how do you do that in a repeatable way. Yeah, and you think cloud's driving the big data market right now. That's what I was saying earlier. I was saying, I think that the cloud is the unsubtext of this show. >> It's enabling. I don't know if it's driving, but it's the enabling factor. It allows for that scale and speed. >> It accelerates. >> Yeah. >> It accelerates... >> That's a better word, accelerates. >> Accelerates that horizontally scalable. Mike, thanks for coming on the CUBE. Really appreciate it. More live action we're going to have some partners on with you guys. Next, stay with us. Live in Manhattan, this is the CUBE. (electronic music)

>> Narrator: Live from midtown Manhattan, it's theCUBE. Covering BigData, New York City, 2017. Brought to you by SiliconANGLE Media and its ecosystem sponsors. >> Welcome back everyone, we're here live in New York City for theCUBE's coverage of BigData NYC, our event we've been running for five years, been covering BigData space for eight years, since 2010 when it was Hadoop World, Strata Conference, Strata Hadoop, Strata Data, soon to be called Strata AI, just a few. We've been theCUBE for all eight years. Here, live in New York, I'm John Furrier. Our next guest is Murthy Mathiprakasam, who is the Director of Product Marketing at Informatica. Cube alumni has been on many times, we cover Informatica World, every year. Great to see you, thanks for coming by and coming in. >> Great to see you. >> You guys do data, so there's not a lot of recycling going on in the data because we've been talking about it all week, total transformation, but the undercurrent has been a lot of AI, AI this, and you guys have the CLAIRE product, doing a lot of things there. But outside of the AI, the undertone is cloud, cloud, cloud. Governance, governance, governance. There's two kind of the drivers I'm seeing as the force of this week is, a lot of people trying to get their act together on those two fronts and you can kind of see the scabs on the industry, people, some people haven't been paying attention. And they're weak in the area. Cloud is absolutely going to be driving the BigData world, 'cause data is horizontal. Cloud's the power source that you guys have been on that. What's your thoughts, what other drivers encourage you? (mumbles) what I'm saying and what else did I miss? Security is obviously in there, but-- >> Absolutely, no, so I think you're exactly right on. So obviously governments security is a big deal. Largely being driven by the GDPR regulation, it's happening in Europe. But, I mean every company today is global, so. Everybody's essentially affected by it. So, I think data until now has always been a kind of opportunistic thing, that there's a couple guys and their organizations were looking at it as oh, let's do some experimentation. Let's do something interesting here. Now, it's becoming government managed so I think there's a lot of organizations who are, like, to your point, getting their act together, and that's driving a lot of demand for data management projects. So now, people say, well, if I got to get my act together, I don't have to hire armies of people to do it, let me look for automated machine learning based ways of doing it. So that they can actually deliver on their audit reports that they need to deliver on, and ensure the compliance that they need to ensure, but do it in a very scalable way. >> I've been kind of joking all week, and I kind of had this meme in my head, so I've been pounding on it all week, calling it the tool shed problem. The tool shed problem is, everyone's got these tools. They throw them into the tool shed. They bought a hammer and the company that sells them the hammer is trying to turn it to a lawnmower, right? You can't mow your lawn with a hammer, it's not going to work, and so this, these tools are great but it defines work. What you do, but, the platforming issue is a huge one. And you start to see people who took that view. You guys were one of them because in a platform centric view with tools that are enabled, to be highly productive. You don't have to worry about new things like a government's policy, the GDPR that might pop up, or the next Equifax that's around the corner. There's probably two or three of them going on right now. So, that's an impact, the data, who uses it, how it's used, and who's at fault or whatever. So, how does a company deal with that? And machine learning has proven to be a great horse that a lot of people are riding right now. You guys are doing it, how does a customer deal with that tsunami of potential threats? Architecture challenges, what is your solution, how do you talk about that? >> Well, I think machine learning, you know, up until now has been seen as the kind of, nice to have, and I think that very quickly, it's going to become a must have. Because, exactly like you're saying, it really is a tsunami. I mean, you could see people who are nervous about the fact that I mean, there's different estimates. It's like 40% growth in data assets from most organizations every year. So, you can try to get around this somehow with one of these (mumbles) tools or something. But at some point, something is going to break, either you just don't, run out of manpower, you can't train the manpower, people start leaving. whatever the operational challenges are, it just isn't going to scale. Machine learning is the only approach. It is absolutely the only approach that actually ensures that you can maintain data for these kind of defensive reasons like you're saying. The structure and compliance, but also the kind of offensive opportunistic reasons, and do it scalably, 'cause there's just no other way mathematically speaking, that when the data is growing 40% a year, just throwing a bunch of tools at it just doesn't work. >> Yeah, I would just amplify and look right in the camera, say, if you're not on machine learning, you're out of business. That's a straight up obvious trend, 'cause that's a precursor to AI, real AI. Alright, let's get down to data management, so when people throw around data management, it's like, oh yeah we've got some data management. There are challenges with that. You guys have been there from day one. But now if you take it out in the future, how do you guys provide the data management in a totally cloud world where now the customer certainly has public and private, or on premise but theirs might have multi cloud? So now, comes a land grab for the data layer, how do you guys play in that? >> Well, I think it's a great opportunity for these kind of middle work platforms that actually do span multiple clouds, that can span the internal environments. So, I'll give you an example. Yesterday we actually had a customer speaking at Astrada here, and he was talking about from him, the cloud is really just a natural extension of what they're already doing, because they already have a sophisticated data practice. This is a large financial services organization, and he's saying well now the data isn't all inside, some of it's outside, you've got partners, who've got data outside. How do we get to that data? Clearly, the cloud is the path for doing that. So, the fact that the cloud is a national extension a lot of organizations were already doing internally means they don't want to have a completely different approach to the data management. They want to have a consistent, simple, systematic repeatable approach to the data management that spans, as you said, on premise in the cloud. That's why I think the opportunity of a very mature and sophisticated platform because you're not rewriting and re-platforming for every new, is it AWS, is it Azure? Is it something on premise? You just want something that works, that shields you from the underlying infrastructure. >> So I put my skeptic hat on for a second and challenge you on this, because this I think is fundamental. Whether it's real or not, it's perceived, maybe in the back of the mind of the CXO or the CDO, whoever is enabled to make these big calls. If they have the keys to the kingdom in Informatica, I'm going to get locked in. So, this is a deep fear. People wake up with nightmares in the enterprise, they've seen locked in before. How do you explain that to a customer that you're going to be an enabling opportunity for them, not a lock in and foreclosing future benefits. Especially if I have an unknown scenario called multi-cloud. I mean, no one's really doing multi-cloud let's face it. I mean, I have multiple clouds with stuff on it, >> At least not intentionally. Sometimes you got a line of businesses and doing things, but absolutely I get it. >> No one's really moving workloads dynamically between clouds in real time. Maybe a few people doing some hacks, but for the most part of course, not a standard practice. >> Right. >> But they want it to be. >> Absolutely. >> So that's the future. From today, how do you preserve that position with the customer where you say hey we're going to add value, but we're not going to lock you in? >> So the whole premise again of, I mean, this goes back to classic three tier models of how you think about technology stacks, right? There's an infrastructure layer, there's a platform layer, there's an analytics layer and the whole premise of the middle of the layer, the platform layer, is that it enables flexibility in the other two layers. It's precisely when you don't have something that's kind of intermediating the data and the use of the data, that's when you run into challenges with flexibility and with data being locked in the data store. But you're absolutely right. We had dinner with a bunch of our customers last night. They were talking about they'd essentially evaluated every version of sort of BigData platform and data infrastructure platform right? And why? It was because they were a large organization and your different teams start stuff and they had to compute them out and stuff. And I was like that must have been pretty hard for you guys. Now what we were using Informatica, so it didn't really matter where the data was, we were still doing everything as far as the data management goes from a consistent layer and we integrate with all those different platforms. >> John: So you didn't get in the way? >> We didn't get in the way. >> You've actually facilitated. >> We are facilitating increased flexibility. Because without a layer like that, a fabric, or whatever you want to call it a data platform that's facilitating this the complexity's going to get very, very crazy very soon. If it hasn't already. The number of infrastructure platforms that are available like you said, on premise and on the cloud now, keeps growing. The number of analytical tools that are available is also growing. And all this is amazing innovation by the way. This is all great stuff, but to your point about it if your the chief officer of an organization going, I got to get this thing figured out somehow. I need some sanity, that's really the purpose of-- >> They just don't want to know the tool for tool's sake, they need to have it be purposeful. >> And that's why this machine learning aspect is very, very critical because I was thinking about an analogy just like you were and I was thinking, in a way you can think of data managing as sort of cleaning stuff up and there are people that have brooms and mops and all these different tools. Well, we are bringing a Roomba to market, right? Because you don't want to just create tools that transfer the laborer around, which is a little bit of what's going on. You want to actually get the laborer out of the equation, so that the people are focused on the context, business strategy and the data management is sort of cleaning itself. It's doing the work for you. That's really what Informatica's vision is. It's about being a kind of enterprise cloud data management vendor that is leveraging AI under the hood so that you can sort of set it and forget it. A lot of this ingestion and the cleansing, telling annals what data they should be looking for. All the stuff is just happening in an automated way and you're not in this total chaos. >> And that can be some tools will be sitting in the back for a long time. In my tool shed, when I had one back in a big enough property back east. No one has tool sheds by the way. No one does any gardening. The issue is in the day, I need to have a reliable partner. So I want you to take a minute and explain to the folks who aren't yet Informatica customers why they should be and the Informatica customers why they should stay with Informatica. >> Absolutely, so certainly the ones we have, a very loyal customer base. In fact the guy who was presenting with us yesterday, he said he's been with Informatica since 1999, going through various versions of our products and adopting new innovations. So we have a very loyal customer base, so I think that loyalty itself speaks for itself as well. As far as net new customers, I think that in a world of this increasing data complexity, it's exactly what you were saying, you need to find an approach that is going to scale. I keep hearing this word from the chief data officer, I kind of got something some going on today, I don't know how I scale it. How is this going to work in 2018 and 2019, in 2025? And it's just daunting for some of these guys. Especially going back to your point about compliance, right? So it's one thing if you have data sitting around, data so to speak, that you're not using it. But god forbid now, you got legal and regulatory concerns around it as well. So you have to get your arms around the data and that's precisely where Informatica can help because we've actually thought through these problems and we've talked about them. >> Most of them were a problem you solved because at the end of the day, we were talking about problems that have massive importance, big time consequences people can actually quantify. >> That's right. >> So what specific problem highest level do you solve is the most important, has the most consequences? >> Everything from ingestion of raw data sets from wherever like you said, in the cloud on premise, all the way through all the processes you need to make it fully usable. And we view that as one problem. There's other vendors who think that one aspect of that is a problem and it is worth solving. We really think, look at the end of the day, you got raw stuff and you have to turn it into useful stuff. Everything in there has to happen, so we might as well just give you everything and be very, very good at doing all those things. And so that's what we call enterprise cloud data management. It's everything from raw material to finished goods of insights. We want to be able to provide that in a consistent integrated and machine learning integrate it. >> Well you guys have a loyal customer base but to be fair and you kind of have to acknowledge that there is a point in time and not throw Informatica's away the big customers, big engagements. But there was a time in Informatica's history where you went private. There was some new management came in. There was a moment where the boat was taking on water, right? And you could almost look at it and say, hmm, you know, we're in this space. You guys retooled around that. Success to the team. Took it to another dimension. So that's the key thing. You know a lot of the companies become big and it's hard to get rid of. So the question is that's a statement. I think you guys done a great job. Yet, the boat might have taken on water, that's my opinion, but you can probably debate that. But I think as you get mature and you're in public, you just went private. But here's the thing, you guys have had a good product chop in Informatica, so I got to ask you the question. What cool things are you doing? Because remember, cool shiny new toys help put a little flash and glam on the nuts and bolts that scales. What are you guys doing? I know you just announced claire, some AI stuff. What's the hot stuff you're doing that's adding value? >> Yeah, absolutely, first of all, this kind of addresses your water comment as well. So we are probably one of the few vendors that spends almost about $200 million in R and D. And that hasn't changed through the acquisition. If anything, I think it actually increased a little bit because now our investors are even more committed to innovation. >> Well you're more nimble in private. A lot more nimble. >> Absolutely, a lot more ideas that are coming to the forefront. So there's never been any water just to be clear. But to answer your follow on question about some examples of this innovation. So I think Ahmed yesterday talked about some of our recent release as well but we really just keep pushing on this idea of, I know I keep saying this but it's this whole machine learning approach here of how can we learn more about the data? So one of the features, I'll give you an example, is if we can actually go look at a file and if we spot like a name and an address and some order information, that probably is a customer, right? And we know that right, because we've seen past data sets. So, there's examples of this pattern matching where you don't even have to have data that's filled out. And this is increasingly the way the data looks we are not dealing with relational tables anymore it's JSON files, it's web blogs, XML files, all of that data that you had to have that data scientists go through and parse and sift through, we just automatically recognize it now. If we can look for the data and understand it, we can match it. >> Put that in context in the order of benefits that, from the old way versus the current way, what's the pain levels? One versus the other, can you put context around that? In terms of, it's pretty significant. >> It's huge because again, back to this sort of volume and variety of data that people are trying to get into systems and do it very rapidly. I'll give you a really tangible customer case. So, this is a customer that presented at Informatica World a couple months ago. It's Jewelry TV, I can actually tell you the name. So there are one of these online kind of shopping sites and they've got a TV program that goes with the online site. So what they do is obviously when you promote something on TV, your orders go up online, right? They wanted to flip it around and they said, look, let's look at the web logs of the traffic that's on the website and then go promote that on the TV program. Because then you get a closed loop and start to have this explosion of sales. So they used Informatica, didn't have to do any of this hand coding. They just build this very quickly and with the graphical user interface that we provide, it leverages sparks streaming under the hood. So they are using all these technologies under the hood, they just didn't have to do any of the manual coding. Got this thing out in a couple days and it works. And they have been able to measure it and they're actually driving increased sales by taking the data and just getting it out to the people that need to see the data very, very quickly. So that's an example of a use case where this isn't just to your point about is this a small, incremental type of thing. No, there is a lot of money behind data if you can actually put it to good use. >> The consequences are grave and I think you've seen more and more, I mean the hacks just amplify it over and over again. It's not a cost center when you think about it. It has to be somehow configured differently as a profit center, even though it might not drive top line revenue directly like an app or anything else. It's not a cost center. If anything it will be treated as a profit center because you get hacked or someone's data is misused, you can be out of business. There is no profit. Look at the results of these hacks. >> The defensive argument is going to become very, very strong as these regulations come out. But, let's be clear, we work with a lot of the most advanced customers. There are people making money off of this. It can be a top line driver-- >> No it should be, it should be. That's exactly the mindset. So the final question for you before we break. I know we're out of time here. There are some chief data officers that are enabled, some aren't and that's just my observation. I don't want to pidgeonhole anyone, but some are enable to really drive change, some are just figureheads that are just managing the compliance risk and work for the CFO and say no to everything. I'm over-generalizing. But that's essentially how I see it. What's the problem with that? Because the cost center issue has, we've seen this moving before in the security business. Security should not be part of IT. That's it's own deal. >> Exactly. >> So we're kind of, this is kind of smoke, but we're coming out of the jungle here. Your thoughts on that. >> Yeah, you're absolutely right. We see a variety of models. We can see the evolution of those models and it's also very contextual to different industries. There are industries that are inherently more regulated, so that's why you're seeing the data people maybe more in those cost center areas that are focused on regulations and things like that. There's other industries that are a lot more consumer oriented. So for them, it makes more sense to have the data people be in a department that seems more revenue basing. So it's not entirely random. There are some reasons, that's not to say that's not the right model moving forward, but someday, you never know. There is a reason why this role became a CXO in the first place. Maybe it is somebody who reports to the CEO and they really view the data department as a strategic function. And it might take a while to get there, but I don't think it's going to take a long time. Again, we're talking about 40% growth in the data and these guys are realizing that now and I think we're going to see very quickly people moving out of the whole tool shed model, and moving to very systematic, repeatable practices. Sophisticated middleware platforms and-- >> As we say don't be a tool, be a platform. Murphy thanks so much for coming on to theCUBE, we really appreciate it. What's going on in Informatica real quick. Things good? >> Things are great. >> Good, awesome. Live from New York, this is theCUBE here at BigData NYC more live coverage continuing day three after this short break. (digital music)

>> Announcer: Live, from Midtown Manhattan, it's theCUBE, covering Big Data New York City 2017. Brought to you by SiliconANGLE Media and its ecosystem sponsors. (upbeat techno music) >> Okay welcome back, we're live here in New York City. It's theCUBE's presentation of Big Data NYC, our fifth year doing this event in conjunction with Strata Data, formerly Strata Hadoop, formerly Strata Conference, formerly Hadoop World, we've been there from the beginning. Eight years covering Hadoop's ecosystem now Big Data. This is theCUBE, I'm John Furrier. Our next guest is Santhosh Mahendiran, who is the global head of technology analytics at Standard Chartered Bank. A practitioner in the field, here getting the data, checking out the scene, giving a presentation on your journey with Data at a bank, which is big financial obviously an adopter. Welcome to theCUBE. >> Thank you very much. >> So we always want to know what the practitioners are doing because at the end of the day there's a lot of vendors selling stuff here, so you got, everyone's got their story. End of the day you got to implement. >> That's right. >> And one of the themes is the data democratization which sounds warm and fuzzy, collaborating with data, this is all good stuff and you feel good and you move into the future, but at the end of the day it's got to have business value. >> That's right. >> And as you look at that, how do you look at the business value? Cause you want to be in the bleeding edge, you want to provide value and get that edge operationally. >> That's right. >> Where's the value in data democratization? How did you guys roll this out? Share your story. >> Okay, so let me start with the journey first before I come to the value part of it, right? So, data democratization is an outcome, but the journey has been something we started three years back. So what did we do, right? So we had some guiding principles to start our journey. The first was to say that we believed in the three S's, which is speed, scale, and it should be really, really flexible and super fast. So one of the challenges that we had was our historical data warehouses was entirely becoming redundant. And why was it? Because it was RDBMS centric, and it was extremely disparate. So we weren't able to scale up to meet the demands of managing huge chunks of data. So, the first step that we did was to re-pivot it to say that okay, let's embrace Hadoop. And what you mean by embracing is just not putting in the data lake, but we said that all our data will land into the data lake. And this journey started in 2015, so we have close to 80% of the Bank's data in the lake and it is end of day data right now and this data flows in on daily basis, and we have consumers who feed off that data. Now coming to your question about-- >> So the data lake's working? >> The data lake is working, up and running. >> People like it, you just got a good spot, batch 'em all you throw everything in the lake. >> So it is not real time, it is end of day. There is some data that is real-time, but the data lake is not entirely real-time, that I have to tell you. But one part is that the data lake is working. Second part to your question is how do I actually monetize it? Are you getting some value out of it? But I think that's where tools like Paxata has actually enabled us to accelerate this journey. So we call it data democratization. So the best part it's not about having the data. We want the business users to actually use the data. Typically, data has always been either delayed or denied in most of the cases to end-users and we have end-users waiting for the data but they don't get access to the data. It was done because primarily the size of the data was too huge and it wasn't flexible enough to be shared with. So how did tools like Paxata and the data lake help us? So what we did with data democratization is basically to say that "hey we'll get end-users to access the data first in a fast manner, in a self-service manner, and something that gives operational assurance to the data, so you don't hold the data and then say that you're going to get a subset of data to play with. We'll give you the entire set of data and we'll give you the right tools which you can play with. Most importantly, from an IT perspective, we'll be able to govern it. So that's the key about democratization. It's not about just giving them a tool, giving them all data and then say "go figure it out." It's about ensuring that "okay, you've got the tools, you've got the data, but we'll also govern it," so that you obviously have control over what they're doing. >> So now you govern it, they don't have to get involved in the governance, they just have access? >> No they don't need to. Yeah, they have access. So governance works both ways. We establish the boundaries. Look at it as a referee, and then say that "okay, there are guidelines that you don't," and within the datasets that key people have access to, you can further set rules. Now, coming back to specific use cases, I can talk about two specific cases which actually helped us to move the needle. The first is on stress testing, so being a financial institution, we typically have to report various numbers to our regulators, etc. The turnaround time was extremely huge. These kind of stress testing typically involve taking huge amount-- >> What were some of the turnaround times? >> Normally it was two to three weeks, some cases a month-- >> Wow. >> So we were able to narrow it down to days, but what we essentially did was as with any stress testing or reporting, it involved taking huge amounts of data, crunching them and then running some models and then showing the output, basically a number of transformations involved. Earlier, you first couldn't access the entire dataset, so that we solved-- >> So check, that was a good step one-- >> That was step one. >> But was there automation involved in that, the Paxata piece? >> Yeah, I wouldn't say it was fully automated end-to-end, but there was definitely automation given the fact that now you got Paxata to work off the data rather than someone extracting the data and then going off and figuring what needs to be done. The ability to work off the entire dataset was a big plus. So stress testing, bringing down the cycle time. The second one use case I can talk about is again anti-money laundering, and in our financial crime compliance space. We had processes that took time to report, given the clunkiness in the various handoffs that we needed to do. But again, empowering the users, giving the tool to them and then saying "hey, this"-- >> How about know your user, because we have to anti-money launder, you need to have to know your user base, that's all set their too? >> Yeah. So the good part is know the user, know your customer, KYCs all that part is set, but the key part is making sure the end-users are able to access the data much more earlier in the life cycle and are able to play with it. In the case of anti-money laundering, again first question of three weeks to four weeks was shortened down to question of days by giving tools like Paxata again in a structured manner and with which we're able to govern. >> You control this, so you knew what you were doing, but you let their tools do the job? >> Correct, so look at it this way. Typically, the data journey has always been IT-led. It has never been business-led. If you look at the generations of what happens is, you source the data which is IT-led, then you model the data which is IT-led, then you prepare then massage the data which is again IT-led and then you have tools on top of it which is again IT-led so the end-users get it only after the fourth stage. Now look at the generations within. All these life cycles apart from the fact that you source the data which is typically an IT issue, the rest need to be done by the actual business users and that's what we did. That's the progression of the generations in which we now we're in the third generation as I call it where our role is just to source the data and then say, "yeah we'll govern it in the matter and then preparation-- >> It's really an operating system and we were talking with Aaron with Elation's co-founder, we used the analogy of a car, how this show was like a car show engine show, what's in the engine and the technology and then it evolved every year, now it's like we're talking about the cars, now we're talking about driver experience-- >> That's right. >> At the end of the day, you just want to drive. You don't really care what's under the hood, you do but you don't, but there's those people who do care what's under the hood, so you can have best of both worlds. You've got the engines, you set up the infrastructure, but ultimately, you in the business side, you just want to drive, that's what's you're getting at? >> That's right. The time-to-market and speed to empower the users to play around with the data rather than IT trying to churn the data and confine access to data, that's a thing of the past. So we want more users to have faster access to data but at the same time govern it in a seamless manner. The word governance is still important because it's not about just give the data. >> And seamless is key. >> Seamless is key. >> Cause if you have democratization of data, you're implying that it is community-oriented, means that it's available, with access privileges all transparently or abstracted away from the users. >> Absolutely. >> So here's the question I want to ask you. There's been talk, I've been saying it for years going back to 2012 that an abstraction layer, a data layer will evolve and that'll be the real key. And then here in this show, I heard things like intelligent information fabric that is business, consumer-friendly. Okay, it's a mouthful, but intelligent information fabric in essence talks about an abstraction layer-- >> That's right. >> That doesn't really compromise anything but gives some enablement, creates some enabling value-- >> That's right. >> For software, how do you see that? >> As the word suggests, the earlier model was trying to build something for the end-users, but not which was end-user friendly, meaning to say, let me just give you a simple example. You had a data model that existed. Historically the way that we have approached using data is to say "hey, I've got a model and then let's fit that data into this model," without actually saying that "does this model actually serve the purpose?" You abstracted the model to a higher level. The whole point about intelligent data is about saying that, I'll give you a very simple analogy. Take zip code. Zipcode in US is very different from zipcode in India, it's very different from zipcode in Singapore. So if I had the ability for my data to come in, to say that "I know it's a zipcode, but this zipcode belongs to US, this zipcode belongs to Singapore, and this zipcode belongs to India," and more importantly, if I can further rev it up a notch, if I say that "this belongs to India, and this zipcode is valid." Look at where I'm going with intelligent sense. So that's what's up. If you look at the earlier model, you have to say that "yeah, this is a placeholder for zipcode." Now that makes sense, but what are you doing with it? >> Being a relational database model, it's just a field in a schema, you're taking it and abstracting it and creating value out of it. >> Precisely. So what I'm actually doing is accelerating the adoption, I'm making it more simpler for users to understand what the data is. So I don't need to as a user figure out "I got a zipcode, now is it a Singapore, India or what zipcode." >> So all this automation, Paxata's got a good system, we'll come back to the Paxata question in a second, I do want to drill down on that. But the big thing that I've been seeing at the show, and again Dave Alonte, my partner, co-CEO of Silicon Angle, we always talk about this all the time. He's more less bullish on Hadoop than I am. Although I love Hadoop, I think it's great but it's not the end-all, be-all. It's a great use case. We were critical early on and the thing we were critical on it was it was too much time being spent on the engine and how things are built, not on the business value. So there's like a lull period in the business where it was just too costly-- >> That's right. >> Total cost of ownership was a huge, huge problem. >> That's right. >> So now today, how did you deal with that and are you measuring the TCO or total cost of ownership cause at the end of the day, time to value, which is can you be up and running in 90 days with value and can you continue to do that, and then what's the overall cost to get there. Thoughts? >> So look I think TCO always underpins any technology investment. If someone said I'm doing a technology investment without thinking about TCO, I don't think he's a good technology leader, so TCO is obviously a driving factor. But TCO has multiple components. One is the TCO of the solution. The other aspect is TCO of what my value I'm going to get out of this system. So talking from an implementation perspective, what I look at as TCO is my whole ecosystem which is my hardware, software, so you spoke about Hadoop, you spoke about RDBMS, is Hadoop cheaper, etc? I don't want to get into that debate of cheaper or not but what I know is the ecosystem is becoming much, much more cheaper than before. And when I talk about ecosystem, I'm talking about RDBMS tools, I'm talking about Hadoop, I'm talking about BI tools, I'm talking about governance, I'm talking about this whole framework becoming cheaper. And it is also underpinned by the fact that hardware is also becoming cheaper. So the reality is all components in the whole ecosystem are becoming cheaper and given the fact that software is also becoming more open-sourced and people are open to using open-source software, I think the whole question of TCO becomes a much more pertinent question. Now coming to your point, do you measure it regularly? I think the honest answer is I don't think we are doing a good job of measuring it that well, but we do have that as one of the criteria for us to actually measure the success of our project. The way that we do is our implementation cost, at the time of writing out our PETs, we call it PETs, which is the Project Execution Document, we talk about cost. We say that "what's the implementation cost?" What are the business cases that are going to be an outcome of this? I'll give you an example of our anti-money laundering. I told you we reduced our cycle time from few weeks to a few days, and that in turn means the number of people involved in this whole process, you're reducing the overheads and the operational folks involved in it. That itself tells you how much we're able to save. So definitely, TCO is there and to say that-- >> And you are mindful of, it's what you look at, it's key. TCO is on your radar 100% you evaluate that into your deals? >> Yes, we do. >> So Paxata, what's so great about Paxata? Obviously you've had success with them. You're a customer, what's the deal. Was it the tech, was it the automation, the team? What was the key thing that got you engaged with them or specifically why Paxata? >> Look, I think the key to partnership there cannot be one ingredient that makes a partnership successful, I think there are multiple ingredients that make a partnership successful. We were one of the earliest adopters of Paxata. Given that we're a bank and we have multiple different systems and we have lot of manual processing involved, we saw Paxata as a good fit to govern these processes and ensure at the same time, users don't lose their experience. The good thing about Paxata that we like was obviously the simplicity and the look and feel of the tool. That's number one. Simplicity was a big point. The second one is about scale. The scale, the fact that it can take in millions of roles, it's not about just working off a sample of data. It can work on the entire dataset. That's very key for us. The third is to leverage our ecosystem, so it's not about saying "okay you give me this data, let me go figure out what to do and then," so Paxata works off the data lake. The fact that it can leverage the lake that we built, the fact that it's a simple and self-preparation tool which doesn't require a lot of time to bootstrap, so end-use people like you-- >> So it makes it usable. >> It's extremely user-friendly and usable in a very short period of time. >> And that helped with the journey? >> That really helped with the journey. >> Santosh, thanks so much for sharing. Santosh Mahendiran, who is the Global Tech Lead at the Analytics of the Bank at Standard Chartered Bank. Again, financial services, always a great early adopter, and you get success under your belt, congratulations. Data democratization is huge and again, it's an ecosystem, you got all that anti-money laundering to figure out, you got to get those reports out, lot of heavylifting? >> That's right, >> So thanks so much for sharing your story. >> Thank you very much. >> We'll give you more coverage after this short break, I'm John Furrier, stay tuned. More live coverage in New York City, its theCube.

>> Announcer: Live from midtown Manhattan, it's the Cube. Covering Big Data New York City 2017. Brought to you by SiliconANGLE Media and its ecosystem sponsors. >> Welcome back everyone, we are here live in New York City, in Manhattan for BigData NYC, our event we've been doing for five years in conjunction with Strata Data which is formerly Strata Hadoop, which was formerly Strata Conference, formerly Hadoop World. We've been covering the big data space going on ten years now. This is the Cube. I'm here with Aaron Kalb, whose Head of Product and co-founder at Alation. Welcome to the cube. >> Aaron Kalb: Thank you so much for having me. >> Great to have you on, so co-founder head of product, love these conversations because you're also co-founder, so it's your company, you got a lot of equity interest in that, but also head of product you get to have the 20 mile stare, on what the future looks, while inventing it today, bringing it to market. So you guys have an interesting take on the collaboration of data. Talk about what the means, what's the motivation behind that positioning, what's the core thesis around Alation? >> Totally so the thing we've observed is a lot of people working in the data space, are concerned about the data itself. How can we make it cheaper to store, faster to process. And we're really concerned with the human side of it. Data's only valuable if it's used by people, how do we help people find the data, understand the data, trust in the data, and that involves a mix of algorithmic approaches and also human collaboration, both human to human and human to computer to get that all organized. >> John Furrier: It's interesting you have a symbolics background from Stanford, worked at Apple, involved in Siri, all this kind of futuristic stuff. You can't go a day without hearing about Alexia is going to have voice-activated, you've got Siri. AI is taking a really big part of this. Obviously all of the hype right now, but what it means is the software is going to play a key role as an interface. And this symbolic systems almost brings on this neural network kind of vibe, where objects, data, plays a critical role. >> Oh, absolutely, yeah, and in the early days when we were co-founding the company, we talked about what is Siri for the enterprise? Right, I was you know very excited to work on Siri, and it's really a kind of fun gimmick, and it's really useful when you're in the car, your hands are covered in cookie dough, but if you could answer questions like what was revenue last quarter in the UK and get the right answer fast, and have that dialogue, oh do you mean fiscal quarter or calendar quarter. Do you mean UK including Ireland, or whatever it is. That would really enable better decisions and a better outcome. >> I was worried that Siri might do something here. Hey Siri, oh there it is, okay be careful, I don't want it to answer and take over my job. >> (laughs) >> Automation will take away the job, maybe Siri will be doing interviews. Okay let's take a step back. You guys are doing well as a start up, you've got some great funding, great investors. How are you guys doing on the product? Give us a quick highlight on where you guys are, obviously this is BigData NYC a lot going on, it's Manhattan, you've got financial services, big industry here. You've got the Strata Data event which is the classic Hadoop industry that's morphed into data. Which really is overlapping with cloud, IoTs application developments all kind of coming together. How do you guys fit into that world? >> Yeah, absolutely, so the idea of the data lake is kind of interesting. Psychologically it's sort of a hoarder mentality, oh everything I've ever had I want to keep in the attic, because I might need it one day. Great opportunity to evolve these new streams of data, with IoT and what not, but just cause you can get to it physically doesn't mean it's easy to find the thing you want, the needle in all that big haystack and to distinguish from among all the different assets that are available, which is the one that is actually trustworthy for your need. So we find that all these trends make the need for a catalog to kind of organize that information and get what you want all the more valuable. >> This has come up a lot, I want to get into the integration piece and how you're dealing with your partnerships, but the data lake integration has been huge, and having the catalog has come up with, has been the buzz. Foundationally if you will saying catalog is important. Why is it important to do the catalog work up front, with a lot of the data strategies? >> It's a great question, so, we see data cataloging as step zero. Before you can prep the data in a tool like Trifacta, PACSAT, or Kylo. Before you can visualize it in a tool like Tableau, or MicroStrategy. Before you can do some sort of cool prediction of what's going to happen in the future, with a data science engine, before any of that. These are all garbage in garbage out processes. The step zero is find the relevant data. Understand it so you can get it in the right format. Trust that it's good and then you can do whatever comes next >> And governance has become a key thing here, we've heard of the regulations, GDPR outside of the United States, but also that's going to have an arms length reach over into the United States impact. So these little decisions, and there's going to be an Equifax someday out there. Another one's probably going to come around the corner. How does the policy injection change the catalog equation? A lot of people are building machine learning algorithms on top of catalogs, and they're worried they might have to rewrite everything. How do you balance the trade off between good catalog design and flexibility on the algorithm side? >> Totally yes it's a complicated thing with governance and consumption right. There's people who are concerned with keeping the data safe, and there are people concerned with turning that data into real value, and these can seem to be at odds. What we find is actually a catalog as a foundation for both, and they are not as opposed as they seem. What Alation fundamentally does is we make a map of where the data is, who's using what data, when, how. And that can actually be helpful if your goal is to say let's follow in the footsteps of the best analyst and make more insights generated or if you want to say, hey this data is being used a lot, let's make sure it's being used correctly. >> And by the right people. >> And by the right people exactly >> Equifax they were fishing that pond dry months, months before it actually happened. With good tools like this they might have seen this right? Am I getting it right? >> That's exactly right, how can you observe what's going on to make sure it's compliant and that the answers are correct and that it's happening quickly and driving results. >> So in a way you're taking the collective intelligence of the user behavior and using that into understanding what to do with the data modeling? >> That's exactly right. We want to make each person in your organization as knowledgeable as all of their peers combined. >> So the benefit then for the customer would be if you see something that's developing you can double down on it. And if the users are using a lot of data, then you can provision more technology, more software. >> Absolutely, absolutely. It's sort of like when I was going to Stanford, there was a place where the grass was all dead, because people were riding their bikes diagonally across it. And then somebody smart was like, we're going to put a real gravel path there. So the infrastructure should follow the usage, instead of being something you try to enforce on people. >> It's a classic design meme that goes around. Good design is here, the more effective design is the path. >> Exactly. >> So let's get into the integration. So one of the hot topics here this year obviously besides cloud and AI, with cloud really being more the driver, the tailwind for the growth, AI being more the futuristic head room, is integration. You guys have some partnerships that you announced with integration, what are some of the key ones, and why are they important? >> Absolutely, so, there have been attempts in the past to centralize all the data in one place have one warehouse or one lake have one BI tool. And those generally fail, for different reasons, different teams pick different stacks that work for them. What we think is important is the single source of reference One hub with spokes out to all those different points. If you think about it it's like Google, it's one index of the whole web even though the web is distributed all over the place. To make that happen it's very important that we have partnerships to get data in from various sources. So we have partnerships with database vendors, with Cloudera and Hortonworks, with different BI tools. What's new are a few things. One is with Cloudera Navigator, they have great technical metadata around security and lineage over HGFS, and that's a way to bolster our catalog to go even deeper into what's happening in the files before things get surfaced and higher for places where we have a deeper offering today. >> So it's almost a connector to them in a way, you kind of share data. >> That's exactly right, we've a lot of different connectors, this is one new one that we have. Another, go ahead. >> I was going to go ahead continue. >> I was just going to say another place that is exciting is data prep tools, so Trifacta and Paxata are both places where you can find and understand an alation and then begin to manipulate in those tools. We announced with Paxata yesterday, the ability to click to profile, so if you want to actually see what's in some raw compressed avro file, you can see that in one click. >> It's interesting, Paxata has really been almost lapping, Trifacta because they were the leader in my mind, but now you've got like a Nascar race going on between the two firms, because data wrangling is a huge issue. Data prep is where everyone is stuck right now, they just want to do the data science, it's interesting. >> They are both amazing companies and I'm happy to partner with both. And actually Trifacta and Alation have a lot of joint customers we're psyched to work with as well. I think what's interesting is that data prep, and this is beginning to happen with analyst definitions of that field. It isn't just preparing the data to be used, getting it cleaned and shaped, it's also preparing the humans to use the data giving them the confidence, the tools, the knowledge to know how to manipulate it. >> And it's great progress. So the question I wanted to ask is now the other big trend here is, I mean it's kind of a subtext in this show, it's not really front and center but we've been seeing it kind of emerge as a concept, we see in the cloud world, on premise vs cloud. On premise a lot of people bring in the dev ops model in, and saying I may move to the cloud for bursting and some native applications, but at the end of the day there is a lot of work going on on premise. A lot of companies are kind of cleaning house, retooling, replatforming, whatever you want to do resetting. They are kind of getting their house in order to do on prem cloud ops, meaning a business model of cloud operations on site. A lot of people doing that, that will impact the story, it's going to impact some of the server modeling, that's a hot trend. How do you guys deal with the on premise cloud dynamic? >> Totally, so we just want to do what's right for the customer, so we deploy both on prem and in the cloud and then from wherever the Alation server is it will point to usually a mix of sources, some that are in the cloud like vetshifter S3 often with Amazon today, and also sources that are on prem. I do think I'm seeing a trend more and more toward the cloud and we have people that are migrating from HGFS to S3 is one thing we hear a lot about it. Strata with sort of dupe interest. But I think what's happening is people are realizing as each Equifax in turn happens, that this old wild west model of oh you surround your bank with people on horseback and it's physically in one place. With data it isn't like that, most people are saying I'd rather have the A+ teams at Salesforce or Amazon or Google be responsible for my security, then the people I can get over in the midwest. >> And the Paxata guys have loved the term Data Democracy, because that is really democratization, making the data free but also having the governance thing. So tell me about the Data Lake governance, because I've never loved the term Data Lake, I think it's more of a data ocean, but now you see data lake, data lake, data lake. Are they just silos of data lakes happening now? Are people trying to connect them? That's key, so that's been a key trend here. How do you handle the governance across multiple data lakes? >> That's right so the key is to have that single source of reference, so that regardless of which lake or warehouse, or little siloed Sequel server somewhere, that you can search in a single portal and find that thing no matter where it is. >> John: Can you guys do that? >> We can do that, yeah, I think the metaphor for people who haven't seen it really is Google, if you think about it, you don't even know what physical server a webpage is hosted from. >> Data lakes should just be invisible >> Exactly. >> So your interfacing with multiple data lakes, that's a value proposition for you. >> That's right so it could be on prem or in the cloud, multi-cloud. >> Can you share an example of a customer that uses that and kind of how it's laid out? >> Absolutely, so one great example of an interesting data environment is eBay. They have the biggest teradata warehouse in the world. They also have I believe two huge data lakes, they have hive on top of that, and Presto is used to sort of virtualize it across a mixture of teradata, and hive and then direct Presto query It gets very complicated, and they have, they are a very data driven organization, so they have people who are product owners who are in jobs where data isn't in their job title and they know how to look at excel and look at numbers and make choices, but they aren't real data people. Alation provides that accessibility so that they can understand it. >> We used to call the Hadoop world the car show for the data world, where for a long time it was about the engine what was doing what, and then it became, what's the car, and now how's it drive. Seeing that same evolution now where all that stuff has to get done under the hood. >> Aaron: Exactly. >> But there are still people who care about that, right. They are the mechanics, they are the plumbers, whatever you want to call them, but then the data science are the guys really driving things and now end users potentially, and even applications bots or what nots. It seems to evolve, that's where we're kind of seeing the show change a little bit, and that's kind of where you see some of the AI things. I want to get your thoughts on how you or your guys are using AI, how you see AI, if it's AI at all if it's just machine learning as a baby step into AI, we all know what AI could be, but it's really just machine learning now. How do you guys use quote AI and how has it evolved? >> It's a really insightful question and a great metaphor that I love. If you think about it, it used to be how do you build the car, and now I can drive the car even though I couldn't build it or even fix it, and soon I don't even have to drive the car, the car will just drive me, all I have to know is where I want to go. That's sortof the progression that we see as well. There's a lot of talk about deep learning, all these different approaches, and it's super interesting and exciting. But I think even more interesting than the algorithms are the applications. And so for us it's like today how do we get that turn by turn directions where we say turn left at the light if you want to get there And eventually you know maybe the computer can do it for you The thing that is also interesting is to make these algorithms work no matter how good your algorithm is it's all based on the quality of your training data. >> John: Which is a historical data. Historical data in essence the more historical data you have you need that to train the data. >> Exactly right, and we call this behavior IO how do we look at all the prior human behavior to drive better behavior in the future. And I think the key for us is we don't want to have a bunch of unpaid >> John: You can actually get that URL behavioral IO. >> We should do it before it's too late (Both laugh) >> We're live right now, go register that Patrick. >> Yeah so the goal is we don't want to have a bunch of unpaid interns trying to manually attack things, that's error prone and that's slow. I look at things like Luis von Ahn over at CMU, he does a thing where as you're writing in a CAPTCHA to get an email account you're also helping Google recognize a hard to read address or a piece of text from books. >> John: If you shoot the arrow forward, you just take this kind of forward, you almost think augmented reality is a pretext to what we might see for what you're talking about and ultimately VR are you seeing some of the use cases for virtual reality be very enterprise oriented or even end consumer. I mean Tom Brady the best quarterback of all time, he uses virtual reality to play the offense virtually before every game, he's a power user, in pharma you see them using virtual reality to do data mining without being in the lab, so lab tests. So you're seeing augmentation coming in to this turn by turn direction analogy. >> It's exactly, I think it's the other half of it. So we use AI, we use techniques to get great data from people and then we do extra work watching their behavior to learn what's right. And to figure out if there are recommendations, but then you serve those recommendations, either it's Google glasses it appears right there in your field of view. We just have to figure out how do we make sure, that in a moment of you're making a dashboard, or you're making a choice that you have that information right on hand. >> So since you're a technical geek, and a lot of folks would love to talk about this, so I'll ask you a tough question cause this is something everyone is trying to chase for the holy grail. How do you get the right piece of data at the right place at the right time, given that you have all these legacy silos, latencies and network issues as well, so you've got a data warehouse, you've got stuff in cold storage, and I've got an app and I'm doing something, there could be any points of data in the world that could be in milliseconds potentially on my phone or in my device my internet of thing wearable. How do you make that happen? Because that's the struggle, at the same time keep all the compliance and all the overhead involved, is it more compute, is it an architectural challenge how do you view that because this is the big challenge of our time. >> Yeah again I actually think it's the human challenge more than the technology challenge. It is true that there is data all over the place kind of gathering dust, but again if you think about Google, billions of web pages, I only care about the one I'm about to use. So for us it's really about being in that moment of writing a query, building a chart, how do we say in that moment, hey you're using an out of date definition of profit. Or hey the database you chose to use, the one thing you chose out of the millions that is actually is broken and stale. And we have interventions to do that with our partners and through our own first party apps that actually change how decisions get made at companies. >> So to make that happen, if I imagine it, you'd have to need access to the data, and then write software that is contextually aware to then run, compute, in context to the user interaction. >> It's exactly right, back to the turn by turn directions concept you have to know both where you're trying to go and where you are. And so for us that can be the from where I'm writing a Sequel statement after join we can suggest the table most commonly joined with that, but also overlay onto that the fact that the most commonly joined table was deprecated by a data steward data curator. So that's the moment that we can change the behavior from bad to good. >> So a chief data officer out there, we've got to wrap up, but I wanted to ask one final question, There's a chief data officer out there they might be empowered or they might be just a CFO assistant that's managing compliance, either way, someone's going to be empowered in an organization to drive data science and data value forward because there is so much proof that data science works. From military to play you're seeing examples where being data driven actually has benefits. So everyone is trying to get there. How do you explain the vision of Alation to that prospect? Because they have so much to select from, there's so much noise, there's like, we call it the tool shed out there, there's like a zillion tools out there there's like a zillion platforms, some tools are trying to turn into something else, a hammer is trying to be a lawnmower. So they've got to be careful on who the select, so what's the vision of Alation to that chief data officer, or that person in charge of analytics to scale operational analytics. >> Absolutely so we say to the CDO we have a shared vision for this place where your company is making decisions based on data, instead of based on gut, or expensive consultants months too late. And the way we get there, the reason Alation adds value is, we're sort of the last tool you have to buy, because with this lake mentality, you've got your tool shed with all the tools, you've got your library with all the books, but they're just in a pile on the floor, if you had a tool that had everything organized, so you just said hey robot, I need an hammer and this size nail and this text book on this set of information and it could just come to you, and it would be correct and it would be quick, then you could actually get value out of all the expense you've already put in this infrastructure, that's especially true on the lake. >> And also tools describe the way the works done so in that model tools can be in the tool shed no one needs to know it's in there. >> Aaron: Exactly. >> You guys can help scale that. Well congratulations and just how far along are you guys in terms of number of employees, how many customers do you have? If you can share that, I don't know if that's confidential or what not >> Absolutely, so we're small but growing very fast planning to double in the next year, and in terms of customers, we've got 85 customers including some really big names. I mentioned eBay, Pfizer, Safeway Albertsons, Tesco, Meijer. >> And what are they saying to you guys, why are they buying, why are they happy? >> They share that same vision of a more data driven enterprise, where humans are empowered to find out, understand, and trust data to make more informed choices for the business, and that's why they come and come back. >> And that's the product roadmap, ethos, for you guys that's the guiding principle? >> Yeah the ultimate goal is to empower humans with information. >> Alright Aaron thanks for coming on the Cube. Aaron Kalb, co-founder head of product for Alation here in New York City for BigData NYC and also Strata Data I'm John Furrier thanks for watching. We'll be right back with more after this short break.

>> Narrator: Live from Midtown Manhattan, it's theCUBE. Covering Big Data New York City 2017. Brought to you by SiliconANGLE Media and its ecosystem sponsors. >> Hello everyone. Welcome back to our CUBE coverage here in New York City, live in Manhattan for theCUBE's coverage of Big Data NYC, our event we've had five years in a row. Eight years covering Big Data, Hadoop World originally in 2010, then it moved to Hadoop Strata Conference, Strata Hadoop, now called Strata Data. In conjunction with that event we have our Big Data NYC event. SiliconANGLE Media's CUBE. I'm John Furrier, your cohost, with Jim Kobielus, analyst at wikibon.com for Big Data. Our next guest is Gus Horn who is the global Big Data analytics and CTO ambassador for NetApp, machine learning, AI, guru, gives talks all around the world. Great to have you, thanks for coming in and spending the time with us. >> Thanks, John, appreciate it. >> So we were talking before the camera came on, you're doing a lot of jet setting really around Evangelize But also educating a lot of folks on the impact of machine learning and AI in particular. Obviously AI we love, we love the hype. It motivates young kids getting into software development, computer science, makes it kind of real for them. But still, a lot more ways to go in terms of what AI really is. And that's good, but what is really going on with AI? Machine learning is where the rubber hits the road. That seems to be the hot air, that's your wheelhouse. Give us the update, where is AI now? Obviously machine learning is super important, it's one of the hot topics here in New York City. >> Well, I think it's super important globally, and it's going to be disruptive. So before we were talking, I said how this is going to be a disruptive technology for all of society. But regardless of that, what machine learning is bringing is a methodology to deal with this influx of IOT data, whether it's autonomous vehicles, active safety in cars, or even looking at predictive analytics for complex manufacturing processes like an automotive assembly line. Can I predict when a welding machine is going to break and can I take care of it during a scheduled maintenance cycle so I don't take the whole line down? Because the impacts are really cascading and dramatic when you have a failure that you couldn't predict. And what we're finding is that Hadoop and the Big Data space is uniquely positioned to help solve these problems, both from quality control and process management and how you can get better uptime, better quality, and then we take it full circle and how can I build an environment to help automotive manufacturers to do test and DEV and retest and retraining and learning of the AI modules and the AI engines that have to exist in these autonomous vehicles. And the only way you can do that is with data, and managing data like a data steward, which is what we do at NetApp. So for us, it's not just about the solution, but the underlying architecture is going to be absolutely critical in setting up the agility you'll need in this environment, and the flexibility you need. Because the other thing that's happening in the space right now is that technology's evolving very quickly. You see this with the DGX from NVIDIA, you see P100 cards from NVIDIA. So I have an architecture that we have in Germany right now where we have multiple NVIDIA cards in our Hadoop cluster that we've architected. But I don't make NVIDIA cards. I don't make servers. I make really good storage. And I have an ecosystem that helps manage where that data is when it needs to be there, and especially when it doesn't need to there so we can get new data. >> Yeah, Gus, we were talking also before camera, the folks watching that you were involved with AI going way back to in your days at MIT, and that's super important. Because a lot of people, the pattern that we're seeing across all the events that we go to, and we'll be at the NetApp event next week, Insight, in Vegas, but the pattern is pretty clear. You have one camp, oh, AI is just the same thing that was going on in the late '70s, '80s, and '90s, but it now has a new dynamic with the cloud. So a lot of people are saying okay, there's been some concepts that have been developed in AI, in computer science, but now with the evolution of hyperconvergence infrastructure, with cloud computing, with now a new architecture, it seems to be turbocharging and accelerating. So I'd like to get your thoughts on why is it so hot now? Obviously machine learning, everyone should be on that, no doubt, but you got the dynamic of the cloud. And NetApp's in the storage business, so that's stores data, I get that. What's the dynamic with the cloud? Because that seems to be the accelerant right now with open source and in with AI. >> Yeah, I think you got to stay focused. The cloud is going to be playing an integral role in everything. And what we do at NetApp as a data steward, and what George Kurian said, our CEO, that data is the currency of today actually, right? It's really fundamentally what drives business value, it's the data. But there's one little slight attribute change that I'd like to add to that, and that it's a perishable commodity. It has a certain value at T-sub zero when you first get it. And especially true when you're trying to do machine learning and you're trying to learn new events and new things, but it rapidly degrades and becomes less valuable. You still need to keep it because it's historical and if we forget historical data, we're doomed to repeat mistakes. So you need to keep it and you have to be a good steward. And that's where we come into play with our technologies. Because we have a portfolio of different kinds of products and management capabilities that move the data where it needs to be, whether you're in the cloud, whether you're near the cloud, like in an Equinox colo, or even on prem. And the key attribute there, and especially in automotive they want to keep the data forever because of liability, because of intellectual property and privacy concerns. >> Hold on, one quick question on that. 'Cause I think you bring up a good point. The perishability's interesting because realtime, we see this now, bashing in realtime is the buzzword in the industry, but you're talking about something that's really important. That the value of the data when you get it fast, in context, is super important. But then the historical piece where you store it also plays into the machine learning dynamics of how deep learning and machine learning has to use the historical perspective. So in a way, it's perishable in the realtime piece in the moment. If you're a self-driving car you want the data in milliseconds 'cause it's important, but then again, the historical data will then come back. Is that kind of where you're getting at with that? >> Yeah, because the way that these systems operate is the paradigm is like deep learning. You want them to learn the way a human learns, right? The only reason we walk on our feet is 'cause we fell down a lot. But we remember falling down, we remember how we got up and could walk. So if you don't have the historical context, you're just always falling down, right? So you have to have that to build up the proper machine learning neural network, the kind of connections you need to do the right things. And then as you get new data and varieties of data, and I'll stick with automotive, because it can almost be thought of as an intractable amount of data. Because most people will keep cars for measured in decades. The quality of the car is incredible now, and they're all just loaded with sensors, right? High definition cameras, radars, GPS tracking. And you want to make sure you get improvements there because you have liability issues coming as well with these same technologies, so. >> Yeah, so we talk about the perishability of the data, that's a given. What is less perishable, it seems to me and Wikibon, is that what you derive from the data, the correlations, the patterns, the predictive models, the meat of machine learning and deep learning, AI in general, is less perishable in the sense that it has a validity over time. What are your thoughts at NetApp about how those data derived assets should be stored, should be managed for backup and recovery and protected? To what extent do those requirements need to be reflected in your storage retention policies if you're an enterprise doing this? >> That's a great question. So I think what we find is that that first landing zone, and everybody talks about that being the cloud. And for me it's a cloudy day, although in New York today it's not. There are lots of clouds and there are lots of other things that come with that data like GDPR and privacy, and what are you allowed to store, what are you allowed to keep? And how do you distinguish one from the other? That's one part. But then you're going to have to ETL it, you're going to have to transform that data. Because like everything, there's a lot of noise. And the noise is really fundamentally not that important. It's those anomalies within the stream of noise that you need to capture. And then use that as your training data, right? So that you learn from it. So there's a lot of processing, I think, that's going to have to happen in the cloud regardless of what cloud, and it has to be kind of ubiquitous in every cloud. And then from there you decide, how am I going to curate the data and move it? And then how am I going to monetize the data? Because that's another part of the equation, and what can I monetize? >> Well that's a question that we hear a lot on theCUBE. On day one we were ripping at some of the concepts that we see, and certainly we talk to enterprise customers. Whether it's a CIO, CVO, chief data officer, chief security officer. There's a huge application development going on in the enterprise right now. You see the opensource booming. This huge security practice is being built up and then it's got this governance with the data. Overlay that with IOT, it's kind of an architectural, I don't want to say reset, but a retrenching for a lot of enterprises. So the question I have for you guys as a critical part of the infrastructure of storage, storage isn't going away, there's no doubt about that, but now the architecture's changing. How are you guys advising your customers? What's your position on when you come into CXO and you give a talk and I said, hey, Gus, the house is on fire, we got so much going on. Bottom line me, what's the architecture? What's best for me, but don't lose the headroom. I need to have some headroom to grow, that's where I see some machine learning, what do I do? >> I think you have to embrace the cloud, and that's one of the key attributes that NetApp brings to the table. We have our core software, our ONTAP software, is in the cloud now. And for us, we want to make sure we make it very easy for our customers to both be in the cloud, be very protected in the cloud with encryption and protection of the data, and also get the scale and all of the benefits of the cloud. But on top of that, we want to make it easy for them to move it wherever they want it to be as well. So for us it's all about the data mobility and the fact that we want to become that data steward, that data engine that helps them drive to where they get the best business value. >> So it's going to be on prem, on cloud. 'Cause I know just for the record, you guys if not the earliest, one of the earliest in with AWS, when it wasn't fashionable. I interviewed you guys on that many years ago. >> And let me ask a related question. What is NetApp's position, or your personal thinking, on what data should be persisted closer to the edge in the new generation of IOT devices? So IOT, edge devices, they do inference, they do actuation and sensing, but they also do persistence. Now should any data be persisted there longterm as part of your overall storage strategy, if you're an enterprise? >> It could be. The question is durability, and what's the impact if for some reason that edge was damaged, destroyed or the data lost. So a lot of times when we start talking about opensource, one of the key attributes we always have to take into account is data durability. And traditionally it's been done through replication. To me that's a very inefficient way to do it, but you have to protect the data. Because it's like if you've got 20 bucks in your wallet, you don't want to lose it, right? You might split it into two 10s, but you still have 20, right? You want that durability and if it has that intrinsic value, you've got to take care of it and be a good steward. So if it's in the edge, it doesn't mean that's the only place it's going to be. It might be in the edge because you need it there. Maybe you need what I call reflexive actions. This is like when a car is well, you have deep learning and machine learning and vision and GPS tracking and all these things there, and how it can stay in the lane and drive, but the sensors themself that are coming from Delphi and Bosch and ZF and all of these companies, they also have to have this capability of being what I call a reflex, right? The reason we can blink and not get a stone in our eye is not because it went to our cerebral cortex. Because it went to the nerve stem and it triggered the blink. >> Yeah, it's cache. And you have to do the same thing in a lot of these environments. So autonomous vehicles is one. It could be using facial recognition for restricting access to a gate. And all the sudden this guy's on a blacklist, and you've stopped the gate. >> Before we get into some of the product questions I have for you, Hadoop in-place analytics, as well as some of the regulations around GDPR, to end the trend segment here is what's your thoughts on decentralization? You see a lot of decentralized apps coming out, you see blockchain getting a lot of traction. Obviously that's a tell sign, certainly in the headroom category of what may be coming down. Not really on the agenda for most enterprises today, but it does kind of indicate that the wave is coming for a lot more decentralization on top of distributed computing and storage. So how do you look at that, as someone who's out on the cutting edge? >> For me it's just yet another industry trend where you have to embrace it. I'm constantly astonished at the people who are trying to push back from things that are coming. To think that they're going to stop the train that's going to run 'em over. And the key is how can we make even those trends better, more reliable, and do the right thing for them? Because if we're the trusted advisor for our customers, regardless of whether or not I'm going to sell a lot of storage to them, I'm going to be the person they're going to trust to give 'em good advice as things change, 'cause that's the one thing that's absolutely coming is change. And oftentimes when you lock yourself into these quote, commodity approaches with a lot of internal storage and a lot of these things, the counterpart to that is that you've also locked yourself in probably for two to four years now, in a technology that you can't be agile with. And this is one of the key attributes for the in-place analytics that we do with our ONTAP product and we also have our E series product that's been around for six plus years in the space, is the defacto performance leader in the space, even. And by decoupling that storage, in some cases very little but it's still connected to the data node, and in other cases where it's shared like an NFS share, that decoupling has enormous benefits from an agility perspective. And that's the key. >> That kind of ties up with the blockchain thing as kind of a tell sign, but you mentioned the in-place analytics. That decoupling gives you a lot more cohesiveness, if you will, in each area. But tying 'em together's critical. How do you guys do that? What's the key feature? Because that's compelling for someone, they want agility. Certainly DevOps' infrastructure code, that's going mainstream, you're seeing that now. That's clearly cloud operation, whatever you want to call it, on prem, off prem. Cloud ops is here. This is a key part of it, what's the unique features of why that works so well? >> Well, some of the unique features we have, so if we look at your portfolio products, so I'll stick with the ONTAP product. One of the key things we have there is the ability to have incredible speed with our AFF product, but we can also Dedoop it, we can clone it, and snapshot it, snapshotting it into, for example, NPS or NetApp Private Storage, which is in Equinox. And now all the sudden I can now choose to go to Amazon, or I can go to Azure, I can go to Google, I can go to SoftLayer. It gives me options as a customer to use whoever has got the best computational engine. Versus I'm stuck there. I can now do what's right for my business. And I also have a DR strategy that's quite elegant. But there's one really unique attribute too, and that's the cloning. So a lot of my big customers have 1000 plus node traditional Hadoop clusters, but it's nearly impossible for them to set up a test DEV environment with production data without having an enormous cost. But if I put it in my ONTAP, I can clone that. I can make hundreds of clones very efficiently. >> That gets the cost of ownership down, but more importantly gets the speed to getting Sandboxes up and running. >> And the Sandboxes are using true production data so that you don't have to worry about oh, I didn't have it in my test set, and now I have a bug. >> A lot of guys are losing budget because they just can't prove it and they can't get it working, it's too clunky. All right, cool, I want to get one more thing in before we run out of time. The role of machine learning we talked about, that's super important. Algorithms are going to be here, it's going to be a big part of it, but as you look at that policy, where the foundational policy governance thing is huge. So you're seeing GDPR, I want to get your comments on the impact of GDPR. But in addition to GDPR, there's going to be another Equifax coming, they're out there, right? It's inevitable. So as someone who's got code out there, writing algorithms, using machine learning, I don't want to rewrite my code based upon some new policy that might come in tomorrow. So GDPR is one we're seeing that you guys are heavily involved in. But there might be another policy I might want to change, but I don't want to rewrite my software. How should a CXO think about that dynamic? Not rewriting code if a new governance policy comes in, and then the GDPR's obvious. >> I don't think you can be so rigid to say that you don't want to rewrite code, but you want to build on what you have. So how can I expand what I already have as a product, let's say, to accommodate these changes? Because again, it's one of those trains. You're not going to stop it. So GDPR, again, it's one of these disruptive regulations that's coming out of EMEA. But what we forget is that it has far reaching implications even in the United States. Because of their ability to reach into basically the company's pocket and fine them for violations. >> So what's the impact of the Big Data system on GDPR? >> It can potentially be huge. The key attribute there is you have to start when you're building your data lakes, when you're building these things, you always have to make sure that you're taking into account anonymizing personal identifying information or obfuscating it in some way, but it's like with everything, you're only as strong as your weakest link. And this is again where NetApp plays a really powerful role because in our storage products, we actually can encrypt the data at rest, at wire speed. So it's part of that chain. So you have to make sure that all of the parts are doing that because if you have data at rest in a drive, let's say, that's inside your server, it doesn't take a lot to beat the heck out of it and find the data that's in there if it's not encrypted. >> Let me ask you a quick question before we wrap up. So how does NetApp incorporate ML or AI into these kinds of protections that you offer to customers? >> Well for us it's, again, we're only as successful as our customers are, and what NetApp does as a company, we'll just call us the data stewards, that's part of the puzzle, but we have to build a team to be successful. So when I travel around the world, the only reason a customer is successful is because they did it with a team. Nobody does it on an island, nobody does it by themself, although a lot of times they think they can. So it's not just us, it's our server vendors that work with us, it's the other layers that go on top of it, companies like Zaloni or BlueData and BlueTalon, people we've partnered with that are providing solutions to help drive this for our customers. >> Gus, great to have you on theCUBE. Looking forward to next week. I know you're super busy at NetApp InSight. I know you got like five major talks you're doing but if we can get some time I think you'd be great. My final question, a personal one. We were talking that you're a search and rescue in Tahoe in case there's an avalanche, a lost skier. A lot of enterprises feel lost right now. So you kind of come in a lot and the avalanche is coming, the waves or whatever are coming, so you probably seen situations. You don't need to name names, but talk about what should someone do if they're lost? You come in, you can do a lot of consulting. What's the best advice you could give someone? A lot of CXOs and CEOs, their heads are spinning right now. There's so much on the table, so much to do, they got to prioritize. >> It's a great question. And here's the one thing is don't try to boil the ocean. You got to be hyper-focused. If you're not seeing a return on investment within 90 days of setting up your data lake, something's going wrong. Either the scope of what you're trying to do is too large, or you haven't identified the use case that will give you an immediate ROI. There should be no hesitation to going down this path, but you got to do it in a manner where you're tackling the biggest problems that have the best hit value for you. Whether it's ETLing goes into your plan of record systems, your enterprise data warehouses, you got to get started, but you want to make sure you have measurable, tangible success within 90 days. And if you don't, you have to reset and say okay, why is that not happening? Am I reinventing the wheel because my consultant said I have to write all this SCOOP and Flume code and get the data in? Or maybe I should have chosen another company to be a partner that's done this 1000 times. And it's not a science experiment. We got to move away from science experiment to solving business problems. >> Well science experiments and boiling of the ocean is don't try to overreach, build a foundational building block. >> The successful guys are the ones who are very disciplined and they want to see results. >> Some call it baby steps, some call it building blocks, but ultimately the foundation right now is critical. >> Gus: Yeah. >> All right, Gus, thanks for coming on theCUBE. Great day, great to chat with you. Great conversation about machine learning impact to organizations. theCUBE bringing you the data here live in Manhattan. I'm John Furrier, Jim Kobielus with Wikibon. More after this short break. We'll be right back. (digital music) (synthesizer music)

>> Live from Midtown Manhattan, it's the CUBE. Covering Big Data New York City 2017. Brought to you by SiliconANGLE Media and its ecosystem sponsors. >> Fifth year of coverage of our own event Big Data NYC where we cover all the action in New York City. For this week in big data, in conjunction with Strata Data which was originally Hadoop World in 2010. We've been covering it for eight years. It became Strata Conference, Strata Hadoop, now called Strata Data. Will probably called Strata AI tomorrow. Who knows, but certainly the trends are going in that direction. I'm John Furrier, your co-host. Our next guest here in New York City is Sergei Rabotai, who is the Head of Business Development at InData Labs from Belarus. In town, doing some biz dev in the big data ecosystem. Welcome to theCUBE. >> Yeah. Good morning. >> Great to have you. So, obviously Belarus is becoming known as the Silicon Valley of Eastern Europe. A lot of great talent. We're seeing that really explode. A lot of great stuff going on globally, even though there's a lot of stuff, you know GDPR and all these other things happening. It's clearly a global economy with tech. Silicon Valley still is magical. I live there in Palo Alto but you're starting to see peering points within these ecosystems of entrepreneurship and now big companies are taking advantage of it as well. What do you guys do? I mean you're in the middle of that. What is InData Labs do in context of all this? >> Well, InData Labs is a full stack data science company. Which means that we provide professional services for data strategy, big data engineering and the data science. So, yeah, like you just said, we are based - my team is based in Minsk, Belarus. We are about 40 people strong at the moment. And in our recent years we have been very successful starting this business and we have been getting customers from all over the world, including United States, Great Britain, and European Union. The company was launched about four years ago and very important thing, that it was launched by two tech leaders who come from very data-driven industries. Our CEO, Ilya Kirillov, has been running several EdTech companies for many years. Our second founder, Marat Karpeko, has been holding C-Level positions in one of the most successful gaming companies in the world. >> John: So they know data. They're data guys. >> Yeah they're data guys. They know data from different aspects and that brings synergy to our business. >> You guys bring that expertise now into professional services for us. Give me an example of some of the things someone might want to call you up on, because the thing we're hearing here in New York City this week is look, we need more data sciences and they got to be more productive. They're spending way too much time wrangling and doing stuff that they shouldn't be doing. In the old days, sysadmins were built to let people be productive and they ran the infrastructure. That's not what data scientists should be doing. They're the users. There's a level of setting things up and then there's a level of provisioning, it's actually data assets, but then the data scientists just want to do their job. How do you help companies do that? >> Well I would probably, if I take all of our activities, I would split them into two big parts. First of all, we are helping big companies, who already have a lot of data. We help them in managing this data more effectively. We help them with predictive analytics. We help them with, helping them build the churn prediction and user segmentation solutions. We have been recently involved into several natural language processing projects. In one of our successful key studies we helped one of the largest gaming companies to automate their customer feedback processing. So, like, a couple years ago they were working manually with their customer feedback and we built them a tool that allows them to instantly get the sentiment of what the user says. It's kind of like a voice of a customer, which means they can be more effective in developing new things for their games. So, we-- >> So what would someone engage? I'm just trying to peg a order of magnitude of the levels of engagements you do. Startups come in? Is it big companies? What kind of size scoped work do you do? >> So I would say at the moment we work with startups, but it's a bit of a different approach than we have with big or well-established companies. When startups typically approach us with asking to help them implement some brand new technologies like neural networks or deep learning. So they want to be effective from the start. They want to use the cutting edge technology to be more attractive, to provide a better value on the market and just to be effective and to be a successful business from the start. The other part, the well-established companies, who already have the data but they understand that so far their data might not be used that effectively as it should have been used. Therefore, they approach us with a request to help them to get more insights out of the data. Let's say, implement some machine learning that can help them. >> How about larger companies? What kind of projects do you work for them? >> It could be a typical project like churn prediction, that is very actual for the companies who have got a lot of customer data. Then it could be companies from such industries like betting industry, where churn is a very big issue. And, the same probably applies to companies who do trading. >> So is scale one of the things you differentiate around? It sounds like your founders have an EdTech background obviously must be a larger, large data set. Is your profile of engagements large scale? Is it ... I'm just trying to get a handle of if someone's watching who, what is the kind of engagements people should be calling you for? Give us an example of that. >> Like, let's say there is a company who has got a lot of customer data, has got some products and they have a problem of churn, or they have a problem of segmenting their customers so they can later address the specific segments of the customers with the right offers at the right time and through the right marketing channel. Then it could be customers or requests where natural text processing is required where we have to automate some understanding of the written or spoken text. Then I should say that we have been getting recently some requests where computer vision skills are required. I think the first stage of AI being really intelligent was the speech recognition and I think nowadays we manage to reach to the level of what we earlier saw in fantastic movies or sci-fi movies. Computer vision is going to be the next leap in all that AI buzz we're having at the moment. >> So you solve, the problem that you solve for customers is data problems. If they're swimming in a lot of data, you can help them. >> Sergei: Yep. >> If they actually want to make that data do things that are cutting edge, you guys can help them. >> Sergei: Yeah. That's-- >> Alright, so here's a question for you. I mean, Belarus has obviously got good things going on. I've heard the press that you guys have been getting, the whole area, and you guys in particular. So I'm a buyer, one of the questions I might ask is "Hey Sergei, how do I know that you'll keep that talent because the churn is always a big problem. I've dealt with outsourcing before and in the US it's hard to keep talent but I've heard there's a churn." How do you guys keep the talent in the country? How do you keep talent on the projects? Is there certain economic rules over there? What's happening in Belarus? Give us the economical. >> Yeah, so, basically what you're saying. The churn problem has always been known for companies who have their development teams in Asian regions. That's a known problem because I have a lot of meetings with clients in the UK and the US, potential prospects, I would say. So they say it is a problem for them. With Belarus, I don't think we have that because from what I know, we have an average churn of under 10 percent. That's the figures across the industry. In smaller companies, the churn is even less and there are specific reasons for that. First of all, that due to Belarusian mentality, we always try to keep to a job that we're having. Yeah? So we do not-- >> John: That's a cultural thing. >> That's just the cultural thing. We do not ... >> You honor, you honor a code, if you will. >> Yeah. >> Okay. >> So, that's one of the things. Another thing is that Belarusian IT industry is very small. We have, I would say, no more than 40 thousand people being involved in different IT companies. The community is very small, so if somebody is hopping jobs from one job to another, it is going to be known and this person is not likely to have like, a good career. >> So job hoppers is kind of like a code of community, honor. Silicon Valley works that way too, by the way. >> Yeah. >> You get identified, that's who you are. >> Yeah. And so nowadays-- >> Economic tax breaks going on over there? What's the government to get involved? >> One of the key things is, the special tax and legal regulations that Belarus has got at the moment. I can definitely say that there is no country in the world that has got the same tax preferences, and the same support from the government. If a Belarusian company, IT company, becomes a part of Belarusian High Tech Park it means the company becomes automatically exempt from BET tax, corporate income tax. The employees of that company having the reliefs on their income, personal income tax rate, and there are a lot more reliefs that make the talent stay in the country. Having this relief for the IT business allows the companies to provide better working conditions for the employees and stop the people from migrating to other parts of the world. That's what we have. >> Sort of created an environment where there's not a lot of migration out of the area. The tech community kind of does it's own policing of behavior for innovation. >> Yeah but I think before those initiatives were adopted there was a certain percentage of people migrating but I think that nowadays even if it happens, yes, you're right, it's not that substantial. >> Great. Tell us ... Great overview of the company and congratulations, it's a good opportunity for folks watching to explore new areas of talent, especially ones that have the work ethic and knowledge you guys have over there. New York here, there's codes here too. Get the job done. Be on time. What's your experience like in New York here? What's your goal this week? What's some of the meetings you're having? Share with the folks kind of your game plan for Big Data NYC. >> Well, yeah, I've really enjoyed my stay here. It, so far, has been a very enjoyable experience. From the business perspective, I had over 10 meetings with the prospective customers. And we are likely to have follow-ups coming in the next couple of weeks. I can definitely say there is a great demand for professional services. You can see that if you go to whichever center you can see there's a lot of jobs being posted on the job boards. It means that there is lack of knowledge here in the US, yeah? One more important thing that I wanted to share with you from my personal observations that USA, UK and maybe Nordic countries, they have very, very strong background for creating the business ideas but Eastern Europe or Eastern European countries and Belarus in particular, they are very strong in actually implementing those ideas. >> Building them. >> Yes, building them. I think we have lots of synergies and we can ... we can ... >> John: Great. >> We can work together. I also got some meetings with our existing customers here in the US and so far we had good experiences. I can see that New York is moving fast. I travel a lot. I've been to over 40 countries in the previous five years and I just ... New York is different. >> It's fun. >> Different. Even different from many other cities in the US. >> Lot of banks are here. Lot of business in New York. New York is a great town. Love New York City. It's one of my favorites. Love coming here as I grew up right across the river in New Jersey. >> Yeah. But, great town, obviously California, Palo Alto, >> Yeah. >> Is a little more softer in terms of weather, but they have a culture there too. Sounds a lot like what's going on in Belarus, so congratulations. If we get some business for you, should we give them theCUBE discount, tell them John sent you and you get 10 percent off? Alright? >> Alright, yes. Sounds great. We can make it a good deal. (laughter) >> Tell them John sent you, you get 10% off. No I'm only kidding because it's services. Congratulations. Final question. What's the number one thing that people are buying for service from you guys? Number one thing. What's the most requested service you provide? >> The most requested services ... First of all, many customers they understand that they have got a lot of data. They want to do something with their data. But before you actually do some implementation you have to do a lot of discovery or preparatory work. I would say, no matter how we end up with a customer, this stage is basically ... The idea of that stage is to identify the ways data science can be implemented and can provide benefits to the business. That's the most important. I think that, like, 95 percent of the customers they approach us with this thing in the first place. And based on the results of that preparatory stage we can then advise the customers. What can they do? Or how they can actually benefit from the existing data? Or what other things they should collect in order to make their business more effective. >> Sergei, thanks for coming on. Belarus has got a lot of builders there. Check 'em out. >> Thanks a lot. >> Builders are critical in this new world. Lots of them with clout, a lot of great opportunities. A lot of builders in Belarus. This is theCUBE, bringing you all the action from New York City. More after this short break. We'll be right back. (theme music) (no audio) >> Hi, I'm John Furrier, the co-founder of SiliconANGLE Media and co-host of theCUBE. I've been in the tech ...