>> Narrator: From San Jose, California, in the heart of Silicon Valley, it's theCUBE. Covering Hadoop Summit 2016. Brought to you by Hortonworks. Here's your host, John Furrier. >> Welcome back everyone. We are here live in Silicon Valley at Hadoop Summit 2016, actually San Jose. This is theCUBE, our flagship program. We go out to the events and extract the signal to the noise. Our next guest, David Richards, CEO of WANdisco. And Joel Horowitz, strategy and business development, IBM analyst. Guys, welcome back to theCUBE. Good to see you guys. >> Thank you for having us. >> It's great to be here, John. >> Give us the update on WANdisco. What's the relationship with IBM and WANdisco? 'Cause, you know. I can just almost see it, but I'm not going to predict. Just tell us. >> Okay, so, I think the last time we were on theCUBE, I was sitting with Re-ti-co who works very closely with Joe. And we began to talk about how our partnership was evolving. And of course, we were negotiating an OEM deal back then, so we really couldn't talk about it very much. But this week, I'm delighted to say that we announced, I think it's called IBM Big Replicate? >> Joel: Big Replicate, yeah. We have a big everything and Replicate's the latest edition. >> So it's going really well. It's OEM'd into IBM's analytics, big data products, and cloud products. >> Yeah, I'm smiling and smirking because we've had so many conversations, David, on theCUBE with you on and following your business through the bumpy road or the wild seas of big data. And it's been a really interesting tossing and turning of the industry. I mean, Joel, we've talked about it too. The innovation around Hadoop and then the massive slowdown and realization that cloud is now on top of it. The consumerization of the enterprise created a little shift in the value proposition, and then a massive rush to build enterprise grade, right? And you guys had that enterprise grade piece of it. IBM, certainly you're enterprise grade. You have enterprise everywhere. But the ecosystem had to evolve really fast. What happened? Share with the audience this shift. >> So, it's classic product adoption lifecycle and the buying audience has changed over that time continuum. In the very early days when we first started talking more at these events, when we were talking about Hadoop, we all really cared about whether it was Pig and Hive. >> You once had a distribution. That's a throwback. Today's Thursday, we'll do that tomorrow. >> And the buying audience has changed, and consequently, the companies involved in the ecosystem have changed. So where we once used to really care about all of those different components, we don't really care about the machinations below the application layer anymore. Some people do, yes, but by and large, we don't. And that's why cloud for example is so successful because you press a button, and it's there. And that, I think, is where the market is going to very, very quickly. So, it makes perfect sense for a company like WANdisco who've got 20, 30, 40, 50 sales people to move to a company like IBM that have 4 or 5,000 people selling our analytics products. >> Yeah, and so this is an OEM deal. Let's just get that news on the table. So, you're an OEM. IBM's going to OEM their product and brand it IBM, Big Replication? >> Yeah, it's part of our Big Insights Portfolio. We've done a great job at growing this product line over the last few years, with last year talking about how we decoupled all the value-as from the core distribution. So I'm happy to say that we're both part of the ODPI. It's an ODPI-certified distribution. That is Hadoop that we offer today for free. But then we've been adding not just in terms of the data management capabilities, but the partnership here that we're announcing with WANdisco and how we branded it as Big Replicate is squarely aimed at the data management market today. But where we're headed, as David points out, is really much bigger, right? We're talking about support for not only distributed storage and data, but we're also talking about a hybrid offering that will get you to the cloud faster. So not only does Big Replicate work with HDFS, it also works with the Swift objects store, which as you know, kind of the underlying storage for our cloud offering. So what we're hoping to see from this great partnership is as you see around you, Hadoop is a great market. But there's a lot more here when you talk about managing data that you need to consider. And I think hybrid is becoming a lot larger of a story than simply distributing your processing and your storage. It's becoming a lot more about okay, how do you offset different regions? How do you think through that there are multiple, I think there's this idea that there's one Hadoop cluster in an enterprise. I think that's factually wrong. I think what we're observing is that there's actually people who are spinning up, you know, multiple Hadoop distributions at the line of business for maybe a campaign or for maybe doing fraud detection, or maybe doing log file, whatever. And managing all those clusters, and they'll have Cloud Arrow. They'll have Hortonworks. They'll have IBM. They'll have all of these different distributions that they're having to deal with. And what we're offering is sanity. It's like give me sanity for how I can actually replicate that data. >> I love the name Big Replicate, fantastic. Big Insights, Big Replicate. And so go to market, you guys are going to have bigger sales force. It's a nice pop for you guys. I mean, it's good deal. >> We were just talking before we came on air about sort of a deal flow coming through. It's coming through, this potential deal flow coming through, which has been off the charts. I mean, obviously when you turn on the tap, and then suddenly you enable thousands and thousands of sales people to start selling your products. I mean, IBM, are doing a great job. And I think IBM are in a unique position where they own both cloud and on-prem. There are very few companies that own both the on-prem-- >> They're going to need to have that connection for the companies that are going hybrid. So hybrid cloud becomes interesting right now. >> Well, actually, it's, there's a theory that says okay, so, and we were just discussing this, the value of data lies in analytics, not in the data itself. It lies in you've been able to pull out information from that data. Most CIOs-- >> If you can get the data. >> If you can get the data. Let's assume that you've got the data. So then it becomes a question of, >> That's a big assumption. Yes, it is. (laughs) I just had Nancy Handling on about metadata. No, that's an issue. People have data they store they can't do anything with it. >> Exactly. And that's part of the problem because what you actually have to have is CPU slash processing power for an unknown amount of data any one moment in time. Now, that sounds like an elastic use case, and you can't do elastic on-prem. You can only do elastic in cloud. That means that virtually every distribution will have to be a hybrid distribution. IBM realized this years ago and began to build this hybrid infrastructure. We're going to help them to move data, completely consistent data, between on-prem and cloud, so when you query things in the cloud, it's exactly the same results and the correct results you get. >> And also the stability too on that. There's so many potential, as we've discussed in the past, that sounds simple and logical. To do an enterprise grade is pretty complex. And so it just gives a nice, stable enterprise grade component. >> I mean, the volumes of data that we're talking about here are just off the charts. >> Give me a use case of a customer that you guys are working with, or has there been any go-to-market activity or an ideal scenario that you guys see as a use case for this partnership? >> We're already seeing a whole bunch of things come through. >> What's the number one pattern that bubbles up to the top? Use case-wise. >> As Joel pointed out, that he doesn't believe that any one company just has one version of Hadoop behind their firewall. They have multiple vendors. >> 100% agree with that. >> So how do you create one, single cluster from all of those? >> John: That's one problem you solved. >> That's of course a very large problem. Second problem that we're seeing in spades is I have to move data to cloud to run analytics applications against it. That's huge. That required completely guaranteed consistent data between on-prem and cloud. And I think those two use cases alone account for pretty much every single company. >> I think there's even a third here. I think the third is actually, I think frankly there's a lot of inefficiencies in managing just HDFS and how many times you have to actually copy data. If I looked across, I think the standard right now is having like three copies. And actually, working with Big Replicate and WANdisco, you can actually have more assurances and actually have to make less copies across the cluster and actually across multiple clusters. If you think about that, you have three copies of the data sitting in this cluster. Likely, an analysts have a dragged a bunch of the same data in other clusters, so that's another multiple of three. So there's amount of waste in terms of the same data living across your enterprise. That I think there's a huge cost-savings component to this as well. >> Does this involve anything with Project Atlas at all? You guys are working with, >> Not yet, no. >> That project? It's interesting. We're seeing a lot of opening up the data, but all they're doing is creating versions of it. And so then it becomes version control of the data. You see a master or a centralization of data? Actually, not centralize, pull all the data in one spot, but why replicate it? Do you see that going on? I guess I'm not following the trend here. I can't see the mega trend going on. >> It's cloud. >> What's the big trend? >> The big trend is I need an elastic infrastructure. I can't build an elastic infrastructure on-premise. It doesn't make economic sense to build massive redundancy maybe three or four times the infrastructure I need on premise when I'm only going to use it maybe 10, 20% of the time. So the mega trend is cloud provides me with a completely economic, elastic infrastructure. In order to take advantage of that, I have to be able to move data, transactional data, data that changes all the time, into that cloud infrastructure and query it. That's the mega trend. It's as simple as that. >> So moving data around at the right time? >> And that's transaction. Anybody can say okay, press pause. Move the data, press play. >> So if I understand this correctly, and just, sorry, I'm a little slow. End of the day today. So instead of staging the data, you're moving data via the analytics engines. Is that what you're getting at? >> You use data that's being transformed. >> I think you're accessing data differently. I think today with Hadoop, you're accessing it maybe through like Flume or through Oozy, where you're building all these data pipelines that you have to manage. And I think that's obnoxious. I think really what you want is to use something like Apache Spark. Obviously, we've made a large investment in that earlier, actually, last year. To me, what I think I'm seeing is people who have very specific use cases. So, they want to do analysis for a particular campaign, and so they may just pull a bunch of data into memory from across their data environment. And that may be on the cloud. It may be from a third-party. It may be from a transactional system. It may be from anywhere. And that may be done in Hadoop. It may not, frankly. >> Yeah, this is the great point, and again, one of the themes on the show is, this is a question that's kind of been talked about in the hallways. And I'd love to hear your thoughts on this. Is there are some people saying that there's really no traction for Hadoop in the cloud. And that customers are saying, you know, it's not about just Hadoop in the cloud. I'm going to put in S3 or object store. >> You're right. I think-- >> Yeah, I'm right as in what? >> Every single-- >> There's no traction for Hadoop in the cloud? >> I'll tell you what customers tell us. Customers look at what they actually need from storage, and they compare whatever it is, Hadoop or any on-premise proprietor storage array and then look at what S3 and Swift and so on offer to them. And if you do a side-by-side comparison, there isn't really a difference between those two things. So I would argue that it's a fact that functionally, storage in cloud gives you all the functionality that any customer would need. And therefore, the relevance of Hadoop in cloud probably isn't there. >> I would add to that. So it really depends on how you define Hadoop. If you define Hadoop by the storage layer, then I would say for sure. Like HDFS versus an objects store, that's going to be a difficult one to find some sort of benefit there. But if you look at Hadoop, like I was talking to my friend Blake from Netflix, and I was asking him so I hear you guys are kind of like replatforming on Spark now. And he was basically telling me, well, sort of. I mean, they've invested a lot in Pig and Hive. So if you think it now about Hadoop as this broader ecosystem which you brought up Atlas, we talk about Ranger and Knox and all the stuff that keeps coming out, there's a lot of people who are still invested in the peripheral ecosystem around Hadoop as that central point. My argument would be that I think there's still going to be a place for distributed computing kind of projects. And now whether those will continue to interface through Yarn via and then down to HDFS, or whether that'll be Yarn on say an objects store or something and those projects will persist on their own. To me that's kind of more of how I think about the larger discussion around Hadoop. I think people have made a lot of investments in terms of that ecosystem around Hadoop, and that's something that they're going to have to think through. >> Yeah. And Hadoop wasn't really designed for cloud. It was designed for commodity servers, deployment with ease and at low cost. It wasn't designed for cloud-based applications. Storage in cloud was designed for storage in cloud. Right, that's with S3. That's what Swift and so on were designed specifically to do, and they fulfill most of those functions. But Joel's right, there will be companies that continue to use-- >> What's my whole argument? My whole argument is that why would you want to use Hadoop in the cloud when you can just do that? >> Correct. >> There's object store out. There's plenty of great storage opportunities in the cloud. They're mostly shoe-horning Hadoop, and I think that's, anyway. >> There are two classes of customers. There were customers that were born in the cloud, and they're not going to suddenly say, oh you know what, we need to build our own server infrastructure behind our own firewall 'cause they were born in the cloud. >> I'm going to ask you guys this question. You can choose to answer or not. Joel may not want to answer it 'cause he's from IBM and gets his wrist slapped. This is a question I got on DM. Hadoop ecosystem consolidation question. People are mailing in the questions. Now, keep sending me your questions if you don't want your name on it. Hold on, Hadoop system ecosystem. When will this start to happen? What is holding back the M and A? >> So, that's a great question. First of all, consolidation happens when you sort of reach that tipping point or leveling off, that inflection point where the market levels off, and we've reached market saturation. So there's no more market to go after. And the big guys like IBM and so on come in-- >> Or there was never a market to begin with. (laughs) >> I don't think that's the case, but yes, I see the point. Now, what's stopping that from happening today, and you're a naughty boy by the way for asking this question, is a lot of these companies are still very well funded. So while they still have cash on the balance sheet, of course, it's very, very hard for that to take place. >> You picked up my next question. But that's a good point. The VCs held back in 2009 after the crash of 2008. Sequoia's memo, you know, the good times role, or RIP good times. They stopped funding companies. Companies are getting funded, continually getting funding. Joel. >> So I don't think you can look at this market as like an isolated market like there's the Hadoop market and then there's a Spark market. And then even there's like an AI or cognitive market. I actually think this is all the same market. Machine learning would not be possible if you didn't have Hadoop, right? I wouldn't say it. It wouldn't have a resurgence that it has had. Mahout was one of the first machine learning languages that caught fire from Ted Dunning and others. And that kind of brought it back to life. And then Spark, I mean if you talk to-- >> John: I wouldn't say it creates it. Incubated. >> Incubated, right. >> And created that Renaissance-like experience. >> Yeah, deep learning, Some of those machine learning algorithms require you to have a distributed kind of framework to work in. And so I would argue that it's less of a consolidation, but it's more of an evolution of people going okay, there's distributed computing. Do I need to do that on-premise in this Hadoop ecosystem, or can I do that in the cloud, or in a growing Spark ecosystem? But I would argue there's other things happening. >> I would agree with you. I love both areas. My snarky comment there was never a market to begin with, what I'm saying there is that the monetization of commanding the hill that everyone's fighting for was just one of many hills in a bigger field of hills. And so, you could be in a cul-de-sac of being your own champion of no paying customers. >> What you have-- >> John: Or a free open-source product. >> Unlike the dotcom era where most of those companies were in the public markets, and you could actually see proper valuations, most of the companies, the unicorns now, most are not public. So the valuations are really difficult to, and the valuation metrics are hard to come by. There are only few of those companies that are in the public market. >> The cash story's right on. I think to Joel' point, it's easy to pivot in a market that's big and growing. Just 'cause you're in the wrong corner of the market pivoting or vectoring into the value is easier now than it was 10 years ago. Because, one, if you have a unicorn situation, you have cash on the bank. So they have a good flush cash. Your runway's so far out, you can still do your thing. If you're a startup, you can get time to value pretty quickly with the cloud. So again, I still think it's very healthy. In my opinion, I kind of think you guys have good analysis on that point. >> I think we're going to see some really cool stuff happen working together, and especially from what I'm seeing from IBM, in the fact that in the IT crowd, there is a behavioral change that's happening that Hadoop opened the door to. That we're starting to see more and more It professionals walk through. In the sense that, Hadoop has opened the door to not thinking of data as a liability, but actually thinking about data differently as an asset. And I think this is where this market does have an opportunity to continue to grow as long as we don't get carried away with trying to solve all of the old problems that we solved for on-premise data management. Like if we do that, then we're just, then there will be a consolidation. >> Metadata is a huge issue. I think that's going to be a big deal. And on the M and A, my feeling on the M and A is that, you got to buy something of value, so you either have revenue, which means customers, and or initial property. So, in a market of open source, it comes back down to the valuation question. If you're IBM or Oracle or HP, they can pivot too. And they can be agile. Now slower agile, but you know, they can literally throw some engineers at it. So if there's no customers in I and P, they can replicate, >> Exactly. >> That product. >> And we're seeing IBM do that. >> They don't know what they're buying. My whole point is if there's nothing to buy. >> I think it depends on, ultimately it depends on where we see people deriving value, and clearly in WANdisco, there's a huge amount of value that we're seeing our customers derive. So I think it comes down to that, and there is a lot of IP there, and there's a lot of IP in a lot of these companies. I think it's just a matter of widening their view, and I think WANdisco is probably the earliest to do this frankly. Was to recognize that for them to succeed, it couldn't just be about Hadoop. It actually had to expand to talk about cloud and talk about other data environments, right? >> Well, congratulations on the OEM deal. IBM, great name, Big Replicate. Love it, fantastic name. >> We're excited. >> It's a great product, and we've been following you guys for a long time, David. Great product, great energy. So I'm sure there's going to be a lot more deals coming on your. Good strategy is OEM strategy thing, huh? >> Oh yeah. >> It reduces sales cost. >> Gives us tremendous operational leverage. Getting 4,000, 5,000-- >> You get a great partner in IBM. They know the enterprise, great stuff. This is theCUBE bringing all the action here at Hadoop. IBM OEM deal with WANdisco all happening right here on theCUBE. Be back with more live coverage after this short break.

Zay California in the heart of Silicon Valley. It's the cube covering Hadoop summit 2016 brought to you by Hortonworks. Here's your host, John furrier. >>Okay, we'll come back here and we are here live in Silicon Valley for the cube. This is our flagship program. We go out to the events and extract the cylinders. Of course. We're here at the big data event. Hadoop summit 2016 have a special guest celebrity now, author of the bestselling book magical at Virginia Heffernan magic and loss rising on the bestseller lists. Welcome to the cube. Thanks in our show, you are my internet friend and now you're my real life friend. You're my favorite Facebook friend that I just now met for the first time. Great to meet you. We had never met and now we, but we know each other of course intimately through the interwebs. So I've been following your writing your time. Send you do some stuff on medium and then you, you kind of advertise. You're doing this book. I saw you do the Google glasses experiment in. >>It was Brooklyn and it might, it was so into Google glass and I will admit it, I fought for everything. I fell for VR and all its incarnations and um, and the Google last year, it was like that thing that was supposed to put the internet all voice activated, just put the internet always in front of your face. So I started to wear it around in Brooklyn, my prototype. I thought everyone would stop me and say how cool it was. In fact they didn't think it was pull it off new Yorkers. That's how you would, how they really feel. Got a problem with that. Um, your book magic and loss is fantastic and I think it really is good because uh, Dan Lyons wrote, disrupted, loved, which was fantastic. Dan lies big fan of him and his work, but it really, it wasn't a parody of civil rights for Silicon Valley. >>The show that's kinda taken that culture and made it mainstream. I had people call me up and say, Hey, you live in Callow Alto. My God, do you live near the house? Something like it's on Newell, which is one of my cross streets. But the point is tech culture now is kind of in a native, my youngest is 13 and you know, we're in an iPad generation for the youth and we're from the generation where there was no cell phones. And Mike, I remember when pages were the big innovation and internet. But I think, I think when I'm telling you, I think, I know I'm talking to a fellow traveler when I say that there was digital culture before the advent of the worldwide web in the early nineties you know, I, I'm sure you did too. Got electronic games like crazy. I would get any Merlin or Simon or whatever that they, they introduced. >>And then I also dialed into a mainframe in the late seventies and the early eighties to play the computer as we call it. We didn't even call it the internet. And the thing about the culture too was email was very, you know, monochrome screens, but again, clunky but still connected. Right? So we were that generation of, you know, putting that first training wheels on and now exposed to you. So in the book, your premise is, um, there's magical things happening in the internet and art countering the whole trolling. Uh, yeah, the Internet's bad. And we know recently someone asked me, how can the internet be art when Twitter is so angry? What do you think art is? You know, this is an art. Art is emotional. Artists know powerful >>emotions represented in tranquility and this is, you know, what you see on the internet all the time. Of course the aid of course are human. It needs a place to live and call it Twitter. For now it used to be YouTube comments. So, but we are always taking the measure of something we've lost. Um, I get the word loss from lossy compression, you know, the engineering term that, how does, how MP3 takes that big broad music signal and flattens it out. And something about listening to music on MP3, at least for me, made me feel a sense that I was grieving for something. It was missing something from my analog life. On the other hand, more than counterbalanced by the magic that I think we all experienced on the internet. We wouldn't have a friendship if it weren't for social media and all kinds of other things. And strange serendipity happens not to mention artistic expression in the form of photography, film, design of poetry and music, which are the five chapters of the book. >>So the book is fantastic. The convergence and connection of people, concepts, life with the internet digitally is interesting, right? So there's some laws with the MP3. Great example, but have you found post book new examples? I'm sure the internet culture, geese like Mia, like wow, this is so awesome. There's a cultural aspect of it is the digital experience and we see it on dating sites. Obviously you see, you know Snapchat, you know, dating sites like Tinder and other hookups apps and the real estate, everything being Uberized. What's the new things that you've, that's coming out and you must have some >>well this may be controversial, but one thing I see happening is anti digital culture. Partly as an epi phenomenon of side effect of digitization. We have a whole world of people who really want to immerse themselves in things like live music maker culture, things made by hand, vinyl records, vinyl records, which are selling more than ever in the days of the rolling stones. Gimme shelter less they sold less than than they do now. The rolling stones makes $1 billion touring a year. Would we ever have thought that in the, in the, you know, at the Genesis of the iPod when it seemed like, you know, recorded music represented music in that MP3 thing that floated through our, our phones was all we needed. No, we want to look in the faces of the rolling stones, get as close as we can to the way the music is actually made and you know, almost defiantly, and this is how the culture works. This is how youth culture works. Um, reject, create experiences that cannot be digitized. >>This is really more of a counter culture movement on the overt saturation of digital. >>Yes. Yes. You see the first people to scale down from, you know, high powered iPhones, um, when we're youth going to flip phones. You know, it's like the greatest like greatest punk, punk, punk tech. Exactly. It's like, yeah, I'm going to use these instruments, but like if I break a string, who cares on a PDs? The simplest one, right? >>My mom made me use my iPhone. Are we going to, how are we going to have that? it'd >>be like, Oh, look at you with your basic iPhone over there. And I've got my just like hack down, downscale, whatever. And you know what, I don't spend the weekends, don't pick up my phone on the weekends. But you know, there are interesting markets there. And interesting. I mean, for instance, the, you know, the live phenomenon, I know that, you know, there's this new company by one of the founders of Netflix movie pass, which um, for a $30 subscription you've seen movies in the theater as much as you want and the theaters are beautiful. And what instead of Netflix and chill, you know, the, the, the contemporary, you know, standard date, it's dinner and movie. You're out again. You're eating food, which can't be digitized with in-company, which can't be digitized. And then sitting in a theater, you know, a public experience, which is, um, a pretty extraordinary way that the culture and business pushes back on digital. >>Remember I was a comma on my undergraduate days in computer science in the 80s. And before when it was nerdy and eh, and there was a sociology class at Hubba computers and social change. And the big thing was we're going to lose social interactions because of email. And if you think about what you're talking about here is that the face to face presence, commitment of being with somebody right now is a scarce resource. You have an abundance of connections. >>I mean, take the fact what has happened is digital culture has jacked up the value of undigital culture. So for instance, you know, I've, I've met on Facebook, we talk on Facebook messenger, we notice that we're, you know, like kindred spirits in a certain way and we like each other's posts and so forth. Then we have an, a more extensive talk in messenger when we meet in person for the first time. Both of us are East coast people, but we hugged tele because it's like, Oh wow, like you in the flesh. You know it's something exciting. >>Connection virtually. That's right. There's a synchronous connection presence, but we're not really, we haven't met face to face. >>Yeah, there's this great as a great little experiment going on, set group of kids and Silicon Valley have decided they're too addicted to their phones and Facebook. Now I am not recommending for your viewers and listeners that anybody do what these kids sounds good, are ready. Go. Hey, all right, so what they do is take an LSD breakfast. Now I don't take drugs. I think you can do this without the LSD, but they put a little bit of a hallucinogen under their skin in the morning and what they find is they lost interest in the boring interface in their phones because people on the bus suddenly looked so fascinating to them. The human face is an ratable interface. It can't be reproduced anywhere, Steve. You know, Johnny ive can't make it. They can't make it at Google. And that I think is something we will see young markets doing, which is this renewed appreciation for nature and analog for humans and for analog culture. >>That's right. The Navy is going to sextants and compasses. You may have seen training, they're training sailors on those devices because of the fear that GPS might be hacked. So you know, the young kids probably don't even know what a cup is is, well, I bought myself a compass recently because I suddenly was like, you know, we talk a lot about digital technology, but what the heck, this thing you can point toward the poles, right in my hands. You know, I was suddenly like, we are this floating ball with these poles with different magnetic charges. And I think it's time. I appreciate it. >>Okay, so I've got to ask the, um, the, the feedback that you've gotten from the book, um, again, we hear that every Geneva magic and loss, great, great book. Go by. It's fantastic and open your mind up. It's a, it's a thought provoking, but really specific good use cases. I got a think that, you know, when you talk at Google and when you talk to some of the groups that you're talking to, certainly book clubs and other online that there must be like, Oh my God, you hit the cultural nerve. What have you heard from some of these, um, folks from my age 50 down to the 20 something year olds? Have you had any aha moments where you said, Oh my God, I hit a nerve here. >>Did not want to, I mean, I didn't want to write one of those books. That's like the one thing you need to know to get your startup to succeed or whatever. You know, I was at the airport and every single one of them is like, pop the only thing you need to do to save this or whatever. And they, they do take a very short view. Now if you're thinking about, you know, whether if you're thinking about your quarterly return or your, you know, what you're going to do this quarter and when you're going to be profitable or user acquisition, those books are good manuals. But if you're going to buy a hardcover book and you're going to really invest in reading every page, not just the bolded part, not just the put, you know, the two points that you have to know. I really wanted readers and at what I had found on the internet, people like you, we have an interest in a long view. You know what, I need a really long view >>in a prose that's not for listicle or you know, shorts. It's like it's just a thought provoker but somebody can go, Hey, you know, at the beach on the weekend say, Hey wow, this is really cool. What F you know, we went analog for awhile or what if, what's best for my kids to let my kids play multiplayer games more Zika simulate life. That was my, so these are the kinds of questions that the digital parents are asked. >>Yeah. So you know, like let's take the parents question, which is, is, you know, a, surprisingly to me it's a surprisingly pressing question. I am a parent, but my kids' digital habits are not, you know, of obsessive interest to me. Sometimes I think the worry about our kids is a proxy for how we worry about ourselves. You know, it's funny because they're the, you know, the model of the parent saying my kid has attention deficit order, zero order. My kid has attention deficit disorder. The kids over here, the parents here, you know, who has the attention deficit disorder. But in any case I have realized that parents are talking about uh, computers on the internet as though something kids have to have a very ambivalent relationship with and a very wary relationship with. So limit the time, and it sounds a little bit like the abstinence movement around sexuality that like, you know, you only dip in, it's very, you know, they're only date, right, right, right. >>Instead of joining sides with their kids and helping to create a durable, powerful, interesting online avatar, which is what kids want to do. And it's also what we want to do. So like in your Facebook profile, there are all kinds of strategic groups you can make as a creator of that profile. We know it as adults. Like, do you, some people put up pictures of their kids, some people don't vacation pictures. Some people promote the heck out of themselves. Some people don't do so much of that. Um, do you put up a lot of photographs? Do whatever. Those are the decisions we started to make when went on Facebook at kitchen making the two small armor to have on their gaming profile. That's kind of how they want to play, you know, play for you, going to wear feathers. These are important things. Um, but the uh, you know, small questions like talking to your kids and I don't mean a touchy feely conversation, but literally during the write in all lower case commit, you know, Brighton, all lower case, you're cute and you're this and that means a certain thing and you should get it and you're going to write in all caps and you're going to talk about white nationalist ideology. >>Well that also has a set of consequences. What have you learned in terms of the virtual space? Actually augmented reality, virtual reality, these promise to be virtual spaces. What, what is one of them? They always hope to replicate the real world. The mean, yes. Will there be any parallels of the kind of commitment in the moment? Gives you one thing. I say kids that, you know, the subtitle of the book is the internet as art, magic and loss. The internet is art and the kind of art, the internet is, is what I think of as real estate art. It purports to be reality. You know, every technology pick a photography film says or think of even the introduction of a third dimension in painting, you know, in Renaissance painting perspective for ports to represent reality better than it's been represented before. And if you're right in sync with the technology, you're typically fooled by it. >>I mean, this is a seductive representation of reality. You know, people watching us now believe they're seeing us flush of let us talk. You know, they don't think they're seeing pixels that are designed in certain ways and certainly it's your ways. So trying to sort out the incredibly interesting immersive, artful experience of being online that has some dangers and has some emotions to do it from real life is a really important thing. And you know, for us to learn first and then a model for our kids. So I had a horrible day on Twitter one day, eight 2012 213 worst day ever on Twitter. It was a great day for me. I spent the day at the beach, my Twitter avatar took sniper fire for me all day. People called her an idiot separated amount. I separated them out. And anyone who like likes roleplay and games knows that like I'm not a high priestess in Dentons and dragons. >>You know, I'm a much smaller person than that. And in, in, you know, in the case of this Twitter battle, I'm a less embattled person than the one that takes your armor from me on Twitter. That's my art. Your armor. So let's talk about poetry. Twitter, you mentioned poetry, Twitter, 140 characters. I did 40 characters is a lot. If like a lot of internet users your to have pictographic language like Chinese. So 140 characters is a novel by, well not a novel, but it's a short story for, you know, a writer of short form, short form Chinese aphorisms like Confucius. So one of the things I wanted to say is there's nothing about it being short that makes it low culture. You know, there's, I mean it takes a second to take, to take an a sculpture or to take an a painting and yet like the amount of craft that went into that might be much more good tweeting and you're excellent at it, um, is not easy. You know, I know that times I've been like, I tagged the wrong person and then I have to delete it. Like, because the name didn't come up or you know, I get the hashtags wrong and then I'm like, Oh, it would have been better this other way or I don't have a smart enough interject >>it's like playing sports. Twitter's like, you know, firing under the tennis ball baseline rallies with people. I mean, it's like, it's like there's a cultural thing. And this is the thing that I love about your book is you really bring in the metaphors around art and the cultural aspect. Have you had any, have you found that there's one art period that we represent right now? That it could be a comparison? >>I mean, you know, it's always tempting to care everything to the Renaissance. But you know, obviously in the Italian Renaissance there was so much technological innovation and so much, um, and so much, uh, so much artistic innovation. But, um, you know, the other thing are the Dawn of it's might be bigger than that, which it sounds grounds grandiose, but we're talking about something that nearly 6 billion people use and have access to. So we're talking about something bigger than we've ever seen is the Donovan civilization. So like, we pay a lot of attention to the Aqua docks and Rome and, and you know, later pay to touch it to the frescoes. I attend in this book to the frescoes, to the sculpture, to the music, to the art. So instead of talking about frescoes as an art historian, might I talk about Instagram? Yeah. >>And you, and this thing's all weave together cause we can back to the global fabric. If you look at the civilization as you know you're not to use the world is flat kind of metaphor. But that book kind of brings out that notion of okay if you just say a one global fabric, yes you have poetry, you have photography of soiling with a Johnny Susana ad in London. He says, you know, cricket is a sport in England, a bug and a delicacy depending on where in the world you are. >>Love that is that, I wonder if that's the HSBC had time to actually a beautiful HSBC job has done a beautiful campaign. I should find out who did it about perspective. And that is also a wonderful way to think about the internet because you know, I know a lot of people who don't like Twitter, who don't like YouTube comments. I do like them because I am perpetually surprised at what people bring to their interpretation. Insights in the comments can be revealing. You know, you know, you don't wanna get your feelings hurt. Sometimes you don't want that much exposure to the micro flora and fauna of ideas that could be frightening. But you know, when you're up for it, it's a really nice test of your immune system, you know. All right. So what's next for you? Virginia Heffernan magic and last great book. I think I will continue to write the tech criticism, which is just this growing field. I at Sarah Watson had a wonderful piece today in the Columbia journalism review about how we really need to bring all our faculties to treat, treating to tech criticism meant and treating tech with, um, with Karen, with proper off. Um, and the next book is on anti digital culture. Um, I will continue writing journalism and you'll see little previews of that book in the next work. >>Super inspirational. And I think the culture needs this kind of rallying cry because you know, there is art and science and all this beautiful beauty in the internet and it's not about mutually exclusive analog world. You can look and take, can come offline. So it's interesting case study of this, this revolution I think, and I think the counter culture, if you'd go back and Southern John Markoff about this, when he wrote his first book, the Dormouse wander about the counter culture in Silicon Valley is what's your grade book? And counter cultures usually create a another wave of innovation. So the question that comes out of this one is there could, this could be a seminal moment in history. I mean, I think it absolutely is. You know, in some ways, every moment is a great moment if you know what to make of it. But I am just tired of people telling us that we're ruining our brands and that this is the end of innovation and that we're at some low period. >>I think we will look back and think of this as an incredibly fertile time for our imaginations. If we don't lose hope, if we keep our creativity fired and if we commit to this incredible period we're in Virginia. Thanks for spending the time here in the queue. Really appreciate where you're live at. Silicon Valley is the cube with author Virginia Heffernan magic. And loss. Great book. Get it. If you don't have it, hard copies still available, get it. We'll be right back with more live coverage here. This is the cube. I'm John furry right back with more if the short break.

>> Narrator: Live from Dublin, Ireland. It's theCUBE, covering Hadoop Summit Europe 2016. Brought to you by Hortonworks. Now your host, John Furrier and Dave Vellante. >> Okay, welcome back here, we are here live in Dublin, Ireland, it's theCUBE SiliconANGLEs flagship program where we go out to the events and extract the signal from the noise, I'm John Furrier, my cohost Dave Vellante, our next guest is Emer Coleman who's with Disruption Limited, Open Data Governance Board in Ireland and Transport API, a growing startup built self-sustainable, growing business, open data, love that keynote here at Hadoop Summit, very compelling discussion around digital goods, digital future. Emer, welcome to theCUBE. >> It's great to be here. >> So what was your keynote? Let's just quickly talk about what you talked about, and then we can get in some awesome conversation. >> Sure. So the topic yesterday was we need to talk about techno ethics. So basically, over the last couple of months, I've been doing quite a lot of research on ethics and technology, and many people have different interpretations of that, but yesterday I said it's basically about three things. It's about people, it's about privacy, and it's about profits. So it's asking questions about how do we look at holistic technology development that moves away from a pure technocratic play and looks at the deep societal impacts that technology has. >> One of the things that we're super excited about and passionate about is this new era of openness going to a whole another level. Obviously, open source tier one software development environment, cloud computing allows for instant access to resources, almost limitless at this point, as you can project it forward with Moore's Law and whatnot. But the notion that digital assets are not just content, it's data, it's people, it's the things you mentioned about, create a whole new operating environment or user experience, user expectations with mobile phones and Internet of Things and Transport API which you have, if it moves, you capture it, and you're providing value there. So a whole new economy is developing around digital capital. Share your thoughts around this, because this is an area that you're passionate about, you've just done work here, what's your thoughts on this new digital economy, digital capital, digital asset opportunity? >> I think there's huge excitement about the digital economy, isn't there? And I think one of the things I'm concerned about is that that excitement will lead us to the same place that we are now, where we're not really thinking through what are the equitable distribution in that economy, because it seems to me that the spoils are going to a very tiny elite at the tops. So if you look at Instagram, 13 employees when it was purchased by Facebook for a billion dollars, but that's all our stuff, so I'm not getting any shares in the billion, those 13 people are. That's fantastic that you can build a business, build it to that stage and sell, but you have to think about two things, really: what are we looking at in terms of sustainable businesses into the future that create ethical products, and also the demands from citizens to get some value for their data back, because we're becoming shadow employees, we're shadow employees of Google, so when we email, we're not just corresponding, we're creating value for that company. >> And Facebook is a great example. >> And Facebook, and the thing is, when we were at the beginning of that digital journey, it was quite naive. So we were very seduced by free, and we thought, "This is great," and so we're happy with the service. And then the next stage of that, we realize what if we're not paying for the service, we're the product? >> John: Yeah. >> But we were too embedded in the platform to extricate ourselves. But now, I think, when we look at the future of work and great uncertainty that people are facing, when their labor's not going to be required to the same degree, are we going to slavishly keep producing capital and value for companies like Google, and ask for nothing more than the service in return? I don't think so. >> And certainly, the future will be impacted, and one of the things we see now in our business of online media and online open data, is that the data's very valuable. We see that, I'll say data is the new capital, new oil, whatever phrases of the day is used, and the brand marketers are the first ones to react to it, 'cause they're very data driven. Who are you, how do I sell stuff to you? And so what we're seeing is, brand marketers are saying, "Hey, I'm going to money to try to reach out to people, "and I'm going to activate that base and connect with, "engage with them on Facebook or other platform. "I'm going to add value to your Facebook or Google platform, "but yet I'm parasitic to your platform for the data. "Why just don't I get it directly?" So again, you're starting to see that thinking where I don't want to be a parasite or parasitic to a network that the value's coming from. The users have not yet gotten there, and you're teasing that out. What's your thoughts there, progression, where we're at, have people realized this? Have you seen any movement in the industry around this topic? >> No, I think there's a silence around... Technology companies want to get all the data they can. They're not going to really declare as much as they should, because it bends their service model a bit. Also, the data is emergent. Zuckerberg didn't start Facebook as something that was going to be a utility for a billion people, he started it as a social network for a university. And what grew out of that, we learned as we went along. So I'm thinking, now that we have that experience, we know that happens, so let's start the thinking now. And also, this notion of just taking data because you can, almost speculatively getting data at the point of source, without even knowing what you want it for but thinking, "I'm going to monetize this in the end." Jaron Lanier in his book Who Owns The Future talks about micro licensing back content. And I think that's what we need to do. We start, at the very beginning, we need to start baking in two things: privacy by design and different business models where it's not a winner takes all. It's a dialog between the user and the service, and that's iterated together. >> This idea that it's not a zero sum game is very important, and I want to go back to your Instagram and Facebook example. At its peak, I think Eastman Kodak had hundreds of thousands of employees, maybe four or five hundred, 450,000 employees, huge. Facebook has many many more photos, but maybe a few thousand employees? Wow, so all the jobs are gone, but at the same time, we don't want to be protecting the past from the future, so how do you square that circle? >> Correct, but I think what we know is that the rise of robotics and software is going to eat jobs, and basically, there's going to be a hollowing out of the middle class. You know, for sure, whether it's medicine, journalism, retail, exactly. >> Dave: It's not future, it's now. (laughs) >> Exactly. So we maybe come into a point where large swaths of people don't have work. Now, what do you do in a world where your labor is no longer required? Think about the public policy implications of that. Do we say you either fit in this economy or you die? Are we going to look at ideas which they are looking at in Europe, which is like a universal wage? And all of these things are a challenge to government, because they're going to have a citizenry who are not included in this brave new world. So some public policy thinking has to go into what happens when our kids can't get jobs. When the jobs that used to be done by people like us are done by machines. I'm not against the movement of technology, what I'm saying is there are deep societal implications that need some thinking, because if we get to a point where we suddenly realize, if all of these people who are unemployed and can't get work, this isn't a future we envisioned where robots would take all the crap jobs and we would go off to do wonderful things, like how are we going to bring the bacon home? >> It seems like in a digital world that the gap is creativity to combine technologies and knowledge. I find that it's scary when you talk about maybe micromanaging wages and things like that, education is the answer, but that's... How do you just transfer that knowledge? That's sort of the discussion that we're having in the United States anyway. >> I think some of the issue is that the technology is so, we're kind of seduced by simplicity. So we don't see the complexity underneath, and that's the ultimate aim of a technology, is to make something so simple, that complexity is masked. That's what the iPhone did wonderfully. But that's actually how society is looking now. So we're seduced by this simplicity, we're not seeing the complexity underneath, and that complexity would be about what do we do in a world where our labor is no longer required? >> And one of the things that's interesting about the hollowing of the middle class is the assumption is there's no replacements, so one of the things that could be counter argued is that, okay, as the digital natives, my daughter, she's a freshman in high school, my youngest son's eighth grade, they're natives now, so they're going to commit. So what is the replacement capital and value for companies that can be sustained in the new economy versus the decay and the darwinism of the old? So the digital darwinism aspect's interesting, that's one dilemma. The other one is business models, and I want to get your thoughts on this 'cause this is something we were teasing out with this whole value extraction and company platform issue. A company like Twitter. Highly valuable company, it's a global network of people tweeting and sharing, but yet is under constant pressure from Wall Street and investors that they basically suck. And they don't, they're good, people love Twitter, so they're being forced to behave differently against their mission because their profit motive doesn't really match maybe something like Facebook, so therefore they're instantly devalued, yet the future of someone connecting on Twitter is significantly high. That being said, I want to get your thoughts on that and your advice to Twitter management, given the fact it is a global network. What should they do? >> It's the same old capitalism, just it's digital, it's a digital company, it's a digital asset. It's the same approach, right? Twitter has been a wonderful thing. I've been a Twitter user for years. How amazing, it's played a role in the Arab Spring, all sorts of things. So they're really good, but I think you need as a company, so for example, in our company, in Transport API, we're not really looking to build to this massive IPO, we're trying to build a sustainable company in a traditional way using digital. So I think if you let yourself be seduced by the idea of phenomenal IPO, you kind of take your eye off the ball. >> Or in case this, in case you got IPOed, now you're under pressure to produce-- >> Emer: Absolutely, yeah. >> Which changes your behavior. But in Twitter's management defense, they see the value of their product. Now, they got there by accident and everyone loves it, but now they're not taking the bait to try to craft a short term solution to essentially what is already a valuable product, but not on the books. >> Yes, and also I think where the danger is, we know that their generation shifts across channel. So teenagers probably look at Facebook, I think one of them said, like an awkward family dinner they can't quite leave. But for next gen, they're just not going to go there, 'cause that's where your grandmother is. So the same is true of Twitter and Snapchat, these platforms come and go. It's an interesting phenomenon then to see Wall Street putting that much money into something which is essentially quite ephemeral. I'm not saying that Twitter won't be around for years, it may be, but that's the thing about digital, isn't it? Something else comes in and it's well, that becomes the platform of choice. >> Well, it's interesting, right? Everybody, us included, we criticize the... Michael Dell calls it the 90 day shock clock. But it's actually worked out pretty well, I mean, economically, for the United States companies. Maybe it doesn't in the future. What are your thoughts on that, particularly from a European perspective? Where you're reporting maybe twice a year, there's not as much pressure, but yet from a technology industry standpoint, companies outside the Silicon Valley in particular seem to be less competitive, why? >> For example, in our company, in Transport API, we've got some pretty heavyweight clients, we have a wonderful angel investor who has given us two rounds of investment. And it isn't that kind of avaricious absolutely built this super price. And that's allowed us to build from starting off with 2, now to a team of 10, and we're just about coming into break even, so it's doable. But I think it's a philosophy. We didn't want necessarily to build something huge, although we want to go global, but it was let's do this in a sustainable way with reasonable wages, and we've all put our own soul and money into it, but it's a different cultural proposition, I think. >> Well, the valuations always drive the markets. It's interesting too, to your point about things come and go channels, kind of reminds me, Dave and I used to joke about social networks like nightclubs, they're hot and then it's just too crowded and nobody goes there, as Yogi Bear would say. And then they shift and they go out of business, some don't open with fanfare, no one goes 'cause it's got different context. You have a contextual challenge in the world now. Technology can change things, so I want to ask you about identity 'cause there was a great article posted by the founder of the company called Secret which is one of these anonymous apps like Yik Yak and whatnot, and he shut it down. And he wrote a post, kind of a postmortem, saying, "These things come and go, they don't work, "they're not sustainable because there's no identity." So the role of identity in a social global virtual world, virtual being not just virtual reality, is interesting. You live in a world, and your company, Transport API, provides data which enables stuff and the role of identity. So anonymous versus identity, thoughts there, and that impact to the future of work? If you know who you're dealing with, and if they're present, these are concepts that are now important, presence, identity, attention. >> And that's the interesting thing, isn't it? Who controls that identity? Mark Zuckerberg said, "You only have one identity," which is what he said when he set up Facebook. You think, really? No, that's what a young person thinks. When we're older, we know. >> He also said that young people are smarter than older people. >> Yeah, right, okay. (John laughs) He could be right there, he could be right there, but we all have different identities in different parts of our lives. Who we are here, the Hadoop summit is different from what we're at home to when we're with friends. So identity is a multifaceted thing. But also, who gets to determine your identity? So I have 16 years of my search life and Google. Now, who am I in that server, compared to who I am? I am the sum total of my searches. But I'm not just the sum total of my searches, am I? Or even that contextualized, so I'll give you an example. A number of years ago I was searching for a large, very large waterproof plastic bag. And I typed it in, and I thought, "Oh my god, that sounds like I'm going to murder my husband "and try to bury him." (John and Dave laugh) It was actually-- >> John: Into the compost. >> Right, right. And I thought, "Oh my god, what does this look like "on the other side?" Now, it was actually for my summer garden furniture. But the point is, if you looked at that in an analytic way, who would I be? And so I think identity is very, you know-- >> John: Mistaken. >> Yeah, and also this idea of what Frank Pasquale calls the black box society. These secret algorithms that are controlling flows of money and information. How do they decide what my identity is? What are the moral decisions that they make around that? What does it say if I search for one thing over another? If I search constantly for expensive shoes, does that make me shallow? What do these things say? If I search for certain things around health. >> And there's a value judgment now associated with that that you're talking about, that you do not control. >> Absolutely, and which is probably linked to other things which will determine things like whether I get credit or not, but these can almost be arbitrary decisions, 'cause I have no oversight of the logic that's creating that decision making algorithm. So I think it's not just about identity, it's about who's deciding what that identity is. >> And it's also the reality that you're in, context, situations. Dark side, bright side of technology in this future where this new digital asset economy, digital capital. There's going to be good and bad, education can be consumed non-linear, new forms of consumptions, metadata, as you're pointing out, with the algorithms. Where do you see some bright spots and where do you see the danger areas? >> I think the great thing is, when you were saying software is the future. It's our present, but it's going to be even more so in our future. Some of the brightest brains in the world are involved in the creation of new technology. I just think they need to be focusing a bit more of that intellectual rigor towards the impact they're having on society and how they could do it better. 'Cause I think it's too much of a technocratic solution. Technologists say, "We can do this." The questions is, should they? So I think what we need to do is to loop them back into the more social and philosophical side of the discussion. And of course it's a wonderful thing, hopefully technology is going to do amazing things around health. We can't even predict how amazing it's going to be. But all I'm saying is that, if we don't ask the hard questions now about the downsides, we're going to be in a difficult societal position. But I'm hoping that we will, and I'm hoping that raising issues like techno ethics will get more of that discussion going. >> Well, transparency and open data make a big difference. >> Emer: Absolutely. >> Well, and public policy, as you said earlier, can play a huge role here. I wonder if you could give us your perspective on... Public policy, we're in the US most of the time, but it's interesting when we talk to customers here. To hear about the emphasis, obviously, on privacy, data location and so forth, so in the digital world, do you see Europe's emphasis and, I think, leading on those types of topics as an advantage in a digital world, or does it create friction from an economic standpoint? >> Yeah, but it's not all about economics. Friction is a good thing. There are some times when friction is a good thing. Most technologists think all friction is bad. >> Sure, and I'm not implying that it's necessarily good or bad, I'm curious though, is it potentially an economic advantage to have thought through and have policy on some of those issues? >> Well, what we're seeing here-- >> Because I feel like the US is a ticking time bomb on a lot of these issues. >> I was talking to VCs, some VC friends of mine here in the UK, and what they said they're seeing more and more, VCs asking what we call SMEs, small to medium enterprises, about their data policies, and SMEs not being able to answer those questions, and VCs getting nervous. So I think over time it's going to be a competitive advantage that we've done that homework, that we're basically not just rushing to get more users, but that we're looking at it across the piece. Because, fundamentally, that's more sustainable in the longer term. People will not be dumb too forever. They will not, and so doing that thinking now, where we work with people as we create our technology products, I think it's more sustainable in the long term. When you look at economics, sustainability is really important. >> I want to ask you about the Transport API business, 'cause in the US, same thing, we've seen some great openness of data and amazing innovations that have come out of nowhere. In some cases, unheard of entrepreneurs and/or organizations that better society for the betterment of people, from delivering healthcare to poor areas and whatnot. What has been the coolest thing, or of things you've seen come out of your enablement of the transport data. Use cases, have you seen any things that surprised you? >> It's quite interesting, because when I worked for the mayor of London as his director of digital projects, my job was to set up the London data store, which was to open all of London's public sector data. So I was kind of there from the beginning as a lobbyist, and when I was asking agencies to open up their data, they'd go, "What's the ROI?" And I'd just say, "I don't know." Because government's one and oh, I'm saying that was a chicken and egg, you got to put it out there. And we had a funny incident where some of the IT staff in transport for London accidentally let out this link, which is to the tracker net feed, and that powers the tube notice boards that says, "Your next tube is in a minute," whatever. And so the developer community went, "Ooh, this is interesting." >> John: Candy! >> Yeah, and of course, we had no documentation with it because it kind of went out under the radar. And one developer called Mathew Somerville made this map which showed the tubes on a map in real time. And it was like surfacing the underground. And people just thought, "Oh my god, that is amazing." >> John: It's illuminating. >> Yeah. It didn't do anything, but it showed the possibility. The newspapers picked it up, it was absolutely brilliant example, and the guy made it in half a day. And that was the first time people saw their transport system kind of differently. So that was amazing, and then we've seen hundreds of different applications that are being built all the time. And what we're also seeing is integration of transport data with other things, so one of our clients in Transport API is called Toothpick, and they're an online dental booking agency. And so you can go online, you can book your dental appointment with your NHS dentist, and then they bake in transport information to tell you how to get there. So we have pubs using them, and screens so people can order their dinner, and then they say, "You've got 10 minutes till the next bus." So all sorts of cross-platform applications. >> That you never could've envisioned. >> Emer: Never. >> And it's just your point earlier about it's not a zero sum game, you're giving so many ways to create value. >> Emer: Right, right. >> Again, I come back to this notion of education and creativity in the United States education system, so unattainable for so many people, and that's a real concern, and you're seeing the middle class get hollowed out. I think the stat is, the average wage in the United States was 55,000 in 1999, it's 50,000 today. The political campaigns are obviously picking at that scab. What's the climate like in Europe from that standpoint? >> In terms of education? >> No, just in terms of, yes, the education, middle class getting hollowed out, the sentiment around that. >> I don't think people are up to speed with that yet, I really don't think that they're aware of the scale. I think when they think robots or automation, they don't really think software. They think robots like there were in the movies, that would come, as I say, and do those jobs nobody wanted. But not like software. So when I say to them, look, E-discovery software, when it's applied retrospectively, what it shows is that human lawyers are only 60% accurate compared to it. Now, that's a no-brainer, right? If software is 100% accurate, I'm going to use the software. And the ratio difference is 1 to 500. Where you needed 500 lawyers before you need 1. So I don't think people are across the scale of change. >> But it's interesting, you're flying to Heathrow, you fly in and out, you're dealing with a kiosk. You drive out, the billboards are all electronic. There aren't guys doing this anymore. So it's tangible. >> And I think, to your point about education, I'm not as familiar with the education system in the US, but I certainly think, in Europe and in the UK, the education system is not capable of dealing even with the latest digital natives. They're still structuring their classrooms in the same way. These kids, you know-- >> John: They have missed the line with the technology. >> Absolutely. >> So reading, writing and arithmetic, fine. And the cost of education is maybe acceptable. But they may be teaching the wrong thing. >> Asynchronous non-linear, is the thing. >> There's a wonderful example of an Indian academic called Sugata Mitra, who has a fabulous project called a Hole in the Wall. And he goes to non-English speaking little Indian villages, and he builds a computer, and he puts a roof over it so only the children can do it. They don't speak English. And he came back, and he leaves a little bit of stuff they have to get around before they can play a game. And he came back six months later, and he said to them, "What did you think?" And one of the children said, "We need a faster CPU and a better mouse." Now, his point is self-learning, once you have access to technology, is amazing, and I think we have to start-- >> Same thing with the non-linear consumption, asynchronous, all this, the API economy enabling new kinds of expectation and opportunities. >> And it was interesting because the example, some UK schools tried to follow his example. And six months later, they rang him up and they said, "It's not working," and he said, "What did you do?" And they said, "Well, we got every kid a laptop." He said, "That's not the point." The point was putting a scarce resource that the children had to collaborate over. So in order to get to the game, they had figure out certain things. >> I think you're right on some of these (mumbles) that no one's talking about. And Dave and I are very passionate on this, and we're actually investing in a whole new e-learning concept. But it's not about doing that laptop thing or putting courseware online. That's old workflow in a new model. Come on, old wine in a new bottle. So that's interesting. I want to get your thoughts, so a personal question to end this segment. What are you passionate about now, what are you working, outside of the venture, which is exciting. You have a lot of background going back to technology entrepreneurship, public policy, and you're in the front lines now, thought leading on this whole new wide open sea of opportunity, confusion, enabling it. What are you passionate about, what are you working on? Share with the folks that are watching. >> So one of the main things we're trying to do. I work as an associate with Ernst & Young in London. And we've been having discussions over the past couple of months around techno ethics, and I've basically said, "Look, let's see if we can get EY "to build to build an EY good governance index." Like, what does good governance look like in this space, a massively complex area, but what I would love is if people would collaborate with us on that. If we could help to draw up an ethical framework that would convene the technology industry around some ethical good governance issues. So that's what I'm going to be working on as hard as I can over the next while, to try and get as much collaboration from the community, because I think we'd be so much more powerful if the technology industry was to say, "Yeah, let's try and do this better "rather than waiting for regulation," which will come, but will be too clunky and not fit for purpose. >> And which new technology that's emerging do you get most excited about? >> Hmm. Drones. (laughter) >> How about anything with bitcoin, block chains? >> Absolutely, absolutely, block chain. Yeah, block chain, you have to say, yeah. I think, 'cause bitcoin, you know, it's worth 20 p today, it's worth 200,000 tomorrow. >> Dave: Yeah, but block chain. >> Right, right. I mean, that is incredible potentiality. >> New terms like federated, that's not a new term, but federation, universal, unification. These are the themes right now. >> Emer: Well, it's like the road's been coated, isn't it? And we don't know where it's going to go. What a time we live in, right? >> Emer Coleman, thank you so much for spending your time and joining us on theCUBE here, we really appreciate the conversation. Thanks for sharing that great insight here on theCUBE, thank you. It's theCUBE, we are live here in Dublin, Ireland. I'm John Furrier with Dave Vellante. We'll we right back with more SiliconANGLEs, theCUBE and extracting the signal from the noise after this short break. (bright music)

>>Live from New York city. It's cute. here's your host? Jeff Frick. >>Hi, Jeff Frick here with the Q we're on the ground at the USS Intrepid at the Hadoop on the Hudson party put on by Matt BARR. It's uh, I think it's the party of the night tonight here in big data week, New York city with strata cough, a dupe world, big data NYC. So Jack a great >>Venue. Yeah, it's excellent. Here. >>The place is filled. I'm just struck by the technology. There's a Gemini capsule over there, about 50 years old. It's about the size of a Volkswagen, I think would be much bigger. And to think that those guys went up into space with probably less technology than is on your four year old flip phone. Amazing. Yeah. >>Not, not much data at all. No. If >>You look at it, just kind of get that bounce on the gravity thing, which I never quite understood. So talk about you guys had some big news today. Once you give us a rundown on some of the announcements, >>We had two big announcements. One was incorporating the map RDB and our community edition that came out. We also reported results from our customers where the majority of customers reported less than a 12 month payback, uh, 65% of five X or greater return and 40%, 10 X or greater. And that included a subset of those customers that had experienced with other distributions. So kind of a Testament to when you get serious about Hadoop, you get serious with Mapbox >>And when they're getting those return on investments, we're always trying to explore where's the big, the big ROI, because it's really in value that's released for the customer. It's not necessarily because it's a cheaper way to do it, >>Right? So, so there are some costs that 63% was cost reduction that was driving it about 41% were top-line revenue projects. And about 23% were related to risk reduction and risk mitigation. And if you add those up, it's greater than a hundred percent because of many customers that are doing multiple applications. >>Great. So you've been coming to Hadoop world for longer than you would admit to me before we came on camera and, and the baseball playoffs are going on right now. I mean, we like to talk in sports analogy. So kind of where are we in, in kind of what inning are we in this adoption of big data and the duke specifically >>Early, early innings. Um, but, uh, what we've seen is the bases are loaded and we're up >>And it's it. And it seems to be we're way past now the POC stage. Now we're really getting in there for that. >>And the, the customer announcement, we did kind of shows how people are hitting it out of the park with Hadoop. And a lot of that is by impacting the operations, impacting the business as it happens. And that's coupling analytics plus this higher arrival rate data from a variety of sources and making adjustments so that you can impact revenue as businesses happening. You can mitigate risk as it's happening. It's not just reporting, looking back >>Function. Right, right. It's being able to react in real time, which is defined by, in time to do something about it. Right. Exactly. All right. Well, thanks for hosting a great party, Jack Norris. Here we are on the ground, uh, at the USS Intrepid at the Hadoop on the Hudson. Uh, uh, if you take a nice picture, tweet that in. I think they got some prizes. Hadoop Hudson is a hashtag Jeff Frick on the ground. You're watching the cube. Thanks. Big ship.

the cube and hadoop summit 2014 is brought to you by anchor sponsor Hortonworks we do have do and headline sponsor when disco we make hadoop invincible okay welcome back and when we're here at the dupe summit live is looking valance the cube our flagship program we go out to the events expect a signal from noise i'm john per year but Jeff Rick drilling down on the topics we're here with wind disco welcome welcome Brett room Stein about senior director tell us what's going on for you guys I'll see you at big presence here so all the guys last night you guys have a great great booth so causing and the crew what's happening yeah I mean the show is going is going very well what's really interesting is we have a lot of very very technical individuals approaching us they're asking us you know some of the tougher more technical in-depth questions about how our consensus algorithm is able to do all this distributor replication which is really great because there's a little bit of disbelief and then of course we get to do the demonstration for them and then suspend disbelief if you will and and I think the the attendance has been great for our brief and okay I always get that you always we always have the geek conversations you guys are a very technical company Jeff and I always comment certainly de volada and Jeff Kelly that you know when disco doesn't has has their share pair of geeks and that dudes who know they're talking about so I'm sure you get that but now them in the business side you talk to customers I want to get into more the outcome that seems to be the show focused this year is a dupe of serious what are some of the outcomes then your customers are talking about when they get you guys in there what are their business issues what are they tore what are they working on to solve yeah I mean I think the first thing is to look at you know why they're looking at us and then and then with the particular business issues that we solve and the first thing and sort of the trend that we're starting to see is the prospects and the customers that we have are looking at us because of the data that they have and its data that matters so it's important data and that's when people start to come to is that's when they look to us as they have data that's very important to them in some cases if you saw some of the UCI stuff you see that the data is you know doing live monitoring of various you know patient activity where it's not just about about about a life and monitoring a life but potentially about saving the life and systems that go down not only can't save lives but they can potentially lose them so you have a demos you want to jump into this demo here what is this all about you know the demo that the demonstration that I'm going to do for you today is I want to show you our non-stop a new product i'm going to show you how we can basically stand up a single HDFS or a single Hadoop cluster across multiple data centers and I think that's one of the tough things that people are really having trouble getting their heads wrapped around because most people when they do multi data center Hadoop they tend to do two different clusters and then synchronize the data between the two of them the way they do that is they'll use you know flume or they'll use some form of parallel ingest they'll use technologies like dis CP to copy data between the data centers and each one of those has sort of an administrative burden on them and then some various flaws in their and their underlying architecture that don't allow them to do a really really detailed job as ensuring that all blocks are replicated properly that no mistakes are ever made and again there's the administrative burden you know somebody who always has to have eyes in the system we alleviate all those things so I think the first thing I want to start off with we had somebody come to our booth and we were talking about this consensus algorithm that we that we perform and the way we synchronize multiple name nodes across multiple geographies and and again and that sort of spirit of disbelief I said you know one of the key tenants of our application is it doesn't underlie it doesn't change the behavior of the application when you go from land scope to win scope and so I said for example if you create a file in one data center and 3,000 miles apart or 7,000 miles apart from that you were to hit the same create file operation you would expect that the right thing happens what somebody gets the file created and somebody gets file already exists even if at 7,000 miles distance they both hit this button at the exact same time I'm going to do a very quick demonstration of that for you here I'm going to put a file into HDFS the my top right-hand window is in Northern Virginia and then 3,000 miles distance from that my bottom right-hand window is in Oregon I'm going to put the etsy hosts file into a temp directory in Hadoop at the exact same time 3,000 miles distance apart and you'll see that exact behavior so I've just launched them both and again if you look at the top window the file is created if you look at the bottom window it says file already exists it's exactly what you'd expect a land scope up a landscape application and the way you'd expect it to behave so that is how we are ensure consistency and that was the question that the prospect has at that distance even the speed of light takes a little time right so what are some of the tips and tricks you can share this that enable you guys to do this well one of the things that we're doing is where our consensus algorithm is a majority quorum based algorithm it's based off of a well-known consensus algorithm called paxos we have a number of significant enhancements innovations beyond that dynamic memberships you know automatic scale and things of that nature but in this particular case every transaction that goes into our system gets a global sequence number and what we're able to do is ensure that those sequence numbers are executed in the correct order so you can't create you know you can't put a delete before a create you know everything has to happen in the order that it actually happened occurred in regardless of the UN distance between data centers so what is the biggest aha moment you get from customer you show them the demo is it is that the replication is availability what is the big big feature focus that they jump on yeah I think I think the biggest ones are basically when we start crashing nodes well we're running jobs we separate the the link between the win and maybe maybe I'll just do that for you now so let's maybe kick into the demonstration here what I have here is a single HDFS cluster it is spanning two geographic territory so it's one cluster in Northern Virginia part of it and the other part is in Oregon I'm going to drill down into the graphing application here and inside you see all of the name notes so you see I have three name nodes running in Virginia three name nodes running in Oregon and the demonstration is as follows I'm going to I'm going to run Terrigen and Terra sort so in other words i'm going to create some data in the cluster I'm then going to go to sort it into a total order and then I'm going to run Tara validate in the alternate data center and prove that all the blocks replicated from one side to the other however along the way I'm going to create some failures I am going to kill some of that active name nodes during this replication process i am going to shut down the when link between the two data centers during the replication paris's and then show you how we heal from from those kinds of conditions because our algorithm treats failure is a first class citizen so there's really no way to deal in the system if you will so let's start unplug John I'm active the local fails so let's go ahead and run the Terrigen in the terrorists or I'm going to put it in the directory called cube one so we're creating about 400 megabytes of data so a fairly small set that we're going to replicate between the two data centers now the first thing that you see over here on the right-hand side is that all of these name nodes kind of sprung to life that is because in an active active configuration with multiple name nodes clients actually load balance their requests across all of them also it's a synchronous namespace so any change that I make to one immediately Curzon immediately occurs on all of them the next thing you might notice in the graphing application is these blue lines over and only in the Oregon data center the blue lines essentially represent what we call a foreign block a block that is not yet made its way across the wide area network from the site of ingest now we move these blocks asynchronously from the site of in jeff's oh that I have land speed performance in fact you can see I just finished the Terrigen part of the application all at the same time pushing data across the wide area network as fast as possible now as we start to get into the next phase of the application here which is going to run terrace sort i'm going to start creating some failures in the environment so the first thing I'm going to do is want to pick two named nodes I'm going to fail a local named node and then we're also going to fail a remote name node so let's pick one of these i'm going to pick HD p 2 is the name of the machine so want to do ssh hd2 and i'm just going to reboot that machine so as I hit the reboot button the next time the graphing application updates what you'll notice here in the monitor is that a flat line so it's no longer taking any data in but if you're watching the application on the right hand side there's no interruption of the service the application is going to continue to run and you'd expect that to happen maybe in land scope cluster but remember this is a single cluster a twin scope with 3,000 miles between the two of them so I've killed one of the six active named nodes the next thing I'm going to do is kill one of the name nodes over in the Oregon data center so I'm going to go ahead and ssh into i don't know let's pick the let's pick the bottom one HTTP nine in this case and then again another reboot operation so I've just rebooted two of the six name nose while running the job but if again if you look in the upper right-hand corner the job running in Oregon kajabi running in North Virginia continues without any interruption and see we just went from 84 to eighty eight percent MapReduce and so forth so again uninterruptedly like to call continuous availability at when distances you are playing that what does continuous availability and wins because that's really important drill down on yeah I mean I think if you look at the difference between what people traditionally call high availability that means that generally speaking the system is there there is a very short time that the system will be unavailable and then it will then we come available again a continuously available system ensures that regardless of the failures that happen around it the system is always up and running something is able to take the request and in a leaderless system like ours where no one single node actually it actually creates a leadership role we're able to continue replication we're and we're also able to continue the coordinator that's two distinct is high availability which everyone kind of know was in loves expensive and then continues availability which is a little bit kind of a the Sun or cousin I guess you know saying can you put in context and cost implementation you know from a from a from a from a perspective of a when disco deployment it's kind of a continuously available system even though people look at us as somewhat traditional disaster recovery because we are replicating data to another data center but remember it's active active that means both data centers are able to write at the same time you have you get to maximize your cluster resources and again if we go back to one of the first questions you asked what are what a customer's doing this with this what a prospects want to do they want to maximize their resource investment if they have half a million dollars sitting in another data center that only is able to perform an emergency recovery situation that means they either have to a scale the primary data center or be what they want to do is utilize existing resource in an active active configuration which is why i say continuous availability they're able to do that in both data centers maximizing all their resource so you versus the consequences of not having that would be the consequences of not being able to do that is you have a one-way synchronization a disaster occurs you then have to bring that data center online you have to make sure that all the appropriate resources are there you have to you have an administrative burden that means a lot of people have to go into action very quickly with the win disco systems right what that would look like I mean with time effort cost and you have any kind of order of magnitude spec like a gay week called some guy upside dude get in the office login you have to look at individual customer service level agreements a number that i hear thrown out very very often is about 16 hours we can be back online within 16 hours really RTO 44 when disco deployment is essentially zero because both sites are active you're able to essentially continue without without any doubt some would say some would say that's contingent availability is high available because essentially zero 16 that's 16 hours I mean any any time down bad but 16 hours is huge yeah that's the service of level agreement then everyone says but we know we can do it in five hours the other of course the other part of that is of course ensuring that once a year somebody runs through the emergency configure / it you know procedure to know that they truly can be back up in line in the service level agreement timeframe so again there's a tremendous amount of effort that goes into the ongoing administrating some great comments here on our crowd chatter out chat dot net / hadoop summit joined the conversation i'll see ya we have one says nice he's talking about how the system has latency a demo is pretty cool the map was excellent excellent visual dave vellante just weighed in and said he did a survey with Jeff Kelly said large portion twenty-seven percent of respondents said lack of enterprises great availability was the biggest barriers to adoption is this what you're referring to yeah this is this is exactly what we're seeing you know people are not able to meet the uptime requirements and therefore applications stay in proof-of-concept mode or those that make it out of proof of concept are heavily burdened by administrators and a large team to ensure that same level of uptime that can be handled without error through software configuration like Linda scope so another comment from Burt thanks Burt for watching there's availability how about security yeah so security is a good one of course we are you know we run on standard dupe distributions and as such you know if you want to run your cluster with on wire encryption that's okay if you want to run your cluster with kerberos authentication that's fine we we fully support those environments got a new use case for crowd chapel in the questions got more more coming in so send them in we're watching the crowd chat slep net / hadoop summit great questions and a lot of people aren't i think people have a hard time partial eh eh versus continues availability because you can get confused between the two is it semantics or is it infrastructure concerns what is what is the how do you differentiate between those two definitions me not I think you know part of it is semantics but but but also from a win disco perspective we like to differentiate because there really isn't that that moment of downtime there is there really isn't that switch over moment where something has to fail over and then go somewhere else that's why I use that word continuous availability the system is able to simply continue operating by clients load balancing their requests to available nodes in a similar fashion when you have multiple data centers as I do here I'm able to continue operations simply by running the jobs in the alternate data center remember that it's active active so any data ingest on one side immediately transfers to the other so maybe let me do the the next part I showed you one failure scenario you've seen all the nodes have actually come back online and self healed the next part of this I want to do an separation I want to run it again so let me kick up kick that off when I would create another directory structure here only this time I'm going to actually chop the the network link between the two data centers and then after I do that I'm going to show you some some of our new products in the works give you a demonstration of that as well well that's far enough Britain what are some of the applications that that this enables people to use the do for that they were afraid to before well I think it allows you know when we look at our you know our customer base and our prospects who are evaluating our technologies it opens up all the all the regulated industries you know things like pharmaceutical companies financial services companies healthcare companies all these people who have strict regulations auditing requirements and now have a very clear concise way to not only prove that they're replicating data that data has actually made its way it can prove that it's in both locations that it's not just in both locations that it's the correct data sometimes we see in the cases of like dis CP copying files between data centers where the file isn't actually copied because it thinks it's the same but there is a slight difference between the two when the cluster diverges like that it's days of administration hour depending on the size of the cluster to actually to put the cluster you know to figure out what went wrong what went different and then of course you have to involve multiple users to figure out which one of the two files that you have is the correct one to keep so let me go ahead and stop the van link here of course with LuAnn disco technology there's nothing to keep track of you simply allow the system to do HDFS replication because it is essentially native HDFS so I've stopped the tunnel between the two datacenters while running this job one of the things that you're going to see on the left-hand size it looks like all the notes no longer respond of course that's just I have no visibility to those nodes there's no longer replicating any data because the the tunnel between the two has been shut down but if you look on the right hand side of the application the upper right-hand window of course you see that the MapReduce job is still running it's unaffected and what's interesting is once I start replicating the data again or once i should say once i start the tunnel up again between the two data centers i'll immediately start replicating data this is at the block level so again when we look at other copy technologies they are doing things of the file level so if you had a large file and it was 10 gigabytes in size and for some reason you know your your file crash but in that in that time you and you were seventy percent through your starting that whole transfer again because we're doing block replication if you had seventy percent of your box that had already gone through like perhaps what I've done here when i start the tunnel backup which i'm going to do now what's going to happen of course is we just continue from those blocks that simply haven't made their way across the net so i've started the tunnel back up the monitor you'll see springs back to life all the name nodes will have to resync that they've been out of sync for some period of time they'll learn any transactions that they missed they'll be they'll heal themselves into the cluster and we immediately start replicating blocks and then to kind of show you the bi-directional nature of this I'm going to run Tara validate in the opposite data center over in Oregon and I'll just do it on that first directory that we created and in what you'll see is that we now wind up with foreign blocks in both sides I'm running applications at the same time across datacenters fully active active configuration in a single Hadoop cluster okay so the question is on that one what is the net net summarized that demo reel quick bottom line in two sentences is that important bottom line is if name notes fail if the wind fails you are still continuously operational okay so we have questions from the commentary here from the crowd chat does this eliminate the need for backup and what is actually transferring certainly not petabytes of data ? I mean you somewhat have to transfer what what's important so if it's important for you to I suppose if it was important for you to transfer a petabyte of data then you would need the bandwidth that support I transfer of a petabyte of data but we are to a lot of Hollywood studios we were at OpenStack summit that was a big concern a lot of people are moving to the cloud for you know for workflow and for optimization Star Wars guys were telling us off the record that no the new film is in remote locations they set up data centers basically in the desert and they got actually provisioned infrastructure so huge issues yeah absolutely so what we're replicating of course is HDFS in this particular case I'm replicating all the data in this fairly small cluster between the two sites or in this case this demo is only between two sites I could add a third site and then a failure between any two would actually still allow complete you know complete availability of all the other sites that still participate in the algorithm Brent great to have you on I want to get the perspective from you in the trenches out in customers what's going on and win disco tell us what the culture there what's going on the company what's it like to work there what's the guys like I mean we we know some of the dudes there cause we always drink some vodka with him because you know likes to tip back a little bit once in a while but like great guy great geeks but like what's what's it like it when disco I think the first you know you touched on a little piece of it at first is there are a lot of smart people at windows go in fact I know when I first came on board I was like wow I'm probably the most unsmoked person at this company but culturally this is a great group of guys they like to work very hard but equally they like to play very hard and as you said you know I've been out with cause several times myself these are all great guys to be out with the culture is great it's a it's a great place to work and you know so you know people who are who are interested should certainly yeah great culture and it fits in we were talking last night very social crowd here you know something with a Hortonworks guide so javi medicate fortress ada just saw him walk up ibm's here people are really sociable this event is really has a camaraderie feel to it but yet it's serious business and you didn't the days they're all a bunch of geeks building in industry and now it's got everyone's attention Cisco's here in Intel's here IBM's here I mean what's your take on the big guys coming in I mean I think the big guys realize that that Hadoop is is is the elephant is as large as it appears elephant is in the room and exciting and it's and everybody wants a little piece of it as well they should want a piece of it Brett thanks for coming on the cube really appreciate when discs are you guys a great great company we love to have them your support thanks for supporting the cube we appreciate it we right back after this short break with our next guest thank you

>>The queue at Hadoop summit, 2014 is brought to you by anchor sponsor Hortonworks. We do, I do. And headline sponsor when disco we make Hadoop invincible >>Okay. Welcome back. Everyone live here in Silicon valley in San Jose. This is a dupe summit. This is Silicon angle and Wiki bonds. The cube is our flagship program. We go out to the events and extract the signal to noise. I'm John barrier, the founder SiliconANGLE joins my cohost, Jeff Kelly, top big data analyst in the, in the community. Our next guest, Jack Norris, COO of map R security enterprise. That's the buzz of the show and it was the buzz of OpenStack summit. Another open source show. And here this year, you're just seeing move after, move at the moon, talking about a couple of critical issues. Enterprise grade Hadoop, Hortonworks announced a big acquisition when all in, as they said, and now cloud era follows suit with their news. Today, I, you sitting back saying, they're catching up to you guys. I mean, how do you look at that? I mean, cause you guys have that's the security stuff nailed down. So what Dan, >>You feel about that now? I think I'm, if you look at the kind of Hadoop market, it's definitely moving from a test experimental phase into a production phase. We've got tremendous customers across verticals that are doing some really interesting production use cases. And we recognized very early on that to really meet the needs of customers required some architectural innovation. So combining the open source ecosystem packages with some innovations underneath to really deliver high availability, data protection, disaster recovery features, security is part of that. But if you can't predict the PR protect the data, if you can't have multitenancy and separate workflows across the cluster, then it doesn't matter how secure it is. You know, you need those. >>I got to ask you a direct question since we're here at Hadoop summit, because we get this question all the time. Silicon lucky bond is so successful, but I just don't understand your business model without plates were free content and they have some underwriters. So you guys have been very successful yet. People aren't looking at map are as good at the quiet leader, like you doing your business, you're making money. Jeff. He had some numbers with us that in the Hindu community, about 20% are paying subscriptions. That's unlike your business model. So explain to the folks out there, the business model and specifically the traction because you have >>Customers. Yeah. Oh no, we've got, we've got over 500 paying customers. We've got at least $1 million customer in seven different verticals. So we've got breadth and depth and our business model is simple. We're an enterprise software company. That's looking at how to provide the best of open source as well as innovations underneath >>The most open distribution of Hadoop. But you add that value separately to that, right? So you're, it's not so much that you're proprietary at all. Right. Okay. >>You clarify that. Right. So if you look at, at this exciting ecosystem, Hadoop is fairly early in its life cycle. If it's a commoditization phase like Linux or, or relational database with my SQL open source, kind of equates the whole technology here at the beginning of this life cycle, early stages of the life cycle. There's some architectural innovations that are really required. If you look at Hadoop, it's an append only file system relying on Linux. And that really limits the types of operations. That types of use cases that you can do. What map ours done is provide some deep architectural innovations, provide complete read-write file systems to integrate data protection with snapshots and mirroring, et cetera. So there's a whole host of capabilities that make it easy to integrate enterprise secure and, and scale much better. Do you think, >>I feel like you were maybe a little early to the market in the sense that we heard Merv Adrian and his keynote this morning. Talk about, you know, it's about 10 years when you start to get these questions about security and governance and we're about nine years into Hadoop. Do you feel like maybe you guys were a little early and now you're at a tipping point, whereas these more, as more and more deployments get ready to go to production, this is going to be an area that's going to become increasingly important. >>I think, I think our timing has been spectacular because we, we kind of came out at a time when there was some customers that were really serious about Hadoop. We were able to work closely with them and prove our technology. And now as the market is just ramping, we're here with all of those features that they need. And what's a, what's an issue. Is that an incremental improvement to provide those kind of key features is not really possible if the underlying architecture isn't there and it's hard to provide, you know, online real-time capabilities in a underlying platform that's append only. So the, the HDFS layer written in Java, relying on the Linux file system is kind of the, the weak underbelly, if you will, of, of the ecosystem. There's a lot of, a lot of important developments happening yarn on top of it, a lot of really kind of exciting things. So we're actively participating in including Apache drill and on top of a complete read-write file system and integrated Hindu database. It just makes it all come to life. >>Yeah. I mean, those things on top are critical, but you know, it's, it's the underlying infrastructure that, you know, we asked, we keep on community about that. And what's the, what are the things that are really holding you back from Paducah and production and the, and the biggest challenge is they cited worth high availability, backup, and recovery and maintaining performance at scale. Those are the top three and that's kind of where Matt BARR has been focused, you know, since day one. >>So if you look at a major retailer, 2000 nodes and map bar 50 unique applications running on a single cluster on 10,000 jobs a day running on top of that, if you look at the Rubicon project, they recently went public a hundred million add actions, a hundred billion ad auctions a day. And on top of that platform, beats music that just got acquired for $3 billion. Basically it's the underlying map, our engine that allowed them to scale and personalize that music service. So there's a, there's a lot of proof points in terms of how quickly we scale the enterprise grade features that we provide and kind of the blending of deep predictive analytics in a batch environment with online capabilities. >>So I got to ask you about your go to market. I'll see Cloudera and Hortonworks have different business models. Just talk about that, but Cloudera got the massive funding. So you get this question all the time. What do you, how do you counter that army and the arms race? I think >>I just wrote an article in Forbes and he says cash is not a strategy. And I think that was, that was an excellent, excellent article. And he goes in and, you know, in this fast growing market, you know, an amount of money isn't necessarily translate to architectural innovations or speeding the development of that. This is a fairly fragmented ecosystem in terms of the stack that runs on top of it. There's no single application or single vendor that kind of drives value. So an acquisition strategy is >>So your field Salesforce has direct or indirect, both mixable. How do you handle the, because Cloudera has got feet on the street and every squirrel will find it, not if they're parked there, parking sales reps and SCS and all the enterprise accounts, you know, they're going to get the, squirrel's going to find a nut once in awhile. Yeah. And they're going to actually try to engage the clients. So, you know, I guess it is a strategy if they're deploying sales and marketing, right? So >>The beauty about that, and in fact, we're all in this together in terms of sharing an API and driving an ecosystem, it's not a fragmented market. You can start with one distribution and move to another, without recompiling or without doing any sort of changes. So it's a fairly open community. If this were a vendor lock-in or, you know, then spending money on brand, et cetera, would, would be important. Our focus is on the, so the sales execution of direct sales, yes, we have direct sales. We also have partners and it depends on the geographies as to what that percentage is. >>And John Schroeder on with the HP at fifth big data NYC has updated the HP relationship. >>Oh, excellent. In fact, we just launched our application gallery app gallery, make it very easy for administrators and developers and analysts to get access and understand what's available in the ecosystem. That's available directly on our website. And one of the featured applications there today is an integration with the map, our sandbox and HP Vertica. So you can get early access, try it and get the best of kind of enterprise grade SQL first, >>First Hadoop app store, basically. Yeah. If you want to call it that way. Right. So like >>Sure. Available, we launched with close to 30, 30 with, you know, a whole wave kind of following that. >>So talk a little bit about, you know, speaking of verdict and kind of the sequel on Hadoop. So, you know, there's a lot of talk about that. Some confusion about the different methods for applying SQL on predicts or map art takes an open approach. I know you'll support things like Impala from, from a competitor Cloudera, talk about that approach from a map arts perspective. >>So I guess our, our, our perspective is kind of unbiased open source. We don't try to pick and choose and dictate what's the right open source based on either our participation or some community involvement. And the reality is with multiple applications being run on the platform, there are different use cases that make difference, you know, make different sense. So whether it's a hive solution or, you know, drill drills available, or HP Vertica people have the choice. And it's part of, of a broad range of capabilities that you want to be able to run on the platform for your workflows, whether it's SQL access or a MapReduce or a spark framework shark, et cetera. >>So, yeah, I mean there is because there's so many different there's spark there's, you know, you can run HP Vertica, you've got Impala, you've got hive. And the stinger initiative is, is that whole kind of SQL on Hadoop ecosystem, still working itself out. Are we going to have this many options in a year or two years from now? Or are they complimentary and potentially, you know, each has its has its role. >>I think the major differences is kind of how it deals with the new data formats. Can it deal with self-describing data? Sources can leverage, Jason file does require a centralized metadata, and those are some of the perspectives and advantages say the Apache drill has to expand the data sets that are possible enabled data exploration without dependency on a, on an it administrator to define that, that metadata. >>So another, maybe not always as exciting, but taking workloads from existing systems, moving them to Hadoop is one of the ways that a lot of people get started with, to do whether associated transformation workloads or there's something in that vein. So I know you've announced a partnership with Syncsort and that's one of the things that they focus on is really making it as easy as possible to meet those. We'll talk a little bit about that partnership, why that makes sense for you and, and >>When your customer, I think it's a great proof point because we announced that partnership around mainframe offload, we have flipped comScore and experience in that, in that press release. And if you look at a workload on a mainframe going to duke, that that seems like that's a, that's really an oxymoron, but by having the capabilities that map R has and making that a system of record with that full high availability and that data protection, we're actually an option to offload from mainframe offload, from sand processing and provide a really cost effective, scalable alternative. And we've got customers that had, had tried to offload from the mainframe multiple times in the past, on successfully and have done it successfully with Mapbox. >>So talk a little bit more about kind of the broader partnership strategy. I mean, we're, we're here at Hadoop summit. Of course, Hortonworks talks a lot about their partnerships and kind of their reseller arrangements. Fedor. I seem to take a little bit more of a direct approach what's map R's approach to kind of partnering and, and as that relates to kind of resell arrangements and things like, >>I think the app gallery is probably a great proof point there. The strategy is, is an ecosystem approach. It's having a collection of tools and applications and management facilities as well as applications on top. So it's a very open strategy. We focus on making sure that we have open API APIs at that application layer, that it's very easy to get data in and out. And part of that architecture by presenting standard file system format, by allowing non Java applications to run directly on our platform to support standard database connections, ODBC, and JDBC, to provide database functionality. In addition to kind of this deep predictive analytics really it's about supporting the broadest set of applications on top of a single platform. What we're seeing in this kind of this, this modern architecture is data gravity matters. And the more processing you can do on a single platform, the better off you are, the more agile, the more competitive, right? >>So in terms of, so you're partnering with people like SAS, for example, to kind of bring some of the, some of the analytic capabilities into the platform. Can you kind of tell us a little bit about any >>Companies like SAS and revolution analytics and Skytree, and I mean, just a whole host of, of companies on the analytics side, as well as on the tools and visualization, et cetera. Yeah. >>Well, I mean, I, I bring up SAS because I think they, they get the fact that the, the whole data gravity situation is they've got it. They've got to go to where the data is and not have the data come to them. So, you know, I give them credit for kind of acknowledging that, that kind of big data truth ism, that it's >>All going to the data, not bringing the data >>To the computer. Jack talk about the success you had with the customers had some pretty impressive numbers talking about 500 customers, Merv agent. The garden was on with us earlier, essentially reiterating not mentioning that bar. He was just saying what you guys are doing is right where the puck is going. And some think the puck is not even there at the same rink, some other vendors. So I gotta give you props on that. So what I want you to talk about the success you have in specifically around where you're winning and where you're successful, you guys have struggled with, >>I need to improve on, yeah, there's a, there's a whole class of applications that I think Hadoop is enabling, which is about operations in analytics. It's taking this, this higher arrival rate machine generated data and doing analytics as it happens and then impacting the business. So whether it's fraud detection or recommendation engines, or, you know, supply chain applications using sensor data, it's happening very, very quickly. So a system that can tolerate and accept streaming data sources, it has real-time operations. That is 24 by seven and highly available is, is what really moves the needle. And that's the examples I used with, you know, add a Rubicon project and, you know, cable TV, >>The very outcome. What's the primary outcomes your clients want with your product? Is it stability? And the platform has enabled development. Is there a specific, is there an outcome that's consistent across all your wins? >>Well, the big picture, some of them are focused on revenues. Like how do we optimize revenue either? It's a new data source or it's a new application or it's existing application. We're exploding the dataset. Some of it's reducing costs. So they want to do things like a mainframe offload or data warehouse offload. And then there's some that are focused on risk mitigation. And if there's anything that they have in common it's, as they moved from kind of test and looked at production, it's the key capabilities that they have in enterprise systems today that they want to make sure they're in Hindu. So it's not, it's not anything new. It's just like, Hey, we've got SLS and I've got data protection policies, and I've got a disaster recovery procedure. And why can't I expect the same level of capabilities in Hindu that I have today in those other systems. >>It's a final question. Where are you guys heading this year? What's your key objectives. Obviously, you're getting these announcements as flurry of announcements, good success state of the company. How many employees were you guys at? Give us a quick update on the numbers. >>So, you know, we just reported this incredible momentum where we've tripled core growth year over year, we've added a tremendous amount of customers. We're over 500 now. So we're basically sticking to our knitting, focusing on the customers, elevating the proof points here. Some of the most significant customers we have in the telco and financial services and healthcare and, and retail area are, you know, view this as a strategic weapon view, this is a huge competitive advantage, and it's helping them impact their business. That's really spring our success. We've, you know, we're, we're growing at an incredible clip here and it's just, it's a great time to have made those calls and those investments early on and kind of reaping the benefits. >>It's. Now I've always said, when we, since the first Hadoop summit, when Hortonworks came out of Yahoo and this whole community kind of burst open, you had to duke world. Now Riley runs at it's a whole different vibe of itself. This was look at the developer vibe. So I got to ask you, and we would have been a big fan. I mean, everyone has enough beachhead to be successful, not about map arbors Hortonworks or cloud air. And this is why I always kind of smile when everyone goes, oh, Cloudera or Hortonworks. I mean, they're two different animals at this point. It would do different things. If you guys were over here, everyone has their quote, swim lanes or beachhead is not a lot of super competition. Do you think, or is it going to be this way for awhile? What's your fork at some? At what point do you see more competition? 10 years out? I mean, Merv was talking a 10 year horizon for innovation. >>I think that the more people learn and understand about Hadoop, the more they'll appreciate these kind of set of capabilities that matter in production and post-production, and it'll migrate earlier. And as we, you know, focus on more developer tools like our sandbox, so people can easily get experienced and understand kind of what map are, is. I think we'll start to see a lot more understanding and momentum. >>Awesome. Jack Norris here, inside the cube CMO, Matt BARR, a very successful enterprise grade, a duke player, a leader in the space. Thanks for coming on. We really appreciate it. Right back after the short break you're live in Silicon valley, I had dupe December, 2014, the right back.

>>Winston Edmundson here at Hadoop summit. We've got Steve woolens from Teradata. He's going to talk to me a little bit about a exciting new announcement that you had with Hortonworks today. Tell me a little bit about that. >>Yeah. So Teradata has been in the data management analytics space for over 30 years. And with the announcement today, we announced data portfolio for Hadoop, which is a collection of products, services, and customer support for an entire portfolio for the products. So we've got turnkey appliances, we've got commodity offerings and with Hortonworks, we've got a shared customer support model, so we can give our customers everything they need around >>Ultimate support. Pretty exciting. Now this seems like it must've been a long process to put all this together. >>Well, we've had a partnership with Hortonworks for about a year. We've had Hadoop product offerings in the market for about six months. We've seen a lot of uptake from our customers, and it's really about broadening that to make sure that customers can buy a dupe standalone integrated in with the rest of their data architecture and make it a trusted component within that next generation data architecture. >>Tell me what excites you right now with the customers that you're helping, you're meeting their needs. Where do you see things going? What trends are you following right now? >>The big thing we're seeing is customers. Our customers want to better serve their customers. And there's so many new interaction points that they have with those customers through social networks, email, and being able to take things like the call center voice records, but that's been data that hasn't really been explored in the past to figure out how to better serve those customers. So now with Hadoop and other MapReduce technologies, we can incorporate that analysis into how we better serve our customers, customers at the end of the day. If that makes sense, that's ultimately, it's about getting deeper insights into how to better service the customers. And I think with all the new data that's out there and the hype around big data, that's really what it's about. >>Do you find the customers are coming to you with their own ideas or are they looking to you for suggestions on just how they can bring these different data sets together and how they can maximize and leverage some of this data? >>Well, the problem is there's so much hysteria in this market. I mean, it's an exciting place to be, but there's a lot of technologies, right? So I think the thing with Teradata is we do provide that trusted advisor status. I mean, we've been implementing data analytics solutions for a long, long time and a lot of the problems aren't new, they're just incorporating new analytics techniques. So they have ideas in terms of things they've heard about. They're not really sure how to implement it sometimes. So part of our offering is we have services, so we can look across their entire data architecture and figure out where does the dupe really fit? What are the best use cases for it? How do we integrate that across the enterprise? So the end users and the applications that can benefit from that data can really get the value from it. >>How important do you think it is or how much is an advantage that you are tried and true. You've been here. I mean, some of these solution providers, you can call them fly by night. I mean, they just, they're just here on, you know, they've just formed. They don't have a track record. It's your track record of success? One of the main things that customers are attracted to? >>I think so. I mean, the reality is we have, we're like in the trenches with our customers, it's not just the technology, but when we have business consulting, people that come in with domain expertise from a given industry, so you can call it a track record or whatever it is, but it's really understanding, not just technology, but the business and how these things come together to really get the most value from all the cool technology that's out there. So yeah, a lot of the fly by nighters, I mean, there's a lot of innovative things that are happening. And at duke five years ago, it was one of those very new things. And so we've been looking at it for a while and now we figured out the best way to incorporate it into our solution portfolio and to roll it out to customers >>When you're helping a customer. And you're, you're looking at the here and now, this is what they, they need to be addressing. I would imagine a lot of customers want to know what's around the corner, what's around the bend that we should be aware of, that we should try to be, be prepared for. What do you, what do you tell them? >>Well, I think, you know, everybody will say there's just more and more data coming at you. I think other analytic techniques like graph analysis is something that people particularly with social networks are trying to figure out how are people interrelated to each other. So it's a lot of different use cases and there's different analytic techniques that can be combined in unique ways. So a lot of our R and D investment is going into how do we bring more of those analytic techniques and unify them for people in one system. So that regardless of your data scientists or business analysts, you can ask really interesting, tough questions that you couldn't answer ask before. So it's about giving answers to sometimes the unknown questions and helping them explore that data through unique ways. >>What would you say are some of the industries that are maybe there's probably more urgency for them to adopt some of these strategies or perhaps just, they're more likely to have a big return on investment? What industries would you point to? >>I mean, for us, it's a lot of the traditional industries where you have a lot of consumers, right? Telecommunications, retail, retail, financial services, anybody who's working with. A lot of customers that have a lot of products, just have a lot of complexity, a lot of customer interaction touchpoints. So I think those are the people that typically we see adopting new technology and really thinking about how to better serve their customers >>For folks that are watching tuning in. And they're pretty excited about what you might be able to help them with. What's the best way for them to get in touch with you or, or >>You just go to teradata.com and check us out there. That's probably the best way to reach us. >>Right. Fantastic. Thanks for your time. Winston Edmondson here with studio B signing out.

live at hadoop summit this is SiliconANGLE and wiki bonds exclusive coverage of hadoop summit this is the cube our flagship program would go out the advanced extract the signal from the noise i'm to enjoy my co-host Jeff Kelly Jeff welcome to the cube Scott welcome to the cube great to have you here so you kicked off help kick off the show this morning with your keynote talking about a number of things among them the new teradata plans for Hadoop brought it on stage which I thought was great i love i love some i was joined by a dancing appliance okay great it was fantastic a good-looking appliance it was but why don't you tell us a little bit about yourself kind of your role and then we'll kind of get into what tara date is doing here at the show and some of the some of the strategies you're taking towards the big data market okay great well I'm Scott now I'm from tarde de labs and turny two labs is actually organization within teradata that is responsible for research development engineering product management product marketing all the products all of the technology that we roll out kind of the innovation engine of teradata is what we're responsible for and we've been obviously affiliated with hadoop summit we were here last year it's really great to be back having been in the in the data warehouse big data kind of data analytics business for a long time the one thing I have to say about this whole movement in the Hadoop space is that it's unlike anything else I've seen in that it's every geography it's every industry and there's so much energy and emotion around it's unlike any other transition that I've seen and even the difference between our visit here last year and this year where we've seen the the promise turned into reality where we've got customers who are implementing where we've got businesses who are driving value from the solutions that they're really that they're integrating with the solutions that they've already got and and being able to demonstrate that value really emphasizes the importance and I think will help to continue the momentum that we feel in this market Scott one of the things I want to ask you was obviously the theme at had dude was off loading data warehouses what they do is a benefit there but you have a relationship with Hortonworks and we've had we were talking early with Murph was an analyst at Gartner was talking about the the early adopters and the mainstream getting it now and but there's always a question of value right where's the value because his legacy involved right so the most of the web based companies are going to be cloud they'll be SAS they might have a Greenfield clean sheet of paper to work with on big data but an existing enterprise large financial institutions insurance company or what have you they have legacy technology and they have to but they want Hadoop they want to bring it in when you talk to folks out there what are some of the challenges and opportunities they have with that environment and the technology specifically sure that was like a long question there's a lot of a lot of threads in there I want to really try to hit on a couple of important themes because you know you hear it here I get asked a lot about it you know one of the things that people often say is you know this why are you here this whole Hadoop thing is offloading data warehouses isn't that bad doesn't that bother you and the answer is absolutely not certainly there's some hype around that and you know those some marketing around that but when you really look at the technology and the value of what it brings to the table it's a new technology that really allows us to harness new kinds of data and store those new kinds of data in the native format and you know storing detailed data in the native format really enables the best world-class analytics we've seen this happen for you know as long as my career is in the traditional data space so that's a really good thing the way I view it though is sure will some work load move around the infrastructure from the data warehouse to a Hadoop cluster potentially right and by the way if Hadoop is a great solution for it it should go there all right but at the same time there is more demand than there is supply of technology and what I mean by that is the demand for analytics is so extreme that actually adding this tool to the toolkit gives customers more choice and gives them the opportunity to really catch up with the backlog of things that they've wanted to invest in overtime and then the final point really I view what's happening here as perhaps one of the single largest opportunities for expansion of the role and size and scope of the data warehouse in an enterprise because one of the big things that Hadoop brings to the table is a whole lot of raw material a whole lot more data data that used to be thrown away data that never existed a year ago is now going to be able to capture be captured be stored be refined be analyzed and as companies start to find relationships as companies start to find actionable tidbits from the analytics in this huge source of raw material I think it's actually an opportunity for upside for them to integrate more data into their data warehouse where they can actually do the real-time interaction and streaming that's going to get them to the demonstrable business benefit so it's the modernization of the enterprise it's its modernization the way I look at it is also it's sometimes the word incremental can be it can sound like it we're trying to downplay it but I see it as incremental in that it's different data and it's incremental data it's incremental subject areas its new stuff that's going to come into the environment and based on what we've seen in the history of analytics right that there's no end to the value that companies find and there's no end to competition in their businesses so this is a huge opportunity for the entire community to deliver more analytics and i think that there's actually more upside for traditional legacy data warehouse vendors and there is anything I think that's a really important point because as you said a lot of people think about that offloading workloads but it's also about offloading we're close but bringing in new data doing more analytics and then moving some of that into back into the data warehouse you can actually create more value from it yeah I mean one of the things that I've seen is you know over time and Moore's law is something that's been going on for some time right and and cost erosion in Hardware has been going on for a long time and you think about the thing that you buy today for your bi implementation the hardware costs what twenty percent of what it costs three four years ago and you know what revenues continue to increase because they're such pent-up demand that as it gets less expensive it becomes more consumable and I think the same thing it's really going to continue to happen as we add in these new technologies and these new data types so one of the things I want to commend teradata for doing is focusing on kind of that reference protector and helping customers understand how this new technology of Hadoop and big data fits in with everything else that they're doing talk a little bit a bit about how from a reference architecture and then maybe even from a product perspective how teradata goes about turning this into a reality for enterprise customers who you know really you know they're not looking to just kick the tires of the Duke they want they want to use this for its really support you know applications and workflows they're really you know critical to their business yeah I think you know one of the biggest things that we can do to help the industry and to help our customers really is to define a realistic roadmap that's consumable for them in their enterprise and so while it's certainly easy to have marketing release or press release it says uh this new technology does everything in slices bread it washes your car does all these things in reality there are very few things like that in the world right but the new technologies and the new innovations really do fit into some very interesting new use cases and so by providing this integrated roadmap of how customers can deploy and fit these technologies together is a really great education process and it's been extremely well received by our customers and prospects I have to tell you that even in advance of the announcement of the things that we had here today we've already got customers who have gone down this path with us because it's such a compelling value proposition the other thing is that we don't actually put specific technology in those boxes it's a reference architecture we hope that there's some teradata product in there but at the same time we you know our customers understand that there is choice in the marketplace and the best solution is going to win and by providing this reference architecture I think we helped elevate ourselves to more of a trusted advisor status with with the the industry and in how we see these things fitting together and providing very effective very low-risk kinds of solutions well I think you hit on something that trusted advisor I think companies and enterprises are just crying out for some leadership and to help to help them really understand how they're going to make this a reality in their organizations and you know you mentioned kind of the openness and being you know allowing enterprises shoots a technology that fits that fits the the work case of course you know you hope that stared at in a lot of cases but it could be something else so talk a little bit about your relationship with hortonworks so I know you announced today kind of a reseller agreement you're going to be actually reselling the the subscription service to Hortonworks service offering talk about that a little bit and also I want to dive into the tech as well the Hadoop appliance I mentioned earlier like you announced and maybe just kind of walk us through some of the news to them sure so I mean obviously we have a strategic relationship with Hortonworks and it's our second year here at Summit and it really started with I think a very common view of what's happening in the marketplace and how these technologies should really play well together at the same time we also really believe that it's important that the community embrace the open source Apache version of the software so that it doesn't become fragmented and become obsolete right so Horton is spot-on in terms of business model and putting everything back into the Apache open source version so that means that I think this is the version that will win and this will be the version that companies can count on to be sustainable so i think that there's an advantage there implied so that's said i think it fits into the right place we've got a great engineering relationship and a great common vision on how the enterprise architecture and how the pieces can fit together and be optimized for different workloads for different service levels and for different applications so having that common vision and kind of I think bringing to Best of Breed providers together with Wharton works on the on the Hadoop side and teradata for what we're very well known for I think it's really the best of all worlds and we work together to lay out this reference architecture and so it's not just you know tur data came down from the mountain said this should be your reference architecture we've got some validation we got some validation of use cases and then we went to work from an engineering perspective on how we go build these things out and make them work and optimize them and support them end to end because obviously not only in you know with the all of the new solutions is their kind of a scarcity of talent and some confusion support becomes really really important so one of the things we added to our portfolio we announced today is an expanded relationship on the support side where customers can come to teradata for integrated support of all of their data analytics environments whether it be teradata whether it be asked her whether it be Hadoop with hdb and you know that's a really nice thing where there's one phone number to call we've got fully integrated processes we can help with a global footprint in the 80 countries where we do business and obviously Hortonworks with the with the extreme depth and ability to manage the content of the kernel can get it done unlike anyone else Scott we've been talking enterprise-grade all morning as you did those the theme of the keynote mer from our garden about security compliance I mean these are meat and potatoes enterprise issues right so I got to ask you what's what are you guys looking at what's what's coming next obviously the platform to do has a stabilized developers going to want to program on it in different environments but the reality in the enterprise is a certain requirement so what are you looking at in the labs that's coming around the corner that's it going to be really really important for customers to realize the value of scaling and harnessing the big data of Hadoop with the existing infrastructure yeah I mean I think there are two things that will continue to do one is will look to build out kind of that framework of ecosystem and in all of the keynotes this morning you know everyone talked about the value of the ecosystem and it's amazing the ecosystem how they're just more and more logos this year than there were last year and I think that that will continue but really building out that ecosystem so that those things that are important can be realized and they can be realized in a very repeatable fashion I think in addition to that kind of ease of use right because despite the fact that we have burgeoning numbers of newly minted data scientists and people getting into the marketplace that's really good there still aren't enough and so de-risking things by making them easier to deploy and easier to support i think is a key focus area and then you know finally I said two things but now third you know finally it will say to me I'd all right we'll continue to look at performance and just making sure that we have the best density the best performance the cost performance value proposition that our customers will want because I also continue to believe that the supply of data will outstrip any customers ability to invest in infrastructure I'd love to get your take on want to go back to mention to what you mentioned about the you know the Hadoop distribution focusing on a patchy and moving a patchy compatible so I take that number one to me and Tara day is not going to be coming out with their own Hadoop distribution absolutely not but how do you think about that yeah I think we can say that pretty definitively so but what about how do you see this whole Hadoop market playing out them you've got a Hortonworks Cloudera map are some others how do you see this playing out in the next year or so I mean is this you mentioned you think again that's kind of the open source of patchy versions going to kind of win when do you think that's going to happen you've got some competitors in the market and different business models hot yeah you know there are different business models and different innovators and you know my crystal ball is probably only about as clear as anyone elses but you know kind of for the long term I think it's best for the industry if if it mimics a model similar to the way Linux is deployed where this kind of a duopoly maybe three vendors it's very largely open source there's a lot of portability between I think that really strengthens the position of Hadoop as a tech as a core technology and foundation for some of the things that we're doing and so I would hope that in you know the most successful outcome would be that we'd end up with a duopoly or or you know maybe three kind of providers around a similar colonel because that would that would remove fragmentation from the market by the way I think it you know where we are software company so I think it's fair for companies to have value add proprietary software that's not a bad thing but at the file system level at a core two level I think the open source community cannot be out innovated right and and so I think that that's a really important thing so I think you know hopefully we'll get to that duopoly or maybe three companies that kind of have that I don't know if we will but I sure hope we do and I think the if I were to bet on it I would say it's odds on that that will be the case now will that be 18 months three years five years I don't know Scott thanks for coming inside the cube obviously you guys have a great position in the market place and the enterprise message is straw here that's what the demand is we're seeing a lot of trends out there that want the enterprise grade big data which is not just once there's but Hadoop's a big part of it Thanks coming inside the cube and sharing your perspective and what you got working on certainly having the new products come out to be great so thanks for coming onto the cube this is SiliconANGLE and wiki bonds coverage of hadoop summit we'll be right back with our next guest after this short break you

>>Ash it's, you know, what will that mean to my investment? And the announcement fusion IO is that, you know, we're 25 times faster on read intensive HBase applications. The combination. So as organizations are deploying Hadoop, and they're looking at technology changes coming down the pike, they can rest assured that they'll be able to take advantage of those in a much more aggressive fashion with map R than, than other distribution. >>Jack, how I got to ask you, we were talking last night at the Hadoop summit, kind of the kickoff party and, you know, everyone was there. All the top execs were there and all the developers, you know, we were in the queue. I think, I think that either Dave or myself coined the term, the big three of big data, you guys ROMs cloud Cloudera map R and Hortonworks, really at the, at the beginning of the key players early on and Charles from Cloudera was just recently on. And, and he's like, oh no, this, this enterprise grade stuff has been kicked around. It's been there from the beginning. You guys have been there from the beginning and Matt BARR has never, ever waffled on your, on your messaging. You've always been very clear. Hey, we're going to take a dupe open source a dupe and turn it into an enterprise grade product. Right. So that's clear, right? That's, that's, that's a great, that's a great, so what's your take on this because now enterprise grade is kind of there, I guess, the buzz around getting the, like the folks that have crossed the chasm implemented. So what can you comment on that about one enterprise grade, the reality of it, certainly from your perspective, you haven't been any but others. And then those folks that are now rolling it out for the first time, what can you share with them around? What does it mean to be enterprise grade? >>So enterprise grade is more about the customer experience than, than a marketing claim. And, you know, by enterprise grade, what we're talking about are some of the capabilities and features that they've grown to expect in their, their other enterprise applications. So, you know, the ability to meet full S SLA is full ha recovery from multiple failures, rolling upgrades, data protection was consistent snapshots business continuity with mirroring the ability to share a cluster across multiple groups and have, you know, volumes. I mean, there's a, there's a host of features that fall under the umbrella enterprise grade. And when you move from no support for any of those features to support to a few of them, I don't think that's going to, to ha it's more like moving to low availability. And, and there's just a lot of differences in terms of when we say enterprise grade with those features mean versus w what we view as kind of an incomplete story. So >>What do you, what do you mean by low availability? Well, I mean, it's tongue in cheek. It's nice. It's a good term. It's really saying, you know, just available when you sometimes is that what you mean? Is this not true availability? I mean, availability is 99.9%. Right? >>Right. So if you've got a, an ha solution that can't recover from multiple failures, that's downtime. If you've got an HBase application that's running online and you have data that goes down and it takes 10 to 30 minutes to have the region servers recover it from another place in the distribution, that's downtime. If you have snapshots that aren't consistent across the cluster, that doesn't provide data protection, there's no point in time recovery for, for a cluster. So, you know, there's a lot of details underneath that, but what it, what it amounts to is, do you have interruptions? Do you have downtime? Do you have the potential for losing data? And our answer is you need a series of features that are hardened and proven to deliver that. >>What about recoverability? You mentioned that you guys have done a lot of work in that area with snapshotting, that's kind of being kicked around, are our folks addressing, what are the comp what's your competition doing in those areas of recoverability just mentioned availability. Okay, got that. Recoverability security, compliance, and usability. Those are the areas that seem to be the hot focus areas what's going on in the energy. How would you give them the grade, the letter grade, if you will, candidly, compared to what you guys offer? Well, the, >>The first of all, it's take recoverability. You know, one of the tenants is you have a point in time recovery, the ability to restore to a previous point that's consistent across the cluster. And right now there's, there's no point in time recovery for, for HDFS, for the files. And there's no point in time recovery for HBase tables. So there's snapshot support. It's being talked about in the open source community with respect to snapshots, but it's being referred to in the JIRAs as fuzzy snapshots and really compared to copy table. >>So, Jack, I want to turn the conversation to the, kind of the topic we've talked about before kind of the open versus a proprietary that, that whole debate we've, we've, we've heard about that. We talked about that before here on the cube. So just kind of reiterate for us your take. I mean, we, we hear perhaps because of the show we're at, there's a lot of talk about the open source nature of Hadoop and some of the purists, as you might call them are saying, it's gotta be open a hundred percent Patrick compatible, et cetera. And then there's others that are taking a different approach, explain your approach and why you think that's the key way to make, to really spur adoption of a dupe and make it >>W w we're we're a part of the community we're, we've got, you know, commitment going on. We've, you know, pioneered and pushed a patchy drill, but we have done innovations as well. And I think that those innovations are really required to support and extend the, the whole ecosystem. So canonical distributes RN, three D distribution. We've got, you know, all our, our packages are, are available on get hub and, and open source. So it's not, it's not a binary debate. And I think the, the point being that there's companies that have jumped ahead and now that Peloton is, is, you know, pedaling faster and, and we'll, we'll catch up. We'll streamline. I think the difference is we rearchitected. So we're basically in a race car and, you know, are, are racing ahead with, with enterprise grade features that are required. And there's a lot of work that still needs to be done, needs to be accomplished before that full rearchitecture is, is in place. >>Well, I mean, I think for me, the proof is really in the pudding when you, when it comes to talk about customers that are doing real things and real production, grade mission, critical applications that they're running. And to me that shows the successor or relative success of a given approach. So I know you guys are working with companies like ancestry.com, live nation and Quicken loans. Maybe you could, could you walk us through a couple of those scenarios? Let's take ancestry.com. Obviously they've got a huge amount of data based on the kind of geological information, where do you guys do >>With them? Yeah, so they've got, I mean, they've got the world's largest family genealogy services available on the web. So there's a massive amount of data that they make accessible and, and, you know, ability for, for analysis. And then they've rolled out new features and new applications. One of which is to ship a kit out, have people spit in a tube, returned back and they do DNA matching and reveal additional details. So really some really fabulous leading edge things that are being done with, with the use of, of Hadoop. >>Interesting. So talk about when you went to, to work with them, what were some of their key requirements? Was it around, it was more around the enterprise enterprise, grade security and uptime kind of equation, or was it more around some of the analytics? What, what, what's the kind of the killer use case for them? >>It's kind of, you know, it's, it's hard with a specific company or even, you know, to generalize across companies. Cause they're really three main areas in terms of ease of use and administration dependability, which includes the full ha and then, and then performance. And in some cases, it's, it's just one of those that kind of drives it. And it's used to justify, in other cases, it's kind of a collection. The ease of use is being able to use a cluster, not only as Hadoop, but to access it and treat it like enterprise storage. So it's a complete POSIX compliance file system underneath that allows the, the mounting and access and updates and using it in dynamic read-write. So what that means from an application level, it's, it's faster, it's much easier to administer and it's much easier and reliable for developers to, to utilize. >>I got to ask you about the marketing question cause I see, you know, map our, you guys have done a good job of marketing. Certainly we want to be thankful to you guys is supporting the cube in the past and you guys have been great supporters of our mission, but now the ecosystem's evolving a lot more competition. Claudia mentioned those eight companies they're tracking in quote Hadoop, and certainly Jeff and I, and, and SiliconANGLE by look at there's a lot more because Hadoop washing has been going on now for the term Hadoop watching me and jumping in and doing Hadoop, slapping that onto an existing solution. It's not been happening full, full, full bore for a year. At least what's the next for you guys to break above the noise? Obviously the communities are very active projects are coming online. You guys have your mission in the enterprise. What's the strategy for you guys going forward is more of the same and anything new even share. >>Yeah, I, I, I think as far as breaking above the noise, it will be our customers, their success and their use cases that really put the spotlight on what the differences are in terms of, of, you know, using a big data platform. And I think what, what companies will start to realize is I'd rather analogy between supply chain and the big, the big revolution in supply chain was focusing on inventory at each stage in the supply chain. And how do you reduce that inventory level and how do you speed the, the flow of goods and the agility of a company for competitive advantage. And I think we're going to view data the same way. So companies instead of raw data that they're copying and moving across different silos, if they're able to process data in place and send small results sets, they're going to be faster, more agile and more competitive. >>And that puts the spotlight on what data platform is out there that can support a broad set of applications and it can have the broadest set of functionality. So, you know, what we're delivering is a mission grade, you know, enterprise grade mission, critical support platform that supports MapReduce and does that high performance provides NFS POSIX access. So you can use it like a file system integrates, you know, enterprise grade, no SQL applications. So now you can do, you know, high-speed consistent performance, real time operations in addition to batch streaming, integrated search, et cetera. So it's, it's really exciting to provide that platform and have organizations transform what they're doing. >>How's the feedback on with Ted Dunning? I haven't seen a lot of buzz on the Twittersphere is getting positive feedback here. He's a, a tech athlete. He's a guru, he's an expert. He's got his hands in all the pies. He's a scientist type. What's he up to? What's his, what's his role within Mapa and he's obviously playing in the open-source community. What's he up to these days, >>Chief application architect, he's on the leading edge of my house. So machine learning, so, you know, sharing insights there, he was speaking at the storm meetup two nights ago and sharing how you can integrate long running batch, predictive analytics with real-time streaming and how the use of snapshots really that, that easy and possible. He travels the world and is helping organizations understand how they can take some very complex, long running processes and really simplify and shorten those >>Chance to meet him in New York city had last had duke world at a, at a, a party and great guy, fantastic geek, and certainly is doing a great work and shout out to Ted. Congratulations, continue up that support. How's everyone else doing? How's John and Treevis doing how's the team at map are we're pedaling as best as you can growing >>Really quickly. No, we're just shifting gears. Would it be on pedaling >>Engine? >>Yeah. Give us an update on the company in terms of how the growth and kind of where you guys are moving that. >>Yeah. We're, we're expanding worldwide, you know, just this, you know, last few months we've opened up offices and in London and Munich and Paris, we're expanding in Asia, Japan and Korea. So w our, our sales and services and engineering, and basically across the whole company continues to expand rapidly. Some really great, interesting partnerships and, and a lot of growth Natalie's we add customers, but it's, it's nice to see customers that continue to really grow their use of map are within their organization, both in terms of amount of data that they're analyzing and the number of applications that they're bringing to bear on the platform. >>Well, that a little bit, because I think, you know, one of the, one of the trends we do see is when a company brings in big data, big data platform, and they might start experiment experimenting with it, build an application. And then maybe in the, maybe in the marketing department, then the sales guys see it and they say, well, maybe we can do something with that. How is that typically the kind of the experience you're seeing and how do you support companies that want to start expanding beyond those initial use cases to support other departments, potentially even other physical locations around the world? How do you, how do you kind of, >>That's been the beauty of that is if you have a platform that can support those new applications. So if you know, mission critical workloads are not an issue, if you support volumes so that you can logically separate makes it much easier, which we have. So one of our customers Zions bank, they brought in Matt BARR to do fraud detection. And pretty soon the fact that they were able to collect all of that data, they had other departments coming to them and saying, Hey, we'd like to use that to do analysis on because we're not getting that data from our existing system. >>Yeah. They come in and you're sitting on a goldmine, there are use cases. And you also mentioned kind of, as you're expanding internationally, what's your take on the international market for big data to do specifically is, is the U S kind of a leaps and bounds ahead of the rest of the world in terms of adoption of the technology. What are you seeing out there in terms of where, where the rest of the, >>I wouldn't say leaps and bounds, and I think internationally, they're able to maybe skip some of the experimental steps. So we're seeing, we're seeing deployment of class financial services and telecom, and it's, it's fairly broad recruit technologies there. The largest provider of recruiting services, indeed.com is one of their subsidiaries they're doing a lot with, with Hadoop and map are specifically, so it's, it's, it's been, it's been expanding rapidly. Fantastic. >>I also, you know, when you think about Europe, what's going on with Google and some of the, the privacy concerns even here, or I should say, is there, are there different regulatory environments you've got to navigate when you're talking about data and how you use data when you're starting to expand to other, other locales? >>Yeah. There's typically by vertical, there's different, different requirements, HIPAA and healthcare, and basal to, and financial services. And so all of those, and it, it, it basically, it's the same theme of when you're bringing Hadoop into an organization and into a data center, the same sorts of concerns and requirements and privacy that you're applying in other areas will be applied on Hindu. >>I'm now kind of turning back to the technology. You mentioned Apache drill. I'd love to get an update on kind of where, where that stands. You know, it's put, then put that into context for people. We hear a lot about the SQL and Hadoop question here, where does drill fit into that, into that equation? >>Well, the, the, you know, there's a lot of different approaches to provide SQL access. A lot of that is driven by how do you, how do you leverage some of the talent and organization that, you know, speak SQL? So there's developments with respect to hive, you know, there's other projects out there. Apache drill is an open source project, getting a lot of community involvement. And the design center there is pretty interesting. It started from the beginning as an open source project. And two main differences. One was in looking at supporting SQL it's, let's do full ANSI SQL. So it's full 2003 ANSI, sequel, not a SQL like, and that'll support the greatest number of applications and, you know, avoid a lot of support and, and issues. And the second design center is let's support a broad set of data sources. So nested sources like Jason scheme on discovery, and basically fitting it into an enterprise environment, which sometimes is kinda messy and can get messy as acquisitions happen, et cetera. So it's complimentary, it's about, you know, enabling interactive, low latency queries. >>Jack, I want to give you the final word. We are out of time. Thanks for coming on the cube. Really preached. Great to see you again, keep alumni, but final word. And we'll end the segment here on the cube is your quick thoughts on what's happening here at Hadoop world. What is this show about? Share with the audience? What's the vibe, the summary quick soundbite on Hadoop. >>I think I'll go back to how we started. It's not, if you used to do putz, how you use to do and, you know, look at not only the first application, but what it's going to look like in multiple applications and pay attention to what enterprise grade means. >>Okay. They were secure. We got a more coverage coming, Jack Norris with map R I'll say one of the big three original, big three, still on the, on the list in our mind, and the market's mind with a unique approach to Hadoop and the mid-June great. This is the cube I'm Jennifer with Jeff Kelly. We'll be right back after this short break, >>Let's settle the PR program out there and fighting gap tech news right there. Plenty of the attack was that providing a new gadget. Let's talk about the latest game name, but just the.

>>Come back here. This is Silicon Valley coverage of ADU Summit. I'm John Fur, the founder. We're, we're pleased to have a friend inside the cube. It's rare to have such luminaries, Ama Aala, good friend and also co-founder of Cloudera. Really the pioneer in the space that helped build this industry that we're living here at at Hadoop Summit. I'm with Dave Ante from wiba.org. Amour, welcome back to the Cube Cub alumni. Thank you for having me here. Wow, what a journey. Are you co-founded Cloudera? I remember when you in Stealth Mo, I really can't talk about it. And, and then of course the history of Silicon Angle being, you know, founded and kind of built in in your office when you only had like 20 something employees. Yep. We owe a great deal of gratitude to you and, and congratulations to you Michael Olson, the team for building an industry. So I just wanted Thank you. Thank you. And welcome to the Cube. >>Thank you. It was great to be here. >>So what do you think, what's your take on the current Hadoop ecosystem right now? I mean, obviously a lot's happened. I mean it's big now. It's growing up fast. Yeah. The word enterprise grade is out there. You're seeing it move from, you know, trying to change the world. Our first interview, you said, I've seen the future, I want to bring it to the mainstream. It's here. Yeah. It's hitting mainstream right now. Yeah. What's your take of the current situation of the ecosystem and it's, and its value? >>Yeah, so I, I have a quick question first. Should I look to you or look to the camera? Look to >>The camera or both? Whatever you, whatever you'd like. >>So I think it's, the ecosystem is definitely growing, which is very, very healthy. However, there is a side question there, which is what do you think of all the competition coming into the space? So five years ago when Cloudera was started was just Cloudera. There was no other commercial vendor trying to support or enable Hadoop in the, in the industry for enterprises. And today there is at least 10 of them trying to compete with us, right? And that includes big companies, established companies that decided, hey, we gonna start addressing the space, but includes many, many newcomers who like Hortonworks, who were founded over the last couple of years. That's a healthy thing. I mean, that's absolutely a sign of a growing market. If the market wasn't growing, if there wasn't money in the market, if there wasn't, if it was just hype, there wouldn't have been all of these new companies and new ventures showing up. That said, I never look at competition as something that worries me, that I'm afraid now or what's gonna happen to me, or that's normal. That's exactly what happens to successful companies. If you look at Red Hat, when Red Hat was launching with the Linux, they had 25 competitors or even more 30 competitors. That's when Red Hat was forming out. And today, even of these 25, 30 competitors, they still have six or seven still left. So I think it's a very, very healthy sign of the graph of this market and the maturity that's reaching. >>What do you think about some of the, the white spaces that are evolving? You guys have obviously been involved in a lot of deployments at Cloudera. Again, you're doing a lot of, lot of work with the top, top names and the clients that you have aren't usually disclosed cuz you really can't disclose them. What, what are you seeing right now as the white spaces for things to do in the Hado platform? >>It's a very, very good question. So first I can't talk about future, future roadmap. Right now we're becoming a big company at that level where we can't comment on future roadmaps. >>Ah, that's sinus sign of the >>Time. You're well media train, good to see they're doing a good job keeping you >>A, You want more information on that? I can connect you with a pt, >>Please. No, no, no, we're good. We're good. We'll get it outta you. But, >>But our vision, our vision for Cloudera from day one, like you were saying earlier, we saw the future, right? So our vision from from day one was really to build this data system where we can have detail of any type, whether that data is structured or unstructured or images, it doesn't matter. And then on top of that data run any type of workloads. That workload could be the initial genesis of Hado, which is map use, which is batch processing. But now as as we made many announcements through the last few years, we also now have Impala for interactive analytics as a workload. We have a very, very strong partner partnership with SaaS for doing machine learning and statistics as a workload. And a few weeks ago we announced search as another workload. So you have multiple types of workloads that can handle different types of problems that you have within your organization and bring all of these workloads to all of your data regardless of type. And that's the vision that we'll continue to deliver on. That's exactly what we're building going into the >>Future. So how's that fit in with yarn, right? We're hearing a lot at this conference about yarn, the ability to, you know, do more with less in a lot of the things that you typically hear with the enter within the enterprise. And, and so talk about that a little bit. >>Yarn is a very core part to our platform. In fact, yarn has been part of CDH four for more than a year now out in the, in the markets. So we did bring, we were one of the, I think we were the first vendor who brought yarn into a distribution of Hado out there. It's very, very fundamental to us because that is how we're gonna coordinate. We are gonna be using yarn to coordinate launching all of these different type of workloads. You're gonna have the map produce workload, which is very batch oriented. The Impala workload, which is very latency sensitive. The, the search workload, which is also very latency sensitive. The machine learning workload, which is more batch oriented, et cetera, et cetera. And yarn is a very, very central piece to helping us coordinate all of these different types of workloads onto the >>Platform. Cloudera has been a great citizen in the community also. You, you mentioned and, and we witnessed that your team create the industry. You guys were there, you took the chance, you were the first ones commercially funded by the venture capitalists, you know, then others will follow and I'll see huge ecosystem here. Yes. A lot of noise. A lot of people trying to get attention. So I got to ask you, because I want you to address this because I know it's been talked about in some of the other blogs is there's a lot of fud going on around who's doing what? Who's doing what, and in some cases maybe flat out, you know, misinformation and that happens in a growing market, you know, the elbows get sharp. Yes. So I want you share with the audience anything that you want say about the fud around what people say about Cloudera or about others or what you're doing. Just to clarify, cuz there has been, I mean I've gotten back channel information around, you know, not sure the committers this, and it's been, it's been well documented. There's a lot of fu out there. What, what would you say to the folks out there to clarify >>That? Yes, I, I would say that our focus should be to continue to work as a community, to push the platform forwards. I would say that at Cloudera we do a lot of contributions. Horton works definitely is one of the top contributors out there as well. I'll acknowledge that. So as many, many, many other companies and we wanna continue to see the platform evolve. I will stress though that at Cloudera we do have a number of the original project founders working at the company. So it's not just the, the contribution that we bring, but the fact that we have the founders of these projects working at Cloudera. And some of these projects actually were created at Cloudera from day one as opposed to created in some other company. And then you hire the employee and they work for you. So I gave you what examples from Cloudera dot cutting. >>He is the creator of Hudu dot Cutting is also the creator of Luine, which became solar, which is part of the search project that we launched recently. Dot Cutting wasn't with Cloudera from day one, right? So, so when he created these technologies, he actually was at Tia for example, when he created had he was at ta, wasn't at Cloudera. However, he now works for Cloudera. So we get that because now that cutting works for Cloudera. So that's one example. On the flip side, there is projects like Flume and Scoop that are now part of every single distribution out there. And flu and Scoop were both created at Calera. They were actually created inside of Cloudera. Yeah. So the key point is, and and that's what I would like all of the vendors out there that are trying to leverage had and get benefit about out Hadoop is please don't be just takers. >>There are some vendors out there who are just takers. Just wanna take from the open source, take from the open source and don't give back. Right? I'm not gonna name them, but there is a few of them out there. Please, please, please. I mean that that, that is very, very a selfish behavior. It's not gonna help the ecosystem in the long term. We would like to see you both take and give at the same time. So that would be my core message. And that's for example, like I thank Hortonworks because that's exactly what Hortonworks is doing. They're both giving and taking at the same >>Time. You guys have always been clear on that. Nobody, I mean here contribution to open source has been well documented and there's, there's no question about that. John and I have talked about it a lot that you guys help get it all started. And even Haak when we had 'em on a couple years ago, when Horton Works came to the market said, Hey, the more people work on an open source, the better. >>Yeah, >>Exactly. So yeah, it's always been, been your posture. You're not playing games there. Anyways, having said that, you you, you have a strategy to layer on top of that open source some of your own proprietary code. And so you have choices to make Yes. In terms of how you allocate those resources. So as an engineering manager, how do you allocate those resources in terms of, okay, what do we do for the community and what do we do for our own, you know, future because of the business model that we chose? How do you make those trade offs? >>Yes, that's a very, very good question. So first it's important to stress that our core platform, CDH, is open source. Everything we put in the core platform is open source. So for example, in Palo, which we launched very recently as a ga, now we launched beta last year, but now's ga is a hundred percent Apache license, a hundred percent open source search, which we announced very recently is also open source. So the platform itself, we're committing to everything in there to be open source. Now we believe fundamentally just from having lots of history in studying the open source markets from our ceo Mike Olson himself being one of the very first open source people in the world with, with sleepy cats, the company that he sold to Oracle before founding Cloudera from our investors, helping many other open source companies. To have a successful open co open source company, you need to have a very good engine between the business model that generates revenue and between the product that you are creating. If you don't have a good feedback loop there between these two, you won't be able to sustain the innovation to continue to push the, the boundaries of how good the product is. So we strongly believe in that if you are, if your product is literally a hundred percent open source, meaning both the management and every, there is nothing proprietary whatsoever inside of your products. I can't tell what that is. It's >>Taking a picture. >>Oh, sorry, I thought somebody was waiting >>For me. >>Sorry about that. >>It's a cheap signal. >>It >>Was like a's really good. >>I thought it's like a card of paper with some writing. You, >>You, you have a fan fans out there. They're storming the, the concert here. >>Okay, that's, that's good to hear. That's good to hear. Sorry about that interruption. So if, if, if you have everything a hundred percent open source, that creates two problems. First you have no differentiation whatsoever, meaning another big corporation without naming who the big corporations could be, we just can take everything you do, literally every single bit of source code you have and say, Hey, we can do it too. Come to us, don't work with those guys. Right? We have the latest, greatest things that they have. Why do you wanna continue to work with them? So no, no differentiation is number one, which is very dangerous. And number two, when it becomes, if, if it's a hundred percent open source and there is lots of other vendors able to take the art, the open source artifact and work with it, then it becomes now purely about maintenance and insurance on the products, which is a commodity product, which obviously the prices for that will go down to the ground and you won't be able to have this sustain this positive feedback effect between your business model and between your product code map and won't be able to build a long-lasting company. >>So that's why we do have a combination of open source artifacts and proprietary artifacts. Now our pro proprietary AR artifacts is always around the management of the system, right? So how do we manage the security of the system? How do we manage the, the data flow within the system? How do we manage the services inside the, of the system across all layers, right? Not just the Hado player but the edge based layer, the zookeeper layer, et cetera, et cetera. So that's where we focus our efforts going forward and that's how we differentiate ourself from our, from other vendors out there. Cloudera manager, Cloudera navigator are very unique to us. Nobody else has anything close to those capabilities out there. >>So it sounds like the contributions you make to open source are cultural of, of, in nature, I mean DNA of sorts of Right. And so you're, that's something that you guys do cuz you've always done it. Absolutely. And then the, the artifacts that are proprietary are essentially around rationalizing the revenue opportunity with the expense that you're gonna apply there and making a business case decided >>How to balance. That's that's one. And then two, the differentiation from other competitors. So these two things, Yes. >>Okay. >>I believe that's fundamental to business to open source business models. >>Yeah, I mean there are many open source business models, right? You can go pure service, you can go, like you said, you can totally bogart the code. >>There is no, there is no pure service open source model company that was able to build the longlasting surviving public company, never happened in history. They always get acquired because it becomes a commodity. I >>Mean, right. I mean, I mean and even ibm, right? >>Tom or I want to ask you about the storage thing. We were talking before camera, the, the hor and worst announcement storage you, what's your take on that? >>Which one? The Gluster, the one with Red Hats? Yes. Yes. So Red Hats and yeah, there has been recent news about Red Hat with, with Hor Works having a version of the Haddo platform that uses map use for the computation but uses Red Hat for the storage, right? So Red Hat has a new storage offering that was built based off of a company they acquired was called Guster. And that, that news was very, very surprising to me. And it, the reason why it was surprising, it's correlated also with a shift in messaging from, from Horton works. If you look at Horton Works last year at had Summit last year, one of the key messages that they deliver to us is that within the next five years or by 2015, the tagline back then by 2015, and you're doing research right now to see if I'm saying the right thing. By 2015, half the world data data will be on, will be stored in had would be stored in had. Yes. If you look today at the slides, it >>Doesn't say that it says within five years, >>Right? No, no, no. It says, well >>That was the second iteration was within five years. And now they say something >>Different. Now say they say within 2015 by, sorry, by 2015, half the world's data will be processed by Hado and instead of stored by Hado. And that's a very, very fundamental So >>It's a nuance. >>It's a, it's a very important >>Nuance. Well it's a big deal because yes, when I first saw that I said, Hmm, what does this all mean? And then it sounds 2015 sounds a little early. Yes. And now you're saying processed by, Okay that's different. >>Yes, exactly. And and the reason why now is we believe s GFS is very, very core to the had platform. S GFS is very core to had platform, the storage system of had we want. It's really the layer that Mid had with is more than anything else is how scalable, how reliable and how economical the sdfs storage layer is. So we, we really, I mean ask qu works and ask all the companies working in the, in the had community not to fragment at the storage layer. We need the storage for had to stay inside of had and not to fragment that out. That's very, very critical. >>Okay. So but so >>You're saying that they're in indicating through the gesture that, that they're not come out saying we're going to fragment Hgfs, but the way that this is position might signal >>No, no, no. The announcement, the announcement with Red Hat is >>That is the direct signal. It's >>Literally, we, you'll be able to run map produce directly on top of Red Hat storage instead of sdfs. >>Okay. So >>I >>Interpreted it, I interpret it as they were just hortonwork was hedging on its prediction, which I said Okay, I'll give 'em a break on that. You're saying it's something different, >>It's a shift in strategy potentially. Yeah. Which can be dangerous. It's shift in strategy. >>Is that a compliance issue? Cuz you know, the, the Dishon Hads poss Yeah. Red Hat does have a lot of enterprise customers. Yeah. So is that just maybe if >>Then invest in making had poss compliance, which actually by the way, we are as a community investing in that. Yeah. Yes. You must have. Yeah. So we are investing in adding compulsive poss compliance to had, we're investing in adding snapshots into had, which will be coming very, very soon overnight. >>Well, do you think that that pick a year, I don't care if it's 2015 2000, 22,000 whenever that the majority of the world's data will be running into do >>The majority of worse data that has to do with analytics. Yes. Okay. So so there is, >>So that is that >>Is it's very important, the caveat. Yes, exactly. Because there is lots of types of data that are not very suitable for, had at all. For example, that data storage for Oracle systems, for Oracle database systems. No, you wanna store that in an NetApp emc you don't wanna store that in Hao the, the, the, the, the data storage for streaming video files, right? For just streaming lots and lots of video files. No, you don't wanna store that indu. It's >>A huge >>Proportion of the data. Yeah. Which is a huge, huge >>Proportion of data files, in fact that could overwhelm the data. >>Yeah. So the new nuance, like I would say like I agree that the half thing but the half thing within the world of data for the purpose of analysis. >>Yeah. Okay. So that's, that's >>Narrow down the >>Yeah, okay. But it's a more reasonable, But I've, I >>Never, It's still a huge market by the way. It is. Yeah, >>It is. Yes. Okay. So, so what's next for you? A are you, you, you've gone on this, this journey, you start this company. You've, you've been traveling around like crazy working with customers. What's the next phase of aara do's, you know, career? >>What >>Do you want to have happen next? I mean, what, what do you, what excites you? What do you, what are you working on? >>Yeah, it's just to continue to grow cloud there to be the biggest company it can be. I mean, we want to be literally, we want be one of the very few companies that we're able to take an open source model and turn that into a large publicly traded corporation. >>So you've talked about that you guys brought a new CEO on Right. Look at the background of the ceo and it's, you know, clearly it's got some IPO chops. Yes. So that's, that's an aspiration that you guys have put forth. Okay. >>And you're outward facing now. So you're doing a lot of travel. Yes. So what, what, where have, what have your travels taken now? You've been in China, you obviously you've got a European office Yeah. Open. So what's going on internationally? Give us some sound bites of, of what's happening in the field. Yeah, >>So in, in internationally, I mean, Europe definitely is our next big focus right now. And we now have a big operation in Europe and we have an office presence in, in Europe and a big team down there. And it's growing very quickly. I would say Europe is about two years behind the US kind of like that's how the, how the growth usually matters. What's happening here. And yeah, so we, our, our next big market is Europe. We are looking at China. We don't have a big process in China right now. Japan, we have a big presence in Japan. Japan is growing very quickly. So yeah, I mean we're obviously Canada with the US growing very quickly as well. >>Great to have you on the cube again, for me personally and, and for, for Dave. And I wanna say thanks to Cloudera for some great support over the years. You guys have been fantastic. You know, I say it's built a great company. It's so hard to build a company. You guys have done a great job. I gotta ask you the final question because you did bring that first sound bite, which was, I saw the future, this is back when you guys were just in your B round in, in Palo Alto office, just ramping up, just starting to ramp what's next? What do you see as around the corner? Obviously we're on a trajectory right now. A lot of things gonna get done. Positive compliance, a lot of stuff's gonna fill in. The platform's gonna get stronger. Yeah. We think that open source will win. Yeah. Through all the democratization of open source. What's next? What's the, what's around the corner that you're watching personally that you're, that's interesting to you? A or around where this will take us? >>Yeah. So what, what's next is having this, having this vision become true. Having this future vision that, that you refer to become true. Meaning having a single platform that can store all of your data and that can, regardless of the type of that data, and allow you to extract value for different types of workloads, whether that be batch, interactive machine learning or search or more, right? There will be more things that will come to the platform, but how to bring your applications, all of your data applications, how to bring them to your data and all of your data as opposed to have the data go to them. >>And what are the landmines out there that you need to avoid Yes. In the industry and community needs to avoid to make that a reality. >>The, the key landmine, it's, it's a bit technical. The landmine is a bit technical, which is making sure that they, they are vision continues to evolve and that we have the capability to properly have a multi workload resource management system that allows me to run all of these type of workloads without having them step on each other's steps. That's the key key step going forward. And >>Of course, playing well together in the sandbox. And as always, competitive competition is good. And again, Hadup is doing great. Amma Aala, co-founder of Cloudera inside the Cube. This is Silicon Angle and Wiki Bond's exclusive coverage of ADU Summit here in Silicon Valley. Right back with our next guest after the short break.

>>Okay. We're back here, live in New York city for big data week. This is siliconangle.tvs, exclusive coverage of Hadoop world strata plus Hadoop world big event, a big data week. And we just wrote a blog post on siliconangle.com calling this the south by Southwest for data geeks and, and, um, it's my prediction that this is going to turn into a, quite the geek Fest. Uh, obviously the crowd here is enormous packed and an amazing event. And, uh, we're excited. This is siliconangle.com. I'm the founder John ferry. I'm joined by cohost update >>Volante of Wiki bond.org, where people go for free research and peers collaborate to solve problems. And we're here with Jack Norris. Who's the vice president of market marketing at map are a company that we've been tracking for quite some time. Jack, welcome back to the cube. Thank you, Dave. I'm going to hand it to you. You know, we met quite a while ago now. It was well over a year ago and we were pushing at you guys and saying, well, you know, open source and nice look, we're solving problems for customers. We got the right model. We think, you know, this is, this is our strategy. We're sticking to it. Watch what happens. And like I said, I have to hand it to you. You guys are really have some great traction in the market and you're doing what you said. And so congratulations on that. I know you've got a lot more work to do, but >>Yeah, and actually the, the topic of openness is when it's, it's pretty interesting. Um, and, uh, you know, if you look at the different options out there, all of them are combining open source with some proprietary. Uh, now in the case of some distributions, it's very small, like an ODBC driver with a proprietary, um, driver. Um, but I think it represents that that any solution combining to make it more open is, is important. So what we've done is make innovations, but what we've made those innovations we've opened up and provided API. It's like NFS for standard access, like rest, like, uh, ODBC drivers, et cetera. >>So, so it's a spectrum. I mean, actually we were at Oracle open world a few weeks ago and you listen to Larry Ellison, talk about the Oracle public cloud mix of actually a very strong case that it's open. You can move data, it's all Java. So it's all about standards. Yeah. And, uh, yeah, it from an opposite, but it was really all about the business value. That's, that's what the bottom line is. So, uh, we had your CEO, John Schroeder on yesterday. Uh, John and I both were very impressed with, um, essentially what he described as your philosophy of we, we not as a product when we have, we have customers when we announce that product and, um, you know, that's impressive, >>Is that what he was also given some good feedback that startup entrepreneurs out there who are obviously a lot of action going on with the startup community. And he's basically said the same thing, get customers. Yeah. And that's it, that's all and use your tech, but don't be so locked into the tech, get the cutters, understand the needs and then deliver that. So you guys have done great. And, uh, I want to talk about the, the show here. Okay. Because, uh, you guys are, um, have a big booth and big presence here at the show. What, what did you guys are learning? I'll say how's the positioning, how's the new news hitting. Give us a quick update. So, >>Uh, a lot of news, uh, first started, uh, on Tuesday where we announced the M seven edition. And, uh, yeah, I brought a demo here for me, uh, for you all. Uh, because the, the big thing about M seven is what we don't have. So, uh, w we're not demoing Regents servers, we're not demoing compactions, uh, we're not demoing a lot of, uh, manual administration, uh, administrative tasks. So what that really means is that we took this stack. And if you look at HBase HBase today has about half of dupe users, uh, adopting HBase. So it's a lot of momentum in the market, uh, and, you know, use for everything from real-time analytics to kind of lightweight LTP processing. But it's an infrastructure that sits on top of a JVM that stores it's data in the Hadoop distributed file system that sits on a JVM that stores its data in a Linux file system that writes to disk. >>And so a lot of the complexity is that stack. And so as an administrator, you have to worry about how data gets permit, uh, uh, you know, kind of basically written across that. And you've got region servers to keep up, uh, when you're doing kind of rights, you have things called compactions, which increased response time. So it's, uh, it's a complex environment and we've spent quite a bit of time in, in collapsing that infrastructure and with the M seven edition, you've got files and tables together in the same layer writing directly to disc. So there's no region servers, uh, there's no compactions to deal with. There's no pre splitting of tables and trying to do manual merges. It just makes it much, much simpler. >>Let's talk about some of your customers in terms of, um, the profile of these guys are, uh, I'm assuming and correct me if I'm wrong, that you're not selling to the tire kickers. You're selling to the guys who actually have some experience with, with a dupe and have run into some of the limitations and you come in and say, Hey, we can solve some of those problems. Is that, is that, is that right? Can you talk about that a little bit >>Characterization? I think part of it is when you're in the evaluation process and when you first hear about Hadoop, it's kind of like the Gartner hype curve, right. And, uh, you know, this stuff, it does everything. And of course you got data protection, cause you've got things replicated across the cluster. And, uh, of course you've got scalability because you can just add nodes and so forth. Well, once you start using it, you realize that yes, I've got data replicated across the cluster, but if I accidentally delete something or if I've got some corruption that's replicated across the cluster too. So things like snapshots are really important. So you can return to, you know, what was it, five minutes before, uh, you know, performance where you can get the most out of your hardware, um, you know, ease of administration where I can cut this up into, into logical volumes and, and have policies at that whole level instead of at an individual file. >>So there's a, there's a bunch of features that really resonate with users after they've had some experience. And those tend to be our, um, you know, our, our kind of key customers. There's a, there's another phase two, which is when you're testing Hadoop, you're looking at, what's possible with this platform. What, what type of analytics can I do when you go into production? Now, all of a sudden you're looking at how does this fit in with my SLS? How does this fit in with my data protection, uh, policies, you know, how do I integrate with my different data sources? And can I leverage existing code? You know, we had one customer, um, you know, a large kind of a systems integrator for the federal government. They have a million lines of code that they were told to rewrite, to run with other distributions that they could use just out of the box with Matt BARR. >>So, um, let's talk about some of those customers. Can you name some names and get >>Sure. So, um, actually I'll, I'll, I'll talk with, uh, we had a keynote today and, uh, we had this beautiful customer video. They've had to cut because of times it's running in our booth and it's screaming on our website. And I think we've got to, uh, actually some of the bumper here, we kind of inserted. So, um, but I want to shout out to those because they ended up in the cutting room floor running it here. Yeah. So one was Rubicon project and, um, they're, they're an interesting company. They're a real-time advertising platform at auction network. They recently passed a Google in terms of number one ad reach as mentioned by comScore, uh, and a lot of press on that. Um, I particularly liked the headline that mentioned those three companies because it was measured by comScore and comScore's customer to map our customer. And Google's a key partner. >>And, uh, yesterday we announced a world record for the Hadoop pterosaur running on, running on Google. So, um, M seven for Rubicon, it allows them to address and replace different point solutions that were running alongside of Hadoop. And, uh, you know, it simplifies their, their potentially simplifies their architecture because now they have more things done with a single platform, increases performance, simplifies administration. Um, another customer is ancestry.com who, uh, you know, maybe you've seen their ads or heard, uh, some of their radio shots. Um, they're they do a tremendous amount of, of data processing to help family services and genealogy and figure out, you know, family backgrounds. One of the things they do is, is DNA testing. Uh, so for an internet service to do that, advanced technology is pretty impressive. And, uh, you know, you send them it's $99, I believe, and they'll send you a DNA kit spit in the tube, you send it back and then they process that and match and give you insights into your family background. So for them simplifying HBase meant additional performance, so they could do matches faster and really simplified administration. Uh, so, you know, and, and Melinda Graham's words, uh, you know, it's simpler because they're just not there. Those, those components >>Jack, I want to ask you about enterprise grade had duped because, um, um, and then, uh, Ted Dunning, because he was, he was mentioned by Tim SDS on his keynote speech. So, so you have some rockstars stars in the company. I was in his management team. We had your CEO when we've interviewed MC Sri vis and Google IO, and we were on a panel together. So as to know your team solid team, uh, so let's talk about, uh, Ted in a minute, but I want to ask you about the enterprise grade Hadoop conversation. What does that mean now? I mean, obviously you guys were very successful at first. Again, we were skeptics at first, but now your traction and your performance has proven this is a market for that kind of platform. What does that mean now in this, uh, at this event today, as this is evolving as Hadoop ecosystem is not just Hadoop anymore. It's other things. Yeah, >>There's, there's, there's three dimensions to enterprise grade. Um, the first is, is ease of use and ease of use from an administrator standpoint, how easy does it integrate into an existing environment? How easy does it, does it fit into my, my it policies? You know, do you run in a lights out data center? Does the Hadoop distribution fit into that? So that's, that's one whole dimension. Um, a key to that is, is, you know, complete NFS support. So it functions like, uh, you know, like standard storage. Uh, a second dimension is undependability reliability. So it's not just, you know, do you have a checkbox ha feature it's do you have automated stateful fail over? Do you have self healing? Can you handle multiple, uh, failures and, and, you know, automated recovery. So, you know, in a lights out data center, can you actually go there once a week? Uh, and then just, you know, replace drives. And a great example of that is one of our customers had a test cluster with, with Matt BARR. It was a POC went on and did other things. They had a power field, they came back a week later and the cluster was up and running and they hadn't done any manual tasks there. And they were, they were just blown away to the recovery process for the other distributions, a long laundry list of, >>So I've got to ask you, I got to ask you this, the third >>One, what's the third one, third one is performance and performance is, is, you know, kind of Ross' speed. It's also, how do you leverage the infrastructure? Can you take advantage of, of the network infrastructure, multiple Knicks? Can you take advantage of heterogeneous hardware? Can you mix and match for different workloads? And it's really about sharing a cluster for different use cases and, and different users. And there's a lot of features there. It's not just raw >>The existing it infrastructure policies that whole, the whole, what happens when something goes wrong. Can you automate that? And then, >>And it's easy to be dependable, fast, and speed the same thing, making HBase, uh, easy, dependable, fast with themselves. >>So the talk of the show right now, he had the keynote this morning is that map. Our marketing has dropped the big data term and going with data Kozum. Is that true? Is that true? So, Joe, Hellerstein just had a tweet, Joe, um, famous, uh, Cal Berkeley professor, computer science professor now is CEO of a startup. Um, what's the industry trifecta they're doing, and he had a good couple of epic tweets this week. So shout out to Joe Hellerstein, but Joel Hellison's tweet that says map our marketing has decided to drop the term big data and go with data Kozum with a shout out to George Gilder. So I'm kind of like middle intellectual kind of humor. So w w w what's what's your response to that? Is it true? What's happening? What is your, the embargo, the VP of marketing? >>Well, if you look at the big data term, I think, you know, there's a lot of big data washing going on where, um, you know, architectures that have been out there for 30 years or, you know, all about big data. Uh, so I think there's a, uh, there's the need for a more descriptive term. Um, the, the purpose of data Kozum was not to try to coin something or try to, you know, change a big data label. It was just to get people to take a step back and think, and to realize that we are in a massive paradigm shift. And, you know, with a shout out to George Gilder, acknowledging, you know, he recognized what the impact of, of making available compute, uh, meant he recognized with Telekom what bandwidth would mean. And if you look at the combination of we've got all this, this, uh, compute efficiency and bandwidth, now data them is, is basically taking those resources and unleashing it and changing the way we do things. >>And, um, I think, I think one of the ways to look at that is the new things that will be possible. And there's been a lot of focus on, you know, SQL interfaces on top of, of Hadoop, which are important. But I think some of the more interesting use cases are taking this machine J generated data that's being produced very, very rapidly and having automated operational analytics that can respond in a very fast time to change how you do business, either, how you're communicating with customers, um, how you're responding to two different, uh, uh, risk factors in the environment for fraud, et cetera, or, uh, just increasing and improving, um, uh, your response time to kind of cost events. We met earlier called >>Actionable insight. Then he said, assigning intent, you be able to respond. It's interesting that you talk about that George Gilder, cause we like to kind of riff and get into the concept abstract concepts, but he also was very big in supply side economics. And so if you look at the business value conversation, one of things we pointed out, uh, yesterday and this morning, so opening, um, review was, you know, the, the top conversations, insight and analytics, you know, as a killer app right now, the app market has not developed. And that's why we like companies like continuity and what you guys are doing under the hood is being worked on right at many levels, performance units of those three things, but analytics is a no brainer insight, but the other one's business value. So when you look at that kind of data, Kozum, I can see where you're going with that. >>Um, and that's kind of what people want, because it's not so much like I'm Republican because he's Republican George Gilder and he bought American spectator. Everyone knows that. So, so obviously he's a Republican, but politics aside, the business side of what big data is implementing is massive. Now that I guess that's a Republican concept. Um, but not really. I mean, businesses is, is, uh, all parties. So relative to data caused them. I mean, no one talks about e-business anymore. We talking to IBM at the IBM conference and they were saying, Hey, that was a great marketing campaign, but no one says, Hey, uh, you and eat business today. So we think that big data is going to have the same effect, which is, Hey, are you, do you have big data? No, it's just assumed. Yeah. So that's what you're basically trying to establish that it's not just about big. >>Yeah. Let me give you one small example, um, from a business value standpoint and, uh, Ted Dunning, you mentioned Ted earlier, chief application architect, um, and one of the coauthors of, of, uh, the book hoot, which deals with machine learning, uh, he dealt with one of our large financial services, uh, companies, and, uh, you know, one of the techniques on Hadoop is, is clustering, uh, you know, K nearest neighbors, uh, you know, different algorithms. And they looked at a particular process and they sped up that process by 30,000 times. So there's a blog post, uh, that's on our website. You can find out additional information on that. And I, >>There's one >>Point on this one point, but I think, you know, to your point about business value and you know, what does data Kozum really mean? That's an incredible speed up, uh, in terms of, of performance and it changes how companies can react in real time. It changes how they can do pattern recognition. And Google did a really interesting paper called the unreasonable effectiveness of data. And in there they say simple algorithms on big data, on massive amounts of data, beat a complex model every time. And so I think what we'll see is a movement away from data sampling and trying to do an 80 20 to looking at all your data and identifying where are the exceptions that we want to increase because there, you know, revenue exceptions or that we want to address because it's a cost or a fraud. >>Well, that's what I, I would give a shout out to, uh, to the guys that digital reasoning Tim asked he's plugged, uh, Ted. It was idolized him in terms of his work. Obviously his work is awesome, but two, he brought up this concept of understanding gap and he showed an interesting chart in his keynote, which was the date explosion, you know, it's up and, you know, straight up, right. It's massive amount of data, 64% unstructured by his calculation. Then he showed out a flat line called attention. So as data's been exploding over time, going up attention mean user attention is flat with some uptick maybe, but so users and humans, they can't expand their mind fast enough. So machine learning technologies have to bridge that gap. That's analytics, that's insight. >>Yeah. There's a big conversation now going on about more data, better models, people trying to squint through some of the comments that Google made and say, all right, does that mean we just throw out >>The models and data trumps algorithms, data >>Trumps algorithms, but the question I have is do you think, and your customer is talking about, okay, well now they have more data. Can I actually develop better algorithms that are simpler? And is it a virtuous cycle? >>Yeah, it's I, I think, I mean, uh, there are there's, there are a lot of debate here, a lot of information, but I think one of the, one of the interesting things is given that compute cycles, given the, you know, kind of that compute efficiency that we have and given the bandwidth, you can take a model and then iterate very quickly on it and kind of arrive at, at insight. And in the past, it was just that amount of data in that amount of time to process. Okay. That could take you 40 days to get to the point where you can do now in hours. Right. >>Right. So, I mean, the great example is fraud detection, right? So we used the sample six months later, Hey, your credit card might've been hacked. And now it's, you know, you got a phone call, you know, or you can't use your credit card or whatever it is. And so, uh, but there's still a lot of use cases where, you know, whether is an example where modeling and better modeling would be very helpful. Uh, excellent. So, um, so Dana custom, are you planning other marketing initiatives around that? Or is this sort of tongue in cheek fun? Throw it out there. A little red meat into the chum in the waters is, >>You know, what really motivated us was, um, you know, the cubes here talking, you know, for the whole day, what could we possibly do to help give them a topic of conversation? >>Okay. Data cosmos. Now of course, we found that on our proprietary HBase tools, Jack Norris, thanks for coming in. We appreciate your support. You guys have been great. We've been following you and continue to follow. You've been a great support of the cube. Want to thank you personally, while we're here. Uh, Matt BARR has been generous underwriter supportive of our great independent editorial. We want to recognize you guys, thanks for your support. And we continue to look forward to watching you guys grow and kick ass. So thanks for all your support. And we'll be right back with our next guest after this short break. >>Thank you. >>10 years ago, the video news business believed the internet was a fat. The science is settled. We all know the internet is here to stay bubbles and busts come and go. But the industry deserves a news team that goes the distance coming up on social angle are some interesting new metrics for measuring the worth of a customer on the web. What zinc every morning, we're on the air to bring you the most up-to-date information on the tech industry with scrutiny on releases of the day and news of industry-wide trends. We're here daily with breaking analysis, from the best minds in the business. Join me, Kristin Filetti daily at the news desk on Silicon angle TV, your reference point for tech innovation 18 months.

>>Okay. We're back live in Silicon valley and San Jose, California for the continuous coverage of siliconangle.tv and have duke world 2012. This is ground zero for the alpha geeks in big data. Uh, just the tech elite. We call them tech athletes and, uh, we're excited to cover it on the ground. Extract the signal from the noise here. This is the cube, our flagship telecast. I'm joining my co-host Jeff Kelly from Wiki bond.org, the best analyst in the business. Jeff, welcome back for another segment. End of the day, day one loving every minute. Okay. We're here with our guest. Jack Norris is a cm of map bar Jack. Welcome back to the cube. You've been on a few times. Um, so you guys have some news. Yes. So let's get right to the news. So you guys are a player in the business, so share with your news, the folks. Excellent jump right in. >>So, uh, two big announcements today, we announced that Amazon is integrating map bar as part of their Lastic MapReduce service and both edition or, or free edition. M three is available as well as M five directly with Amazon, Amazon in the cloud. >>So what's the value proposition. Why would a customer say, all right, I want to do this in the cloud manpower, an Amazon cloud rather than doing it on premise. >>Okay. So let's start with, I mean, there's a lot of value propositions, all balled up into one here. Uh, first of all, in the cloud, it allows them to spin up very quickly. Within a couple minutes, you can get, uh, you know, hundreds of nodes available. Um, and, uh, and depending on where you're processing the data, if you've got a lot of data in the cloud already makes a lot of sense to do the Hadoop processing directly there. So that's, that's one area. A second is you might have an on-premise cloud deployment and need to have a disaster recovery. So map R provides point in time, snapshots, uh, as well as, as a white area replication. So you can use mirroring having Amazon available as a target is a huge advantage. And then there's also a third application area where you can do processing of the data in the cloud and then synchronize those results to an on-premise. So basically process where the data is combined the results into a cluster on premise. So you >>Don't have to move the raw data. Uh, >>On-premise actually, it's all about let's do the processing on the data. Well, you know, the whole, >>The value proposition and big data in general is let's not move, move data as little as possible. Yep. Uh, you know, so you bring the computation to the data, if you can. Uh, so what are your take on this event? I mean, we've got, uh, this is a, you know, the 4th of June summit, uh, you know, Hortonworks is now fully taken over the show and talk about what you see out here in terms of, uh, the other vendors that play. And, uh, just to kind of the attendees, the vibe you're seeing, >>Uh, it's a lot of excitement. I think a big difference between last year, which seemed to be very developer focused. We're seeing a lot of, a lot of presentations by customers. A lot of information was shared by our customers today. It was fun to see that, uh, comScore's shared, uh, shared their success. Boeing gap map is, uh, it was great for us. >>Fantastic. We look at Amazon, Amazon, first of all, is the gold standard for public cloud. Right? They've knocked it out of the park. Everyone knows Amazon. Um, but they've been criticized on the big data front because of the cycle times involve on. Um, and some developers and mean for web service spending up and down. No problem. Um, and we're seeing businesses like Netflix run on Amazon. So Amazon is not a stranger to running scale for cloud, but Hadoop has kind of been a klugey thing for Amazon. So I think, you know, talk about why Amazon and you guys is a good fit out to the market. The market reach is great. So you guys know and have a huge addressable market. Are you guys helping solve some of that complexity with the, uh, with the MapReduce side? What's, >>What's the core, I guess the first comment first response would be, I think every customer should have that type of Kluge. Uh, uh, they could have the success that Amazon has in Hadoop. They have a huge number of, of, uh, of Hadoop deployments have been very, very successful. I think, >>I mean, you know what I mean by it's natural, it's, cloogy everywhere right now. That's the problem. But Amazon has huge scale, um, and had not a natural fit. There >>Is not a natural fit >>For the data for the data component. And, uh, uh, the HBase for example, >>Component. So where were Amazons, you know, made it very frictionless is the ability to spin up Hadoop to do the analysis. The gap that was missing is some of the, the ha capabilities. The data protection features the disaster recovery, and, you know, we're map are now it gives options to those customers. You know, if they want those kinds of enterprise enterprise grade features, now they have an option within EMR. It can select a M five and, and get moving if they want a performance. And in NFS, they've got the M three options. >>Well, congratulations. I think it's a great deal for you guys and for Amazon customers. My question for you is, as you guys explore the enterprise ready equation, which has been a big topic this week, um, what does that mean to you guys? Cause it means different things to different people depends on where, how high up to OLTB do you go? Right? I mean, we're how far from batch to real time transactional, um, levels you go, I mean, low bash, no problem. But as you start to get more near real time, it's going to be a little bit different gray in this house used security HDFS. Yeah. >>Yeah. So, so duke represents the strategic platform, right? Deploying that in an organization, um, you know, moving from kind of an experimental kind of lab based to production environment creates a different set of feature requirements. How available is it? How easy is it to integrate, right? How do I kind of protect that information and how do I share it? So when we say enterprise grade, we mean you can have SLA, she can put the data there and, and be confident that the data will remain there, that you can have a point in time recovery for an application error or user mistake. Uh, you can have a disaster recovery features in place. And then the integration is about not recreating the wheel to get access to the information. So Hadoop is very powerful, but it requires interacting through an HDFS API. If you can leverage it like through map bar with NFS standard file based access standard ODBC access, open it up. >>So I can use a standard file browser applications to see and manipulate the data really opens up the use cases. And then finally, what we announced in two dot oh, was multitenancy features. So as you share that information, all of a sudden the SLA is of different groups and well, these guys need it immediately. And if you've got some low grade batch jobs are going to impact that. So you want the ability to protect, to isolate, to secure information, and basically have virtual clusters within a cluster. And those features are important to cloud, but they're also important to on-premise >>So great for the hybrid cloud environments out there. I mean, the multitenancy cracking the code on that. Exactly huge. I mean, that is basically, I mean, right now most enterprises are like private cloud because it's like, they're basically extension of their data center and you're seeing a lot more activity in the hybrid cloud as a gateway to the public cloud. So, >>And, and, you know, frankly, people are kind of struggling with in an experimental with Apache Hadoop and the other distributions, the policies are either at the individual file level or the whole cluster. And it all almost forced the creation of separate physical clusters, which kind of goes against the whole Hadoop concept. So the ability to manage it, a logical layer have separate volumes where you can apply policies to apply that applies to all the content underneath really kind of makes it much, much easier for administrators to kind of deal with these multiple use cases. >>Amazon, Amazon has always been one of those cases for the enterprise where it's been one of those and they've, this has been talked about for years, put the credit card down, go play on Amazon, but then bring it back into the it group for certification. And so I think this is a nice product for you guys to bring that comfort. You know, we're very >>Excited the enterprise saying, Hey, >>Come play in Amazon. It's Bulletproof enterprise. Ready? So congratulations. >>I wonder, can we talk, uh, talk use cases. So what are you seeing in terms of, uh, evolving use cases as, as, uh, duke continues to become more enterprise grade, uh, depending on your definition, uh, but how is that impacting what you're seeing in terms of, even if it's just, uh, you know, the, the, um, the mindset even people think now, okay, now it's enterprise grade, well, maybe, you know, in, in, depending on who you talk to, it's been that way for a bit, but what kind of, uh, use cases are you seeing develop now that it's kind of starting to gain acceptance? It's like, okay, we can trust our data is going to be there, et cetera. >>So th there's a huge range of use cases that, uh, different by industry, different by kind of dataset that's being used against everything from really a deep store where you can do analytics on it. So you're selecting the content to something that's very, very analytic machine learning intensive, where you're doing sophisticated clustering algorithms, uh, et cetera, um, where we've seen kind of an expansion of use cases are around real-time streaming and you get streaming data sets that are kind of entering into the cloud. And, um, some of the more mission, critical data moving beyond just maybe click stream data or things that if you happen to drop a few, you know, not a big deal, right. Versus the kind of trust the business type of content. >>Talk a little bit about the streaming, uh, aspects, uh, because of course, you know, we think of duke, we think of a batch system in terms of streaming data into Hadoop. You know, that's, that's a different, uh, that's something we don't, we haven't heard a lot about. So how do you guys approach that? >>So, uh, one of the artifacts of, of HDFS, which is a, is a distributed file system that scores in the underlying Linux file system, it's append only. So as an administrator, you decide, how frequently do I close the file item? I going to do that an hourly basis on it every eight hours, because you have to close the file for other applications to see the data that's been written. Right? So one of the innovations that, uh, that we pursued was to rewrite that create this dynamic read-write layer. So you can continue to write data in any application is seeing the latest data that's written. So you can Mount the cluster as if it's storage and just continue to write data. There really opens up what's, uh, what's possible companies like Informatica, they're all from a messaging product integrates directly in with, with Matt BARR and provides. >>So what kind of advantage does that provide to the end user? What w w translate that into real business value? Why, why is that important? >>Well, so one example is comScore, comScore handles 30 billion, uh, objects a day, uh, as they go out and try to measure the use of, of the web and being able to continually write and stream that information and scale and handle that in a real time and do analytics and turn around data faster, has tremendous business value to them. If they're stuck in a batch environment where the load times lengthen to the point where all of a sudden they can't keep up and they're actually reporting on, you know, old news. And I think the analogy is forecasting rain a day after it's wet. Isn't exactly valuable. >>Yeah. So you guys, obviously a great deal of the enterprise ready for Amazon, big story, big coup for the company. What's next for you. I want to ask that and make sure you get that out there on your agenda for the next year, but then I want you to take a step back a year, maybe a year and a half ago. Look back at how much has changed in this landscape. Um, share your perspective because the market has gone through an evolution where there's been a market opportunity, and then everyone goes, oh my God, it's bigger than we actually thought. I mean, Jeff, Kelly's a groundbreaking report about the $50 billion market is now being talked about as too low. So big data has absolutely opened up to a huge, and it's changed some of the tactics around strategies. So your strategy, Hortonworks strategy, even cloud era. So, and it's still evolving. So what's changed for the folks out there from a year and a half ago, a year ago to today, and then look out for the next 12 months. What's on your agenda. >>Well, if, if you look back, I think we've been fairly consistent. Um, uh, I'm, I'm not going to take credit for the vision of our CEO and CTO. Uh, but they recognized early on that Hadoop was, uh, was a strategic platform and to be a strategic platform that applied to the broadest number of use cases and organizations required some, some areas, uh, of innovation and particularly the how it, how it scaled, how it was managed, how you stored and protected the information needed a rearchitecture. And I think that, you know, architecture matters when you're going through a paradigm shift, having the right one in place creates this, this ability, you know, to speed innovation. And I think that's, if there's anything that's changed, I think it's the speed of innovation has even increased in the Hadoop community. I think it's, it's created a focus on these enterprise grade features on how do we store this valuable information and, and continue to explore. >>And I think one of the observations I'll make is that on that note is that it really focuses everyone to be just mind your own business and get the products out. You know what I'm saying? We've seen everyone, the product focus be the number one conversation. >>What we've seen is customers, you know, start and they expand rapidly. Some of that student data growth, but a lot of it is student more and more applications are being delivered and, and, uh, and, and the values kind of extracted from the hoop platform and success breeds success. Well, >>Congratulations for all your success, great win with Amazon web services and make that a little bit more easier, more robust, and more, more features for them and you, uh, more revenue for part of our, um, and I want to personally thank you for your support to the cube. Uh, we've expanded with a new studio B software for extra extra interviews, um, and wanna expand the conversation, thanks to your generous support. You can bring the independent coverage out to the market and, um, great community, thanks for helping us out. And we appreciate it. So thank you. Okay. Jack Dorsey with Matt bar, we'll be right back to wrap up day one with that. Jeff and I will give our analysis right at the short break.

okay we're back live in new york city for hadoop world 2011 john furrier its founder SiliconANGLE calm and we have a special walk-in guest tomorrow and allah the vp of engineering co founder of Cloudera who's going to be on at two thirty eastern time on the cube to go more in depth but since we saw her in the hallway we had a quick spot wanted to grab him in here this is the cube our flagship telecast where we go out to the event atop the smartest people and i'm here with my co-host i'm dave vellante Wikibon door welcome back you're a longtime cube alum so appreciate you coming back on and doing a quick drive by here thanks for the nice welcome so you know we go talk to the smart people in the room you're one of the smartest guys that I know and we've been friends for years and it was your my tweet heard around the world by you to find space and we've been sharing the office space at Cloudera a year didn't have you I meant to have you we're going to be trying to find space because you're expanding so fast we have to get in a new home sorry about that but I wanted to really thank you personally appear on live you've enabled SiliconANGLE Wikibon to we figured it out early because of you I mean we had our nose sniffing around the big data area before it's called big data but when we met talked we've been tracking the social web and really it's exploded in an amazing way and I'm just really thankful because I've been had a front-row seat in the trenches with you guys and and it's been amazing so I want to thank you're welcome and that's great to have you on board and so so you you've been evangelizing in the trenches at Yahoo you were a ir a textile partners announcing the hundred million dollar fund which is all great news today but you've been the real spark get cloudy air is one of the 10 others one of them but I know one of the main sparks a co-founder a lots of ginger cuz I'm Rebecca and my co-founder from facebook I mean we both we said this before like we saw the future like an hour companies we saw the future where everybody is gonna go next and now Jeff's gonna be on as well he's now taking this whole date of science thing art yep building out a team you gotta drilled that down with him what do you what do you think about all this I mean like right now how do you feel personally emotionally and looking at the marketplace share with us your yeah I'm very emotional today actually yeah lots of the good news is you heard about the funding news yes million dollars for startups but no but the 14 oh yeah yeah it is more most actually the news was supposed to come out today came out a bit earlier sir day but yeah I'm very very emotional because of that it's a very Testament from very big name investor's of how well we were doing and recognition of how big this wave really is also the hundred million fun from Excel that's also a huge testament and lots of hopefully lots of new innovations or startups will come out of that so I'm very emotional about that but also overwhelmed by the by the the size of this event and how many people are really gravitating towards the technology which shows how much work we still have to do going forward it was very very August of a great a bit scared a bit scared Michaels is a great CEO on stage they're great guy we love Mike just really he's geeky and he's pragmatic Jerry strategist and you got Kirk who's the operator yeah but he showed a slide up at his keynote that showed the evolution of Hadoop yes the core Hadoop and then he showed ya year-by-year and now we got that columns extending and you got new new components coming out take us through that that progression just go back a few years in and walk us through why is this going on so fast and what are the what's the what's the community doing and just yeah and what happened in 2008 it doesn't need was one mr. yeah when we when we started so I mean first 2008 when we started and what he was believing us back then that hey this thing is going to be big like we had the belief because we saw it happen firsthand but many folks were dismissive and no no no this this big data thing is a fat and nobody will care about it and look and behold today it's obviously proving not to be the case in terms of the maturity of the of the platform you're absolutely right i mean the slide that Mike showed should but only thirty percent of the contributions happening today are in the Hadoop core layer and and and and the overall kind of vision there is very system very similar to the operating system right except what this really is it's a data operating system right it's how to operate large amounts of data in a big data center so sorry it's like an operating system for many machines as opposed to Linux which does not bring system for a single machine right so Hadoop when it came out Hadoop is only the colonel it's only that inner layers which if you look at any opening system like windows or linux and so on the core functionality is two things storing files and running applications on top of these files that's what windows does that's what linux does that was loop does at the heart but then to really get an opening system to work you need many ancillary components around it that really make it functional you need libraries in it applications in eat integration IO devices etc etc and that's really what's happening in the hadoop world so started with the core OS layer which is Hadoop HDFS for storage MapReduce for computation but then now all of these other things are showing around that core kernel to really make it a fully functional extensible data opening system I which made a little replay button but let's just put the paws on that because this is kind of an important point in folks out there there's a lot of different and a lot of people and metaphors are used in this business so it's the Linux I want to be it's just like Red Hat right yeah we kind of use that term the business model is talk a little bit about that we just mentioned you know not like Linux just unpack that a little bit deeper for us what's the difference you mentioned Linux is can you replay what you just said that was really so I was actually talking about the similarity the similarity and then i can and then i can talk about the difference the similarity is the heart of Hadoop is a system for storing files which is sdfs and a system for running applications on top of these files which is MapReduce the heart of Linux is the same thing assistant for storing files which is a txt for and a system for scheduling applications on top of these files that's the same heart of Windows and so on the difference though so that's the similarity I got a difference is Linux is made to run on a single note right and when this is made to run on a single note Hadoop is really made to run on many many notes so hadoo bicester cares about taking a data center of servers a rack of servers or a data center of servers and having them look like one big massive mainframe built out of commodity hardware that can store arbitrary amounts of data and run any type of hence the new components like the hives of the world so now so now these new components coming up like high for example I've makes it easier to write queries for Hadoop it's it's a sequel language for writing queries on top of Hadoop so you don't have to go and write it in MapReduce which we call that assembly language of Hadoop so if you write it and MapReduce you will get the most flexibility you will get the most performance but only if you know what you're doing very similar when you do machine code if you do machine cool assembly you will able do anything but you can also shoot yourself in the foot sunbelt is that right the same thing with MapReduce right when you use hive hive abstracts that out for you so your rights equal and then hive takes care of doing all of the plumbing work to get that compulsion to map it is for you so that's hive HBase for example is a very nice system that augments a dupe makes it low latency and makes it makes it support update and insert and delete transactions which are HDFS does not support out of the box so small like a database it's more like my sequel yeah the energy of my sequel to Linux is very similar to hbase to HDFS and what's your take on were from you know your founders had on now yeah on the business model similarities and differences with with redhead yes so actually they are different I mean that the sonority the similarity stops at open source we are both open source right in the sense that the core system is open source is available out there you can look at the source code again the and so on the difference is with redhead red that actually has a license on their bits so there's the source code and then there's the bits so when Red Hat compiles the source code and two bits these bits you cannot deploy them without having a red hat license with us is very different is now we have the source code which is Apache is all in the patchy we compile the source code into a bunch of bits which is our distribution called cdh these bits are one hundred percent open-source 103 can deploy them use them you don't have to face anything the only reason why you would come back and pay us is for Cloudera enterprise which is really when you go operational when become operational a mission-critical cloud enterprise gives you two things first it gives you a proprietary management suite that we built and it's very unique to us nobody in the market has anything close to what we have right now that makes it easier for you to deploy configure monitor provision do capacity planning security management etc for a loop nobody else has anything close what we have right now for that management's that is unique to cloud area and not part of a patchy open source yes it's not part of the vet's office you only get that as a subscriber to cloud era we do have a free version of that that's available for download and it can run up to 15 hours just for you to get up and running quickly yeah and it's really very simple has a very simple installer like you should be able to go fire off that software and say install Hadoop these are one of my servers and would take care of everything else for you it's like having these installers you know when windows came out in the beginning and he had this nice progress bar and you can install applications very easily imagine that now for a cluster of servers right that's ready what this is the other reason why people subscribe to the cloud enterprise in addition to getting this management suite is getting our support services right and support is necessary for any software even if it's free even for hardware think if I give you a free airplane right now just comment just give it here you go here is an airplane right you can run this airplane make money from passengers you still need somebody to maintain their plane for you right you can still go higher your mechanics maybe we'd have a tweetup bummer you can hire your own mechanics to maintain that airplane but we tell you like if you subscribe with us as the mechanics for your airplane the support you will get with us will be way better than anything else and economics of it also would be way better than having your own stuff for doing the maintenance for that airplane okay final question and we got a one-minute because we slid you in real quick we're going to come back for folks armor is going to come back at two-thirty so come back its eastern time and we'll have a more in-depth conversation but just share with the folks watching your view of what's going on in the patchy and you know there's all these kind of weird you know Fudd being thrown around that clutter is not this and that and you guys clearly the leader we talked with Kirk about that we don't need to go into that but just surely this what's going on what's the real deal happening with Apache the code and you have a unique offering which I mean the real deal and I advise people to go look at this blog post that our CEO wrote called by Michaelson road called the community effect and the real deal is there is a very big healthy community developing the source code for Hadoop the core system which is actually fsm MapReduce and all the components around around that core system we at Cloudera employ a very large engineering organization and tactile engineering relation is bigger than many of these other companies in the space that's our engineering is bigger if you look at the whole company itself is much much bigger than any of these other players so we we do a lot of contributions and to the core system and to the projects around it however we are part of the community and we're definitely doing this with the community it's not just a clowder thing for the core platform so that that's the real deal all right yeah so here we are armor that co-founder congratulations great funding hundred L from accel partners who invested in you guys congratulations you're part of the community we all know that just kind of clarifying that for the record and you have a unique differentiator management suite and the enterprise stuff and say expand the experience experience yeah I think a huge differentiation we have is we have been doing this for three years I had over everybody else we have the experience across all the industries that matter so when you come to us we know how to do this in the finance industry in the retail industry and the health industry and the government so that that's something also that so I'll just for the audience out there arm is coming back at two third you're gonna go deeper in today's the highly decorated or a general because there is there a leak oh and thanks for the small extra info he's in the uniform to the cloud era logo yes sir affecting some of those for us to someday great so what you see you again love love our great great friend

Yeah, I'm Aala, They're the co-founder back to back. This is the cube silicon angle.com, Silicon angle dot TV's production of the cube, our flagship telecasts. We go out to the event. That was a great conversation. I was really just, just cool. I could have, we could have probably hit on a few more things, obviously well read. Awesome. Co-founder of Cloudera a. You were, you did a good job teaming up with that co-founder, huh? Not bad on the cube, huh? He's not bad on the cube, isn't he? He, >>He reads the internet. >>That's what I'm saying. >>Anything is going on. >>He's a cube star, you know, And >>Technology. Jeff knows it. Yeah. >>We, we tell you, I'm smarter just by being in Cloudera all those years. And I actually was following what he was saying, Sad and didn't dust my brain. So, Okay, so you're back. So we were talking earlier with Michaels and about the relational database thing. So I kind of pick that up where we left off with you around, you know, he was really excited. It's like, you know, hey, we saw that relational database movement happen. He was part of that. Yeah, yeah. That generation. And then, but things were happening or kind of happening the same way in a similar way, still early. So I was trying to really peg with him, how early are we, like, so, you know, as the curve, you know, this is 1400, it's not the Javit Center yet. Maybe the Duke world, you know, next year might be at the Javit Center, 35,000 just don't go to Vegas. So I'm trying to figure out where we are on that curve. Yeah. And we on the upwards slope, you know, down here, not even hitting that, >>I think, I think, I think we're moving up quicker than previous waves. And actually if you, if you look for example, Oracle, I think it took them 15, 20 years until they, they really became a mature company, VM VMware, which started about, what, 12, 13 years ago. It took them about maybe eight years to, to be a big company, met your company, and I'm hoping we're gonna do it in five. So a couple more years. >>Highly accelerated. >>Yes. But yeah, we see, I mean, I'm, I'm, I've been surprised by the growth. I have been, Right? I've been told, warned about enterprise software and, and that it takes long for production to take place. >>But the consumerization trend is really changing that. I mean, it seems to be that, yeah, the enterprises always last. Why the shorter >>Cycle? I think the shorter cycle is coming from having the, the, the, the right solution for the right problem at the right time. I think that's a big part of it. So luck definitely is a big part of this. Now, in terms of why this is changing compared to a couple of dec decades ago, why the adoption is changing compared to a couple of decades ago. I, I think that's coming just because of how quickly the technology itself, the underlying hardware is evolving. So right now, the fact that you can buy a single server and it has eight cores to 16 cores has 12 hards to terabytes. Each is, is something that's just pushing the, the, the, the limits what you can do with the existing systems and hence making it more likely for new systems to disrupt them. >>Yeah. We can talk about a lot. It's very easy for people to actually start a, a big data >>Project. >>Yes. For >>Example. Yes. And the hardest part is, okay, what, what do I really, what problem do I need to solve? How am I gonna, how am I gonna monetize it? Right? Those are the hard parts. It's not the, not the underlying >>Technology. Yes, Yes, that's true. That's true. I mean, >>You're saying, eh, you're saying >>Because, because I'm seeing both so much. I'm, I'm seeing both. I'm seeing both. And like, I'm seeing cases where you're right. There's some companies that was like, Oh, this Hadoop thing is so cool. What problem can I solve with it? And I see other companies, like, I have this huge problem and, and, and they don't know that HA exists. It's so, And once they know, they just jump on it right away. It's like, we know when you have a headache and you're searching for the medicine in Espin. Wow. It >>Works. I was talking to Jeff Hiba before he came on stage and, and I didn't even get to it cuz we were so on a nice riff there. Right. Bunch of like a musicians playing the guitar together. But like he, we talked about the it and and dynamics and he said something that I thoughts right. On money and SAP is talking the same thing and said they're going to the lines of business. Yes. Because it is the gatekeeper that's, it's like selling mini computers to a mainframe selling client servers from a mini computer team. Yeah. >>There's not, we're seeing, we're seeing both as well. So more likely the, the former one meaning, meaning that yes, line of business and departments, they adopt the technology and then it comes in and they see there's already these five different departments having it and they think, okay, now we need to formalize this across the organization. >>So what happens then? What are you seeing out there? Like when that happens, that mean people get their hands on, Hey, we got a problem to solve. Yeah. Is that what it comes down to? Well, Hadoop exist. Go get Hadoop. Oh yeah. They plop it in there and I what does it do? They, >>So they pop it into their, in their own installation or on the, on the cloud and they show that this actually is working and solving the problem for them. Yeah. And when that happens, it's a very, it's a very easy adoption from there on because they just go tell it, We need this right now because it's solving this problem and it's gonna make, make us much >>More money moving it right in. Yes. No problems. >>Is is that another reason why the cycle's compressed? I mean, you know, you think client server, there was a lot of resistance from it and now it's more much, Same thing with mobile. I mean mobile is flipped, right? I mean, so okay, bring it in. We gotta deal with it. Yep. I would think the same thing. We, we have a data problem. Let's turn it into an >>Opportunity. Yeah. In my, and it goes back to what I said earlier, the right solution for the right problem at the right time. Like when they, when you have larger amounts of unstructured data, there isn't anything else out there that can even touch what had, can >>Do. So Amar, I need to just change gears here a minute. The gaming stuff. So we have, we we're featured on justin.tv right now on the front page. Oh wow. But the numbers aren't coming in because there's a competing stream of a recently released Modern Warfare three feature. Yes. Yes. So >>I was looking for, we >>Have to compete with Modern Warfare three. So can you, can we talk about Modern Warfare three for a minute and share the folks what you think of the current version, if any, if you played it. Yeah. So >>Unfortunately I'm waiting to get back home. I don't have my Xbox with me here. >>A little like a, I'm talking about >>My lines and business. >>Boom. Water warfares like a Christmas >>Tree here. Sorry. You know, I love, I'm a big gamer. I'm a big video gamer at Cloudera. We have every Thursday at five 30 end office, we, we play Call of of Beauty version four, which is modern world form one actually. And I challenge, I challenge people out there to come challenge our team. Just ping me on Twitter and we'll, we'll do a Cloudera versus >>Let's, let's, let's reframe that. Let team out. There am Abalas company. This is the geeks that invent the future. Jeff Haer Baer at Facebook now at Cloudera. Hammerer leading the charge. These guys are at gamers. So all the young gamers out there am are saying they're gonna challenge you. At which version? >>Modern Warfare one. >>Modern Warfare one. Yes. How do they fire in? Can you set up an >>External We'll >>We'll figure it out. We'll figure it out. Okay. >>Yeah. Just p me on Twitter and We'll, >>We can carry it live actually we can stream that. Yeah, >>That'd be great. >>Great. >>Yeah. So I'll tell you some of our best Hadooop committers and Hadoop developers pitch >>A picture. Modern Warfare >>Three going now Model Warfare three. Very excited about the game. I saw the, the trailers for it looks, graphics look just amazing. Graphics are amazing. I love the Sirius since the first one that came out. And I'm looking forward to getting back home to playing the game. >>I can't play, my son won't let me play. I'm such a fumbler with the Hub. I'm a keyboard controller. I can't work the Xbox controller. Oh, I have a coordination problem my age and I'm just a gluts and like, like Dad, sorry, Charity's over. I can I play with my friends? You the box. But I'm around big gamer. >>But, but in terms of, I mean, something I wanted to bring up is how to link up gaming with big data and analysis and so on. So like, I, I'm a big gamer. I love playing games, but at the same time, whenever I play games, I feel a little bit guilty because it's kind of like wasted time. So it's like, I mean, yeah, it's fun and I'm getting lots of enjoyment on it makes my life much more cheerful. But still, how can we harness all of this, all of these hours that gamers spend playing a game like Modern Warfare three, How can we, how can we collect instrument, all of the data that's coming from that and coming up, for example, with something useful with predicted. >>This is exactly, this is exactly the kind of application that's mainstream is gaming. Yeah. Yeah. Danny at Riot G is telling me, we saw him at Oracle Open World. He's up there for the Java one. He said that they, they don't really have a big data platform and their business is about understanding user behavior rep tons of data about user playing time, who they're playing with. Yeah, Yeah. How they want us to get into currency trading, You know, >>Buy, I can't, I can't mention the names, but some of the biggest giving companies out there are using Hadoop right now. And, and depending on CDH for doing exactly that kind of thing, creating >>A good user experience >>Today, they're doing it for the purpose of enhancing the user experience and improving retention. So they do track everything. Like every single bullet, you fire everything in best Ball Head, you get everything home run, you do. And, and, and in, in a three >>Type of game consecutive headshot, you get >>Everything, everything is being Yeah. Headshot you get and so on. But, but as you said, they are using that information today to sell more products and, and, and retain their users. Now what I'm suggesting is that how can you harness that energy for the good as well? I mean for making money, money is good and everything, but how can you harness that for doing something useful so that all of this entertainment time is also actually productive time as well. I think that'd be a holy grail in this, in this environment if we >>Can achieve that. Yeah. It used to be that corn used to be the telegraph of the future of about, of applications, but gaming really is, if you look at gaming, you know, you get the headset on. It's a collaborative environment. Oh yeah. You got unified communications. >>Yeah. And you see our teenager kids, how, how many hours they spend on these things. >>You got play as a play environments, very social collaborative. Yeah. You know, some say, you know, we we're saying, what I'm saying is that that's the, that's the future work environment with Skype evolving. We're our multiplayer game's called our job. Right? Yeah. You know, so I'm big on gaming. So all the gamers out there, a has challenged you. Yeah. Got a big data example. What else are we seeing? So let's talk about the, the software. So we, one of the things you were talking about that I really liked, you were going down the list. So on Mike's slide he had all the new features. So around the core, can you just go down the core and rattle off your version of what, what it means and what it is. So you start off with say H Base, we talked about that already. What are the other ones that are out there? >>So the projects that we have right there, >>The projects that are around those tools that are being built. Cause >>Yeah, so the foundational, the foundational one as we mentioned before, is sdfs for storage map use for processing. Yeah. And then the, the immediate layer above that is how to make MAP reduce easier for the masses. So how can, not everybody knows how to learn map, use Java, everybody knows sql, right? So, so one of the most successful projects right now that has the highest attach rate, meaning people usually when they install had do installed as well is Hive. So Hive takes sequel and so Jeff Harm Becker, my co-founder, when he was at Facebook, his team built the Hive system. Essentially Hive takes sql so you don't have to learn a new language, you already know sql. And then converts that into MAP use for you. That not only expands the developer base for how many people can use adu, but also makes it easier to integrate Hadoop through all DBC and JDBC integrated with BI tools like MicroStrategy and Tableau and Informatica, et cetera, et cetera. >>You mentioned R too. You mentioned R Program R >>As well. Yeah, R is one of our best partnerships. We're very, very happy with them. So that's, that's one of the very key projects is Hive assisted project to Hive ISS called Pig. A pig Latin is a language that ya invented that you have to learn the language. It's very easy, it's very easy to learn compared to map produce. But once you learn it, you can, you can specify very deep data pipelines, right? SQL is good for queries. It's not good for data pipelines because it becomes very convoluted. It becomes very hard for the, the human brain to understand it. So Pig is much more natural to the human. It's more like Pearl very similar to scripting kind of languages. So with Peggy can write very, very long data pipelines, again, very successful projects doing very, very well. Another key project is Edge Base, like you said. So Edge Base allows you to do low latencies. So you can do very, very quick lookups and also allows you to do transactions. So you can do updates in inserts and deletes. So one of the talks here that had World we try to recommend people watch when the videos come out is the Talk by Jonathan Gray from Facebook. And he talked about how they use Edge Base, >>Jonathan, something on here in the Cube later. Yeah. So >>Drill him on that. So they use Edge Base now for many, many things within Facebook. They have a big team now committed to building an improving edge base with us and with the community at large. And they're using it for doing their online messaging system. The live mail system in Facebook is powered by Edge Base right now. Again, Pro and eBay, The Casini project, they gave a keynote earlier today at the conference as well is using Edge Base as well. So Edge Base is definitely one of the projects that's growing very, very quickly right now within the Hudu system. Another key project that Jeff alluded to earlier when he was on here is Flum. So Flume is very instrumental because you have this nice system had, but Hadoop is useless unless you have data inside it. So how do you get the data inside do? >>So Flum essentially is this very nice framework for having these agents all over your infrastructure, inside your web servers, inside your application servers, inside your mobile devices, your network equipment that collects all of that data and then reliably and, and materializes it inside Hado. So Flum does that. Another good project is Uzi, so many of them, I dunno how, how long you want me to keep going here, But, but Uzi is great. Uzi is a workflow processing system. So Uzi allows you to define a series of jobs. Some of them in Pig, some of them in Hive, some of them in map use. You can define a series of them and then link them to each other and say, only start this job when these other jobs, two jobs finish because I'm waiting for the input from them before I can kick off and so on. >>So Uzi is a very nice framework that will will do that. We'll manage the whole graph of jobs for you and retry things when they fail, et cetera, et cetera. Another good project is where W H I R R and where allows you to very easily start ADU cluster on top of Amazon. Easy two on top of Rackspace, virtualized environ. It's more for kicking off, it's for kicking off Hadoop instances or edge based instances on any virtual infrastructure. Okay. VMware, vCloud. So that it supports all of the major vCloud, sorry, all of the me, all of the major virtualized infrastructure systems out there, Eucalyptus as well, and so on. So that's where W H I R R ARU is another key project. It's one, it's duck cutting's main kind of project right now. Don of that gut cutting came on stage with you guys has, So Aru ARO is a project about how do we encode with our files, the schema of these files, right? >>Because when you open up a text file and you don't know how to what the columns mean and how to pars it, it becomes very hard to work for it. So ARU allows you to do that much more easily. It's also useful for doing rrp. We call rtc remove procedure calls for having different services talk to each other. ARO is very useful for that as well. And the list keeps going on and on Maha. Yeah. Which we just, thanks for me for reminding me of my house. We just added Maha very recently actually. What is that >>Adam? I'm not >>Familiar with it. So Maha is a data mining library. So MAHA takes some of the most popular data mining algorithms for doing clustering and regression and statistical modeling and implements them using the map map with use model. >>They have, they have machine learning in it too or Yes, yes. So that's the machine learning. >>So, So yes. Stay vector to machines and so on. >>What Scoop? >>So Scoop, you know, all of them. Thanks for feeding me all the names. >>The ones I don't understand, >>But there's so many of them, right? I can't even remember all of them. So Scoop actually is a very interesting project, is short for SQL to Hadoop, hence the name Scoop, right? So SQ from SQL and Oops from Hadoop and also means Scoop as in scooping up stuff when you scoop up ice cream. Yeah. And the idea for Scoop is to make it easy to move data between relational systems like Oracle metadata and it is a vertical and so on and Hadoop. So you can very simply say, Scoop the name of the table inside the relation system, the name of the file inside Hadoop. And the, the table will be copied over to the file and Vice and Versa can say Scoop the name of the file in Hadoop, the name of the table over there, it'll move the table over there. So it's a connectivity tool between the relational world and the Hadoop world. >>Great, great tutorial. >>And all of these are Apache projects. They're all projects built. >>It's not part of your, your unique proprietary. >>Yes. But >>These are things that you've been contributing >>To, We're contributing to the whole ecosystem. Yes. >>And you understand very well. Yes. And >>And contribute to your knowledge of the marketplace >>And Absolutely. We collaborate with the, with the community on creating these projects. We employ committers and founders for many of these projects. Like Duck Cutting, the founder of He works in Cloudera, the founder for that UIE project. He works at Calera for zookeeper works at Calera. So we have a number of them on stuff >>Work. So we had Aroon from Horton Works. Yes. And and it was really good because I tell you, I walk away from that conversation and I gotta say for the folks out there, there really isn't a war going on in Apache. There isn't. And >>Apache, there isn't. I mean isn't but would be honest. Like, and in the developer community, we are friends, we're working together. We want to achieve the, there's >>No war. It's all Kumbaya. Everyone understands the rising tide floats, all boats are all playing nice in the same box. Yes. It's just a competitive landscape in Horton. Works >>In the business, >>Business business, competitive business, PR and >>Pr. We're trying to be friendly, as friendly as we can. >>Yeah, no, I mean they're, they're, they're hying it up. But he was like, he was cool. Like, Hey, you know, we know each other. Yes. We all know each other and we're just gonna offer free Yes. And charge with support. And so are they. And that's okay. And they got other things going on. Yes. But he brought up the question. He said they're, they're launching a management console. So I said, Tyler's got a significant lead. He kind of didn't really answer the question. So the question is, that's your core bread and butter, That's your yes >>And no. Yes and no. I mean if you look at, if you look at Cloudera Enterprise, and I mentioned this earlier and when we talked in the morning, it has two main things in it. Cloudera Enterprise has the management suite, but it also has the, the the the support and maintenance that we provide to our customers and all the experience that we have in our team part That subscription. Yes. For a description. And I, I wanna stress the point that the fact that I built a sports car doesn't mean that I'm good at running that sports car. The driver of the car usually is much better at driving the car than the guy who built the car, right? So yes, we have many people on staff that are helping build had, but we have many more people on stuff that helped run Hado at large scale, at at financial indu, financial industry, retail industry, telecom industry, media industry, health industry, et cetera, et cetera. So that's very, very important for our customer. All that experience that we bring in on how to run the system technically Yeah. Within these verticals. >>But their strategies clear. We're gonna create an open source project within Apache for a management consult. Yes. And we sell support too. Yes. So there'll be a free alternative to management. >>So we have to see, But I mean we look at the product, I mean our products, >>It's gotta come down to product differentiation. >>Our product has been in the market for two years, so they just started building their products. It's >>Alpha, It's just Alpha. The >>Product is Alpha in Alpha right now. Yeah. Okay. >>Well the Apache products, it is >>Apache, right? Yeah. The Apache project is out. So we'll see how it does it compare to ours. But I think ours is way, way ahead of anything else out there. Yeah. Essentially people to try that for themselves and >>See essentially, John, when I asked Arro why does the world need Hortonwork? You know, eventually the answer we got was, well it's free. It needs to be more open. Had needs to be more open. >>No, there's, >>It's going to be, That's not really the reason why Warton >>Works. >>No, they want, they want to go make money. >>Exactly. We wasn't >>Gonna say them you >>When I kept pushing and pushing and that's ultimately the closest we can get cuz you >>Just listens. Not gonna >>12 open source projects. Yes. >>I >>Mean, yeah, yeah. You can't get much more open. Yeah. Look >>At management >>Consult, but Airs not shooting on all those. I mean, I mean not only we are No, no, not >>No, no, we absolutely >>Are. No, you are contributing. You're not. But that's not all your projects. There's other people >>Involved. Yeah, we didn't start, we didn't start all of these projects. Yeah, that's >>True. You contributing heavily to all of them. >>Yes, we >>Are. And that's clear. Todd Lipkin said that, you know, he contributed his first patch to HPAC in 2008. Yes. So I mean, you go back through the ranks >>Of your people and Todd now is a committer on Edge base is a committer on had itself. So on a number >>Of you clearly the lead and, and you know, and, but >>There is a concern. But we, we've heard it and I wanna just ask you No, no. So there's a concern that if I build processes around a proprietary management console, Yes. I'm gonna end up being locked into that proprietary management CNA all over again. Now this is so far from ca Yes. >>Right. >>But that's a concern that some people have expressed. And, and, and I think one of the reasons why Port Works is getting so much attention. So Yes. >>Talk about that. It's, it's a very good, it's a very good observation to make. Actually, >>There there is two separate things here. There's the platform where all the data sets and then there's this management parcel beside the platform. Now why did we make the management console why the cloud didn't make the management console? Because it makes our job for supporting the customers much more achievable. When a customer calls in and says, We have a problem, help us fix this problem. When they go to our management console, there is a button they click that gives us a dump of the state, of the cluster. And that's what allows us to very quickly debug what's going on. And within minutes tell them you need to do this and you to do that. Yeah. Without that we just can't offer the support services. There's >>Real value there. >>Yes. So, so now a year from, But, but, but you have to keep in mind that the, the underlying platform is completely open source and free CBH is completely a hundred percent open source, a hundred percent free, a hundred percent Apache. So a year from now, when it comes time to renew with us, if the customer is not happy with our management suite is not happy with our support data, they can, they can go to work >>And works. People are afraid >>Of all they can go to ibm. >>The data, you can take the data that >>You don't even need to take the data. You're not gonna move the data. It's the same system, the same software. Every, everything in CDH is Apache. Right? We're not putting anything in cdh, which is not Apache. So a year from now, if you're not happy with our service to you and the value that we're providing, you can switch. There is no lock in. There is no lock. And >>Your, your argument would be the switching costs to >>The only lock in is happiness. The only lock in is which >>Happiness inspection customer delay. Which by, by the way, we just wrote a piece about those wars and we said the risk of lockin is low. We made that statement. We've got some heat for it. Yes. And >>This is sort of at scale though. What the, what the people are saying, they're throwing the tomatoes is saying if this is, again, in theory at scale, the customers are so comfortable with that, the console that they don't switch. Now my argument was >>Yes, but that means they're happy with it. That means they're satisfied and happy >>With it. >>And it's more economical for them than going and hiding people full-time on stuff. Yeah. >>So you're, you're always on check as, as long as the customer doesn't feel like Oracle. >>Yeah. See that's different. Oracle is very, Oracle >>Is like different, right? Yeah. Here it's like Cisco routers, they get nested into the environment, provide value. That's just good competitive product strategy. Yes. If it they're happy. Yeah. It's >>Called open washing with >>Oracle, >>I mean our number one core attribute on the company, the number one value for us is customer satisfaction. Keeping our people Yeah. Our customers happy with the service that we provide. >>So differentiate in the product. Yes. Keep the commanding lead. That's the strategist. That's the, that's what's happening. That's your goal. Yes. >>That's what's happening. >>Absolutely. Okay. Co-founder of Cloudera, Always a pleasure to have you on the cube. We really appreciate all the hospitality over the beer and a half. And wanna personally thank you for letting us sit in your office and we'll miss you >>And we'll miss you too. We'll >>See you at the, the Cube events off Swing by, thanks for coming on the cube and great to see you and congratulations on all your success. >>Thank >>You. And thanks for the review on Modern Warfare three. Yeah, yeah. >>Love me again. If there any gaming stuff, you know, I.

>>Ed, welcome to the Cube. All right, Thanks guys. Good >>To see you. Thanks. Good to see you as well, >>John. Okay. Ed runs Biz dev for Cloudera, Industry veteran, worked at VMware. Ed, gotten to know you the past year. You guys have been doing great. What a difference one year makes, right? I mean, absolutely. Tell us, just let's start it off with what's happened in a year. I mean, you know, here at Hadoop World Cloudera, the ecosystem. Just give us your view of your perspective of what a difference one year makes. >>I think more than double is probably the, the fastest answer I could give you, which is, I mean, even looking around at the conference, it's, it itself is literally double from what it was last year. But in terms of the number of partners that have entered the market and really decided to work with, with Cloudera, but also in general, just the, the, the, the scope and size of the ecosystem itself, investors from every angle. You've got companies really well-branded marquee companies like Oracle coming into the mix and saying, Hey, Hadoop is the, is the real deal and we need to invest here. Marquee companies like IBM and EMC also doing the same. And of course, you know, as a result, you know, lots and lots of customer interest in the technology. And Cloudera's been fortunate to have been in the market early and really made the right investments with the right team. And so we're able to serve a lot of those customer needs. So it's been really, it's been a fantastic year for the company. >>So we had a great day yesterday with Cloudera. We had Kirk on, we had AER on twice, who by the way went viral with his modern warfare review, but we had Jeff Harmar Baer on, so we had pretty much the brain trust, Mike and Michaelson. Yep. The brain trust, the Cloudera. So we talked about the risk factors for Cloudera. Obviously you guys are number one, you've been kind of had untouchable lead and then all of a sudden boom competition. So Mike talked about that. So the strategy and the product side, they addressed, you're on the, the biz dev side, so you know, when you were number one, everyone wants to stand next to you and your phone rings off the hook from tier one partners all the way down to anyone's just getting in the business. Who wants a big data strategy on the execution. Now, what are you guys doing right now to, to continue your lead on the, on the sales marketing biz dev? I mean, I know you get the partner program, but what's your strategy for Phil, how to continue >>In that lead? The, the beautiful thing is honestly, our strategy hasn't changed at all. And I know that might sound counterintuitive, but we started off with a, a really crisp vision. And we want, what we wanna do is create a very attractive platform for partners. And, and, you know, one of the core, you know, sort of corporate strategy, Edix for Quadera is a recognition that the end of the day, the platform itself, Hado is an input into a solution. And Quadra is not likely to deliver the complete solution to market. Instead, it's going to be companies like Dell, for example, or it's going to be companies on the, on the ISV side like Informatica, which you're gonna deliver not only a base platform, but also the, the, the, the BI or analytics or data integration technologies on top. And as a result, what we've done is we've really focused in on creating a very attractive platform to vendors to build on. >>And one of the, I think one of the biggest misconceptions that I'm excited about that, you know, we are now having an opportunity to correct and that's a result, frankly, of the additional competitive dynamic. And I think the, the Wiki bond team pointed that out rather pointedly in their most recent articles. But is, is the sort of the lack of understanding around what CDH is and also the, some of the other investments that we're making to create a truly attractive platform for vendors to build on. And you know, I mean, I think you, you may have familiarity with exactly what CDH is, but for the sake of the audience here, what I'd like to do is say, say, first off, you know, first and foremost this is a hundred percent free in Apache license open source. But more importantly, it is everything that we build on the platform, meaning it's completely full featured. >>We put all of that out in the open. There's no turbo version of Hadoop that we've got hiding in the closet for our, our four pay customers. We're absolutely making investment. But I think, you know, when you think about it from the vendor perspective, and that's my bias. So I always think about, I treat all of the potential partners as really my customer. And when you think about it from that perspective, the things that matter most to vendors, number one, transparency. They need to understand exactly what our business model is, where we plan to make money and where we plan, don't make money. They need to know what we're really good at developing and what we're not so good at developing. And sort of where we draw the, the boundaries around that investment. I think, you know, a testament to that, for example, is tomorrow we're hosting a partner summit. >>So after this event, there are gonna be over 60 individuals, but they max two per per vendor. So we're gonna have over 35 vendors attending this event. And what they're gonna hear from is our entire management team is as deeply as we can and as open as we can. And you know, it, it's, it's, it's funny, you know, I think I saw this article in Forbes the other day about Cloudera. It was this, the title of the article was something like Spies Like Us. And it it, and it, what it highlighted was that some, some competitor of Cloudera had actually hired a, a, a competitive intelligence agency to go on and, and try to engage with, you know, and, and try to learn more about Cloudera. And so they went on to Cora, which we have a lot of active engineers on Cora. And they, you know, they went out and they asked a bunch of product related questions to our to, to someone on Cora. And our engineers immediately responded and they started being very transparent, completely open to what, what they're building and why they're building it. And the article basically summarized to say, Hey, you know what, you know, clearly some people aren't all that sophisticated in figuring out, you know, who they're talking to. And it's really important to do that. And they got the absolute wrong conclusion. Our engineers are actually encouraged and in fact rewarded for being extremely transparent in the market because we believe that it's transparency will ultimately allow us to be that platform vendor. >>And that's what attracts me. Jeff Hummer Bucker, who's active on core as well, he's recruiting there too. So you guys are out engaging the community. Yeah. So just let me just review, cuz this is cool that you're addressing this because Hortonworks and others, and I'll say the name Hortonworks has been pumping up the PR and creating a lot of noise around open and kind of Depositioning Cloudera. So you guys are completely open, a hundred percent Hadoop, open source, everything you build in, in every way, in every way. You have engineers building core, you've got tools and all the other stuff is being built in Cloudera then contributing into the community. >>Actually it's the other way around. We build it and the community@apache.org. So all of our technology is built@apache.org. It's, it's developed there. It's, it's, it's initially shared there. And then we have another team inside our company that pulls down bits from apache.org and then assembles them and integrates them. So it's really, it's a really key thing. And there's no, we do, we have no bits that we don't develop@apache.org that are part of cdh. So there, I mean there can be no mistake that everything that that is in CDH is everything we got. >>So CDH is free. >>It is free >>And every it's open source. It's open you >>Charge enterprise edition. That's the only thing that's different you guys charge >>Yeah. Which is your management console, right. >>Management >>Suite and all kinds of >>The tools. And that's not free and that's not open source. That's correct. Just to be clear. Yep. But so AER took us yesterday through, I don't know, half a dozen probably open source projects and then the one is the, the management console. And that's what you charge for, that's where you're gonna make money? >>Yeah. We, we manufacture, essentially we manufacture two products, but we sell one. So we manufacture the Quadera distribution, including Apache Duke, that's free. It's free. And then we all in open source and built it Apache and, and really heavily tested and well documented and, and, and well integrated. And then we also manufacture quadera Enterprise, which includes support and indemnities and warranties for that full featured CDH product and also includes the Quadra management suite. And >>That's a subscription. >>And that's a subscription. And so customers can, can run cdh, they can then buy and license Cloudera Enterprise and then someday if they decide they don't need Cloud Air Enterprise for whatever reason, if they're, if their team are scripting wizards and they've decided that they, you know, they don't need the extra opportunity for being able to track all of the things that Cloudier Enterprise allows 'em to, they can step off of cloud enterprise and continue to use full feature to do as they see >>Fit. So take an example of one of your partners that you announced this week. NetApp NetApp's gonna package your cdh CDH and the subscription Correct. To their, their customers. And then they're gonna let their channel either, you know, they'll pre bule it or do a reference architecture, you'll get paid for that subscription that's bundled. That's correct. Will make money off of its filers. Yes. And the customer gets a package solution. >>Exactly. Right. And in fact, that's another important thing that you know, is probably worth discussing, which is our go to market model. I don't know if you guys had a chance to talk with anyone yesterday on that, but I'm responsible for our channel strategy and one of the key things that we've agreed to as a, as a company is that we really are gonna go to market through channel partners. Yeah. >>We covered sgi, that was a great announcement. >>Yep, a >>Hundred percent >>As, as close as we can get. Okay. I mean that is our, he's >>Still doing the direct deals. You still have that belly to belly sales force because it's still early, right? So there's a mix of direct and indirects, not a pure >>Indirect, but as, and that's only, that's only as we're able to, until we're able to ramp up our partners fully, in which case we really want our, the current team that is working belly to belly to really support our partners. >>So all so VMware like, but I I wanted to ask >>You VMware, like NetApp, like very similar. >>Yes. Very, very NetApp. Like NetApp probably 75%, you know. Exactly. What are the similarities and differences with VMware in, in the ecosystem? You know it well, >>I do know it well. Yeah. I spent several years working at VMware and you know, I think, I mean the first and most obvious difference is that when you think, when I think about platform software in general, you know, there are a few different flavors of platform. One of the things that makes Hadoop very unique, very unique relative to other platforms is that it, not only is it Apache license, but it really is, it's dependent upon other external innovators to, to create the entire full value of the ecosystem. So, or, or you know, of the solution, right? So unlike for example, so like, let's take a platform like everyone's familiar with like Apple iTunes, right? What happens is Apple creates the platform and they put it kind of in the middle on top of and behind the scenes is the innovator, the app builder, he builds it, he publishes it on Apple, and then Apple controls all access to the >>Customer. Yep. >>That's not adu, right? Right. Let's take VMware or Red Hat for example. So in that case, they publish a platform they own and control the, the absolute structure and boundaries of what that platform is. And then on top of that application vendors build and then they deliver to the, the customer. But you know, at the end of the day, the, you know, the relationship really is, you know, from that external innovator straight down, and there's no, there's, you know, there's no way for them to really modify the platform. And you take kadu, which is a hundred percent Apache licensed to open source, and you really, you really open up the opportunity for vendors to take ADU as an input into their system and then deliver it straight to their customers or for customers themselves to say, I want straight up vanilla Hadoop, I'm gonna go this way and I'm gonna add on my own be app of applications. So you're, we're seeing all sorts of variants right now in the market. We're seeing software as a service being delivered that's based on Hadoop. There was a great announcement a few weeks ago from a company named Tidemark, previously known as Per Ferry, and they're taking all of cdh. They're, but they're, the customer doesn't know that they're, and what they're doing is they're delivering software as a, as a service based on adu. >>Yeah. So I mean, you know, we are psyched that you're clearing this up because obviously we're seeing, we saw all that stuff, but I really think that indirect strategy as a home run, I'm said it when we talked about the SGI thing, and it's accelerates you guys, you enable, but you know, channels is an interesting business. I mean the, you have to have pure transparency as you mentioned, but they need comp, people need confidence and, and they don't, they worry about competition. So channel conflict is always the big issue, right? Right. Is Cloudera gonna compete with us? So talk that, talk us through that, that strategy. So obviously the market's growing, new solutions are coming around the corner, These guys wanna make money. I mean channel, it's all about, you know, what have you done for me today? >>Right. That, that is exactly right. And you know what, that's, that's why we decided on the channel strategy specifically around our product is because we recognize that each and every single potential channel partner of ours can actually innovate themselves on top of and create differentiation. And we're not an obstacle to that process. So we provide our platform as an input and we're capable of managing that platform, but ultimately creating differentiation is all in the hands of our partners and we're there to help, but it gives them wide latitudes. So take for example, the differences between Dell and NetApp solution, they are very different reference architectures leveraging the exact same platform. >>Yeah. And they have to make money. I mean, the money making side of it is, you know, people have kind of, don't really talk about that, but, you know, channel partners loyalty is all about who can help them make cash. Right. Right. Exactly. What are you hearing there in terms of the ecosystem? Has the channels Bess and the partnerships or the more as size, what's the profile of your, of your partners? I mean, can you give us the breakdown of Sure. We have what you look like from Dell. We know Dell and NetApp, but they're gear guys. But, >>So a big part of our strategy is to work with IHVs and then Ihv resellers. So you're talking about companies like Dell, like sgi, like NetApp, for example, independent hardware manufacturers. Another part of our strategy though, and a key, a key requirement from our customers is to work with a whole variety of ISVs, particularly in the data management space. So you've got really marquee companies in the database space like IBM's Netezza or Terradata. You've got in companies like Informatica and Talent, you've got companies on the BI side, like Micro Strategy and Tableau. These kinds of technologies are currently in play at our customers that have made substantial investments. And ultimately they want to be able to continue to leverage them with the data platform, whichever data platform that they end up choosing. So we invest considerably there. A big part of that has been our Qera Connect partner program. >>It's an opportunity for us to help the customer to understand which technologies work and work well with, with our platform. It's also an opportunity for us to engage directly and assist the vendor. So one of the things that we created as part of that program is first off, immediate and absolute discounted access to any part of our training. Second, lots of free information, access to our world class knowledge base, access to our support team, direct access to our support team. The, the vendors also get access to a developer portal that would created specifically for them. So if, if you think about it this way, Hadoop gets built@apache.org, but solutions don't get built@apache.org. Right? So what we're really trying to help our vendors do is be able to develop their solutions by having real clear visibility to the API level points of Hadoop. They're not necessarily interested in, in trying to figure out how, how MR two works or, or contributing code to that. >>But they absolutely are interested in figuring out how to run and execute their software on top of a do. So when I think about the things that matter to create an attractive platform, and at the end of the day, that's what we're really trying to do, first and foremost is transparency, right? Second really ultimately is really clear visibility to the APIs and the documentation of that platform so that there's no ambiguity that the, the vendor, this is the user in this case, it's building a solution, can absolutely absorb all of that content really cleanly. And then ultimately, you know, I think it's customers, right? Users of the technology. And I think our download numbers are, they're, they're, there's something we're proud of. >>We, we are, we're hearing good feedback. I mean, the feedback we hear from folks is, yeah, I love how they take away the complexity of handling versions and whatnot. So, you know, I think totally is a great way, The CDH is a great bundle. You know, the questions that we have for you is what are you hearing about the other products, the ones you're actually selling? Does that create the lock in? So that's something that we asked Elmer directly, you know, is that the, is that the lock in and what happens when the deployments get so big? You know, >>I mean, the way, I >>Don't really see an issue there, but that's what people are afraid of. I mean, that's kind of the, it's more of fear. I mean, some people can use that fear and, and >>Play against. I think, I think what we've seen in other markets is that management tools are ultimately interchangeable. And the only way that we're gonna retain a customer is by out innovating the competition on the management side, the lock in, the lock in component, as you will, is not really part of our business model. It's very difficult to achieve with an Apache licensed platform and a management suite that sits on outside of that, that licensed artifact. So ultimately, if we don't owe innovate, we're gonna lose. So we're working on the innovation and that's, >>How's the hiring go? Oh, go ahead. >>I, I had a, I wanted to come back to that. You mentioned download numbers. Can you share the numbers >>With the others? I can't, I can't share them publicly, but what I can say is that they've been on an incredible trajectory. Okay. That, and what we've seen is month to month growth rates, every single month we continue to see really significant growth rates. >>And then I, I had a follow up question on, you talked about the, the partner program. How do you manage all those partners? How do you prioritize them? I mean, the, the hardware vendors, it's pretty easy. There's a few big whales, but the, the ISVs, they're, I mean, your phone, like John said, must be ringing off the hook. How do you juggle that and, and can you do it better than VMware, for example? >>Well, we do it, we handle the, the influx of partner interest in two ways. One, we've been relatively structured with the Quadra Connect partner program, and we make real investments there. So we have dedicated folks that are there to help. We have our engineering team that is actually feeding inputs, and we're, we're leveraging some of the same resources that we provide to our customers and feeding those directly to our partners as well. So that's one way that we handle it. But the other way, frankly, is, I mean, customers help here having access to and, and a real customer population, they help you set priorities pretty quickly. And so we're able to understand what we track in inside of our systems, which, which technologies our customers use. So we know, for example, what percentage of our customer base has has SaaS installed, and we'd like to use that with a, do we know which percentage of our customer base is currently running on Red Hat and which is not. So having core visibility, that helps us to prioritize. >>How about incentives? I mean, obviously channel businesses as, like I said, very fickle people, you know, you know the channel business, I spent, you know, almost a decade in, in HP's channel organization and you know, you have to provide soft dollars. There's a lot of kind of blocking and tackling. You guys are clearly building out that tier one with the SGIs of the world and other vendors, and then get the partner connect program for kinda everyone else who's gonna grow up into a tier one. Yeah. Training, soft dollars incentives. You guys have that going yet, or is the >>Roadmap? We do. And in fact, you know, in addition to the sort of more wide publicized relationships you see with companies like Dell and Cloudera, we're actually building a very successful network of independent ours. And the VAs in general. What we do is we prioritize and select ours based on the top level relationships that we have, because that really helps them to hone in. They've got validation from, for, for example, someone that sells resells. SGI is an organization that now is heard really loud and clear from sgi the, the specific platform configurations that they're gonna represent to their customers, and they ultimately wanna represent them directly. And how we make investments is we're, I mean, the investments we're making ultimately in our sales org, I'm gonna lose the word direct from that conversation because our sales org is being built to help our partners succeed. And I think that's where you're, >>The end game is to go completely indirect and have all your support go into managing that channel. What, what's the mix of revenue generation from your partners? Obviously as a, you know, with sgi they have pre-built channels that you're funneling in, you got NetApp and they're wrapping their products and services around it. How much is services and how much is a solution specifically? Do you have any visibility or a feel for that at this >>Point? I mean, services relative to, You mean for Cloudera particularly, or for our >>Partner? No, for the, for the part. I mean, if I'm a partner, I'm like, Hey, okay, I'm gonna use cdh. I'm on bundles. I don't mind paying you a wholesale if I'm gonna be able to throw off more cash on, you know, deployment and cloud and services, et cetera. And or if I'm a product manufacturer, a product, a solution I fund you in. I need to have that step >>Up a absolutely great question. So depending upon the partner we're dealing with, they like to either monetize or generate their revenue in different ways. So for example, NetApp, NetApp is a company that has very limited services, and their, their focus is a business is really on delivering hardware and software configured together. And they, they rely heavily on a services channel to fulfill, you take in, in contrast to a company like, for example, Dell, which has a very successful services business and really is excited about having service offerings around Hadoop. So it depends upon the company. But when we talk about our VAR channel in particular, one of the things that's a, in an internal acronym, but I'll share it publicly here. We, we call our, our supervisors and what makes them super and why, why we've selected the, the, the organizations that we are selecting right now to be our bar is that they not only can fulfill orders for hardware and software, particularly data management or infrastructure software, but they also have a services team on hand because we recognize that there is a services opportunity with every Hadoop deployment. And we want our partners to have that. So as an organization, we're structuring our, our services staff to facilitate and enable our partners not to be sold >>Directly. Okay. So that's the follow up that I had tomorrow when the partners ask, Okay, what do you want to be when you're really growing up? Is it services, is it software? >>Is it Carter is a software company, Crewing through, >>Oh, er we kind of got ett, well, he didn't say it, but we said it's a operating system. Yeah. >>So given that, so given that, I mean, you can make money on services, right? People need services. Okay, great. >>And partners will make that money for >>Us. And, and you know, early on you, you had to do some of that and you're, you've been very clear about where it's going. It's hard to make money in software when you're given all the software away for free. Well, >>We're not giving all >>The software. I know you've got that piece now, but, but here's my question. As ADU goes into the enterprise, which is clearly doing, is that that whole bundling, like what you're doing with NetApp is that really ultimately how you're gonna start to, to monetize and, and successfully monetize your software, >>Is by pushing it through >>Yeah. Packaging and that bundling that solution, in other words, our enterprise customer is gonna be more receptive to that solution package than say the, the fridge that has been using Hadoop for the last >>Two or three years. I think there's no question about it. If you, if you look at what Quadra Enterprise does, I don't know if, if you've had a chance to attend any of the sessions, maybe where Quadra Enterprise is, is currently being demonstrated. >>We just had Alex Williams as about on the air. Did a review, >>Okays >>Been going good and impressed with it? >>Yeah, there's no question about it. And I, I don't, and Alex probably hasn't seen the new version that, you know, our team is working on and it's, you know, quietly working on in the background. Incredible, incredible developments in, And that's really a function of when you have direct access to so many customers and you're getting so much input and feedback and they're the kinds of access to the kinds of customers we ultimately wanna serve. So real enterprises, what you get is really fast innovation from a really talented team that knows to do well. I mean, we are years ahead on the management side. Absolutely. Years ahead. And you know, I, so I was a guy who worked at VMware for several years, and I can tell you that while the hypervisor itself was, was a core component to VMware success, the monetization strategy was very squarely around vCenter. Yeah. Yes. Out. And we're not ignorant to that. Yeah. >>You can learn a lot from your VMware experience cause absolutely. The, the market changed significantly. And, you know, >>There were free hypervisors available all of a sudden. VMware itself had a free hypervisor. We had, we had VMware server and we had also our VMware player products, right? And those were all free. And they were very good technology. They were the best available in the market for free. And they were better, in my opinion, they were better than anything else. Open or not. No, our time >>Too, since still >>Are, they were, they, they were, they were superior products in every way. But yet how VMware was successful was recognizing that in the interest of running a production environment with an sola, you need management software. And they've also built the best management software. And there's no question that we understand that strategy and >>A phenomenal ecosystem. I mean, there's the >>Similarities, right? They did. And you, and the, and the ecosystem was in, in large part predicated on transparency act, very clear access to the APIs and a willingness to help partners be successful with those APIs. And ultimately drawing a very tight box about what the company wanted to do and didn't want to do. >>I mean, look, you're not, you're not gonna lose friends when you make people money. That's my philosophy, right? I agree. So when you're in that business where you can come in and enable a channel and have options on your growth strategy, which you do, I mean, you can say, Okay, bundling, I can go, you know, I can have this sold direct, or at least as long as you've got the options, you can grow with that market. So, you know, again, the, it's a money making opportunity for the partnerships, but there's >>More than that, right? Because you mentioned Apple, iTunes, Oracle's another example. And the way you make money with Apple and the way you make money with Oracle is different than the way you make money with VMware and presumably Cloudera. >>Yeah, I mean, our strategy is, if you make this base platform easier to install, more reliable, and you make it ultimately, you know, really rock solid from an integration standpoint, more people are gonna use it. So what happens when more people use it? First thing that happens is more solu, it's out there. So it's more solutions get built. When more solutions get built, then you see more clusters get developed. When more clusters are out there, they start to move into production. And then they, they need an sla when they need an sla, Cloudera and Enterprise gets purchased. But along that path, when those solutions got built, guess what else happened? More cloud units got sold, more servers got sold, more networking. Gear got sold, more services got created. You get, you get ultimately more operating systems got sold, more databases, got data into them, more BI clients got created. The ecosystem is deep and rich, and a lot of people stand to make money hop >>In people. The water's great. >>What about, what about support? Okay, so, you know, the other guys are saying, We're just gonna make money on support. I mean support, You guys still are doing support, right? I mean, you're selling >>Support. There's no question. Quad Enterprise contains two things, right? The management suite and support this is, this is not uncomplicated technology and having a world class support team is of value and customers do want to pay for that value. But we, we believe that support in and of itself is not enough. And that ultimately, when you wanna deliver an sla, being able to call when you have a problem is the wrong approach. You want to be proactive and understand the problem well in advance of it actually occurring. That's really important. When, for example, if you're a customer, a lot of our customers have a data pipeline that >>They, they're building out basically. I mean they're, it's, it's new and emerging. So they're building out, It's not just support. They need other tools. >>Yeah. And it building out I think is an understatement for some, where some of our customers are. I mean, when you have a thousand node cluster that you're operating Yeah, Yeah. To, that's mission critical to your business. I don't think that's building out anymore. I think that's an investment in a technology that's mission critical. And what you wanna see when you have a mission critical technology is you wanna know early and often when a problem may emerge. Not, Oh, oh my gosh, we have a problem now I need to go, you know, phone a friend, phone a friend is, is kind of a last resort. We offer that. But what we really do is, and that's the, that's the beau, That's why we don't decouple our support from our management suite. It's not about phone a friend. It's about understanding the operation of your cluster the entire way through 24. >>And the other op the other thing that people don't talk about in the support is that with open source, a lot of support gets handled in the community as well. So like That's right. So in a way, you're already pre cannibalized with the community >>By us and by others. Absolutely. But you, you'll never see to that Forbes article I referenced earlier. You will never, you will not see our, our engineers are not trained to withhold information and under any circumstances to anyone free or paying. Yeah. This is about getting, You >>Don't wanna hold back your business. I mean, you have nothing to hide. It's open rights. >>Open source. It's open. And we're here to help. We're here to help. Whether you're paying us or not, >>This is value to that anticipatory >>Remediation. Yeah. That's what you're packaging and clearing up the air. Great. Great cube guest, you're awesome on the cube. Gonna have you more on because great to get the info out there. Really impressed with the channel strategy. Love the love the growth strategy, the cloud air. You guys are really impressive. I'm really, really impressed to see that you guys got everything pumping on all cylinders, Kirk, and you are cranking out on the business execution. We're in the team playing this chest mask open. Perfect. So great. Congratulations. Great. Thanks. You guys just in the financing. >>Oh, thank you as >>Well. Hey, Ed from Cloudera, clearing it up here inside the cube. We're gonna take a quick break and we'll be right back with more video. >>Thanks guys. All right.