>> Announcer: Live from Madrid, Spain, it's theCUBE, covering HPE Discover Madrid 2017, brought to you by Hewlett Packard Enterprise. >> Welcome back to Madrid, everybody, this is theCUBE, the leader in live tech coverage and my name is Dave Vellante, and I'm here with Peter Burris, this is day two of HPE Hewlett Packard Enterprise Discover in Madrid, this is their European version of a show that we also cover in Las Vegas, kind of six month cadence of innovation and organizational evolution of HPE that we've been tracking now for several years. Sharad Singal is here, he covers software architecture for the machine at Hewlett Packard Enterprise, and Matthias Becker, who's a postdoctoral researcher at the University of Bonn. Gentlemen, thanks so much for coming in theCUBE. >> Thank you. >> No problem. >> You know, we talk a lot on theCUBE about how technology helps people make money or save money, but now we're talking about, you know, something just more important, right? We're talking about lives and the human condition and >> Peter: Hard problems to solve. >> Specifically, yeah, hard problems like Alzheimer's. So Sharad, why don't we start with you, maybe talk a little bit about what this initiative is all about, what the partnership is all about, what you guys are doing. >> So we started on a project called the Machine Project about three, three and a half years ago and frankly at that time, the response we got from a lot of my colleagues in the IT industry was "You guys are crazy", (Dave laughs) right. We said we are looking at an enormous amount of data coming at us, we are looking at real time requirements on larger and larger processing coming up in front of us, and there is no way that the current architectures of the computing environments we create today are going to keep up with this huge flood of data, and we have to rethink how we do computing, and the real question for those of us who are in research in Hewlett Packard Labs, was if we were to design a computer today, knowing what we do today, as opposed to what we knew 50 years ago, how would we design the computer? And this computer should not be something which solves problems for the past, this should be a computer which deals with problems in the future. So we are looking for something which would take us for the next 50 years, in terms of computing architectures and what we will do there. In the last three years we have gone from ideas and paper study, paper designs, and things which were made out of plastic, to a real working system. We have around Las Vegas time, we'd basically announced that we had the entire system working with actual applications running on it, 160 terabytes of memory all addressable from any processing core in 40 computing nodes around it. And the reason is, although we call it memory-driven computing, it's really thinking in terms of data-driven computing. The reason is that the data is now at the center of this computing architecture, as opposed to the processor, and any processor can return to any part of the data directly as if it was doing, addressing in local memory. This provides us with a degree of flexibility and freedom in compute that we never had before, and as a software person, I work in software, as a software person, when we started looking at this architecture, our answer was, well, we didn't know we could do this. Now if, given now that I can do this and I assume that I can do this, all of us in the programmers started thinking differently, writing code differently, and we suddenly had essentially a toy to play with, if you will, as programmers, where we said, you know, this algorithm I had written off decades ago because it didn't work, but now I have enough memory that if I were to think about this algorithm today, I would do it differently. And all of a sudden, a new set of algorithms, a new set of programming possibilities opened up. We worked with a number of applications, ranging from just Spark on this kind of an environment, to how do you do large scale simulations, Monte Carlo simulations. And people talk about improvements in performance from something in the order of, oh I can get you a 30% improvement. We are saying in the example applications we saw anywhere from five, 10, 15 times better to something which where we are looking at financial analysis, risk management problems, which we can do 10,000 times faster. >> So many orders of magnitude. >> Many, many orders >> When you don't have to wait for the horrible storage stack. (laughs) >> That's right, right. And these kinds of results gave us the hope that as we look forward, all of us in these new computing architectures that we are thinking through right now, will take us through this data mountain, data tsunami that we are all facing, in terms of bringing all of the data back and essentially doing real-time work on those. >> Matthias, maybe you could describe the work that you're doing at the University of Bonn, specifically as it relates to Alzheimer's and how this technology gives you possible hope to solve some problems. >> So at the University of Bonn, we work very closely with the German Center for Neurodegenerative Diseases, and in their mission they are facing all diseases like Alzheimer's, Parkinson's, Multiple Sclerosis, and so on. And in particular Alzheimer's is a really serious disease and for many diseases like cancer, for example, the mortality rates improve, but for Alzheimer's, there's no improvement in sight. So there's a large population that is affected by it. There is really not much we currently can do, so the DZNE is focusing on their research efforts together with the German government in this direction, and one thing about Alzheimer's is that if you show the first symptoms, the disease has already been present for at least a decade. So if you really want to identify sources or biomarkers that will point you in this direction, once you see the first symptoms, it's already too late. So at the DZNE they have started on a cohort study. In the area around Bonn, they are now collecting the data from 30,000 volunteers. They are planning to follow them for 30 years, and in this process we generate a lot of data, so of course we do the usual surveys to learn a bit about them, we learn about their environments. But we also do very more detailed analysis, so we take blood samples and we analyze the complete genome, and also we acquire imaging data from the brain, so we do an MRI at an extremely high resolution with some very advanced machines we have, and all this data is accumulated because we do not only have to do this once, but we try to do that repeatedly for every one of the participants in the study, so that we can later analyze the time series when in 10 years someone develops Alzheimer's we can go back through the data and see, maybe there's something interesting in there, maybe there was one biomarker that we are looking for so that we can predict the disease better in advance. And with this pile of data that we are collecting, basically we need something new to analyze this data, and to deal with this, and when we heard about the machine, we though immediately this is a system that we would need. >> Let me see if I can put this in a little bit of context. So Dave lives in Massachusetts, I used to live there, in Framingham, Massachusetts, >> Dave: I was actually born in Framingham. >> You were born in Framingham. And one of the more famous studies is the Framingham Heart Study, which tracked people over many years and discovered things about heart disease and relationship between smoking and cancer, and other really interesting problems. But they used a paper-based study with an interview base, so for each of those kind of people, they might have collected, you know, maybe a megabyte, maybe a megabyte and a half of data. You just described a couple of gigabytes of data per person, 30,000, multiple years. So we're talking about being able to find patterns in data about individuals that would number in the petabytes over a period of time. Very rich detail that's possible, but if you don't have something that can help you do it, you've just collected a bunch of data that's just sitting there. So is that basically what you're trying to do with the machine is the ability to capture all this data, to then do something with it, so you can generate those important inferences. >> Exactly, so with all these large amounts of data we do not only compare the data sets for a single person, but once we find something interesting, we have also to compare the whole population that we have captured with each other. So there's really a lot of things we have to parse and compare. >> This brings together the idea that it's not just the volume of data. I also have to do analytics and cross all of that data together, right, so every time a scientist, one of the people who is doing biology studies or informatic studies asks a question, and they say, I have a hypothesis which this might be a reason for this particular evolution of the disease or occurrence of the disease, they then want to go through all of that data, and analyze it as as they are asking the question. Now if the amount of compute it takes to actually answer their questions takes me three days, I have lost my train of thought. But if I can get that answer in real time, then I get into this flow where I'm asking a question, seeing the answer, making a different hypothesis, seeing a different answer, and this is what my colleagues here were looking for. >> But if I think about, again, going back to the Framingham Heart Study, you know, I might do a query on a couple of related questions, and use a small amount of data. The technology to do that's been around, but when we start looking for patterns across brain scans with time series, we're not talking about a small problem, we're talking about an enormous sum of data that can be looked at in a lot of different ways. I got one other question for you related to this, because I gotta presume that there's the quid pro quo for getting people into the study, is that, you know, 30,000 people, is that you'll be able to help them and provide prescriptive advice about how to improve their health as you discover more about what's going on, have I got that right? >> So, we're trying to do that, but also there are limits to this, of course. >> Of course. >> For us it's basically collecting the data and people are really willing to donate everything they can from their health data to allow these large studies. >> To help future generations. >> So that's not necessarily quid pro quo. >> Okay, there isn't, okay. But still, the knowledge is enough for them. >> Yeah, their incentive is they're gonna help people who have this disease down the road. >> I mean if it is not me, if it helps society in general, people are willing to do a lot. >> Yeah of course. >> Oh sure. >> Now the machine is not a product yet that's shipping, right, so how do you get access to it, or is this sort of futures, or... >> When we started talking to one another about this, we actually did not have the prototype with us. But remember that when we started down this journey for the machine three years ago, we know back then that we would have hardware somewhere in the future, but as part of my responsibility, I had to deal with the fact that software has to be ready for this hardware. It does me no good to build hardware when there is no software to run on it. So we have actually been working at the software stack, how to think about applications on that software stack, using emulation and simulation environments, where we have some simulators with essentially instruction level simulator for what the machine does, or what that prototype would have done, and we were running code on top of those simulators. We also had performance simulators, where we'd say, if we write the application this way, this is how much we think we would gain in terms of performance, and all of those applications on all of that code we were writing was actually on our large memory machines, Superdome X to be precise. So by the time we started talking to them, we had these emulation environments available, we had experience using these emulation environments on our Superdome X platform. So when they came to us and started working with us, we took their software that they brought to us, and started working within those emulation environments to see how fast we could make those problems, even within those emulation environments. So that's how we started down this track, and most of the results we have shown in the study are all measured results that we are quoting inside this forum on the Superdome X platform. So even in that emulated environment, which is emulating the machine now, on course in the emulation Superdome X, for example, I can only hold 24 terabytes of data in memory. I say only 24 terabytes >> Only! because I'm looking at much larger systems, but an enormously large number of workloads fit very comfortably inside the 24 terabytes. And for those particular workloads, the programming techniques we are developing work at that scale, right, they won't scale beyond the 24 terabytes, but they'll certainly work at that scale. So between us we then started looking for problems, and I'll let Matthias comment on the problems that they brought to us, and then we can talk about how we actually solved those problems. >> So we work a lot with genomics data, and usually what we do is we have a pipeline so we connect multiple tools, and we thought, okay, this architecture sounds really interesting to us, but if we want to get started with this, we should pose them a challenge. So if they can convince us, we went through the literature, we took a tool that was advertised as the new optimal solution. So prior work was taking up to six days for processing, they were able to cut it to 22 minutes, and we thought, okay, this is a perfect challenge for our collaboration, and we went ahead and we took this tool, we put it on the Superdome X that was already running and stepped five minutes instead of just 22, and then we started modifying the code and in the end we were able to shrink the time down to just 30 seconds, so that's two magnitudes faster. >> We took something which was... They were able to run in 22 minutes, and that was already had been optimized by people in the field to say "I want this answer fast", and then when we moved it to our Superdome X platform, the platform is extremely capable. Hardware-wise it compares really well to other platforms which are out there. That time came down to five minutes, but that was just the beginning. And then as we modified the software based on the emulation results we were seeing underneath, we brought that time down to 13 seconds, which is a hundred times faster. We started this work with them in December of last year. It takes time to set up all of this environment, so the serious coding was starting in around March. By June we had 9X improvement, which is already a factor of 10, and since June up to now, we have gotten another factor of 10 on that application. So I'm now at a 100X faster than what the application was able to do before. >> Dave: Two orders of magnitude in a year? >> Sharad: In a year. >> Okay, we're out of time, but where do you see this going? What is the ultimate outcome that you're hoping for? >> For us, we're really aiming to analyze our data in real time. Oftentimes when we have biological questions that we address, we analyze our data set, and then in a discussion a new question comes up, and we have to say, "Sorry, we have to process the data, "come back in a week", and our idea is to be able to generate these answers instantaneously from our data. >> And those answers will lead to what? Just better care for individuals with Alzheimer's, or potentially, as you said, making Alzheimer's a memory. >> So the idea is to identify Alzheimer long before the first symptoms are shown, because then you can start an effective treatment and you can have the biggest impact. Once the first symptoms are present, it's not getting any better. >> Well thank you for your great work, gentlemen, and best of luck on behalf of society, >> Thank you very much >> really appreciate you coming on theCUBE and sharing your story. You're welcome. All right, keep it right there, buddy. Peter and I will be back with our next guest right after this short break. This is theCUBE, you're watching live from Madrid, HPE Discover 2017. We'll be right back.

>> Announcer: Live from Orlando, Florida, it's theCUBE, covering PentahoWorld 2017. Brought to you by Hitachi Vantara. >> Welcome back to theCUBE's live coverage of PentahoWorld, brought to you, of course, by Hitachi Vantara. I'm your host, Rebecca Knight, along with my co-host, James Kobielus. We have two guests today, we have Michael Becker, a senior chief inspector, and Henry Liebrenz, the police sergeant of the German Federal Police, the Bundespolizei. Welcome, gentlemen. Thanks so much for joining us. So do you want to start out by telling us, telling our viewers a little bit about Bundespolizei and what you do there? >> Okay. The Federal German Police employs about 41,000 people, and as part of Federal German Ministry of Interior, we have, the police is responsible for many demanding and varied tasks, like air control or air safety, rail patrol, water control, crime reduction, and patrol the high seas. And besides an internal task, we do many international missions, police missions all over the world and missions in the European Union for neighboring. And our job, our main job is to development specialty police software. You couldn't buy (foreign words) products, and the development was our own framework based on lamp. >> Classical open source systems plus open source databases plus PHP, it's script language, on the top of it's end. And we built our own absolute framework on this, it's exclusively for us and that's our main job, to build applications on this top. >> And besides our name, our main job we are responsible for the data warehouse and responsible for integration, data integration technologies of the Federal Police. >> So you're both within the IT organization of Bundespolizei, okay. >> Yes, we stay in the IT department that belongs to headquarter. In Germany, or in German police, we have one headquarter, we have 11 district offices, about 80 regional offices, and about 160 local offices. >> All over Germany, is it. >> So when you're thinking about your software challenges, you have a lot of different obstacles: safety, operational, security. What are some of the things that you're taking into account when you're implementing software? >> Um, what we take in account? Not so easy to (speaking in foreign language). >> What is your approach? What are the things on your mind that is keeping you up at night? >> We have two different ways. The main way is to build software. And we have in special case. In turn case we build software that bring is on the point for this case. The other way is we have a way to product data in this cases. That's the other way. What can we do with this data? That's the other case around Pentaho. We want to have more benefit with this kind of data. >> What sort of data driven application development do you do or do you oversee for Bundespolizei? Can you describe some of the applications within their specific functions? >> We have one main application is our time planning tool. So all the shifts on the agencies it's possible to plan. In one case that we build on this platform and it's exclusively for us. We have the situation that other polices in Germany ask us about. Hey, that's very a good solution. Maybe we can take it also for us. But because it's a little bit different for normal situations outside and in other companies. Because we have the situation 24 hours, seven days a week, 365 days a year to bring our services. We have a big many rules about this kind of working. The offices get some more money in the night or it's Saturday and something like this it's not so easy to implement with normal software. So we were at the case what we do. Then okay we do it ourself and that's exactly on point. >> You describe the rules, you're describing the rules that are provided from the European Union or from your government in terms of security, privacy, and so forth. Is that what you're describing? How have this whole total set of rules and policies and mandates shaped your data management strategy within your organization? How does the Pentaho set of solutions support those requirements? >> I think with Pentaho I told it yesterday also it was for us definitely the game changer. It's definitely true. Before we don't have the chance to build something like this only was two us. But now we have the big Swiss knife. We get entrance with especially with the Ketel, solution, PDI. >> With Ketel everything is possible. >> It's not possible to build your own. >> That was the entrance to build a strategy about it. Then at this point we had the solution to let the data flow wherever you want. Then we start okay, when can we have data every time at every point. So what can we do with it? What is the benefit for us? We start to come in discussion with our other departments inside what is your problem? What can we do to help you to get more benefit about it? >> How much sharing goes on between departments? >> Henry: The sharing? >> Yes, in terms of as you said, how can I help you? Oh, we are doing something over here. >> I think it's a classic job like other. (speaking in foreign language) We do it inside so we go to the other departments and have this part of discussion. We try to bring it in the right way. >> What degree of this sharing is intergovernmental? Meaning you are reaching out to your peer agencies within the European Union maybe through Interpol to other nations? Is any of that going on and is Pentaho playing a role in terms of helping you in that regard? (speaking in foreign language) >> How we have to say? >> If you don't want to say or can't say. >> Actually I think in German or in European it's not so big. I don't know why, I can't believe it. But it's also to take advantage at Pentaho that you can start at any time. You can start as a community. We work also before, two years with the (voice is muffled). And started this year with enterprise and we have only one day for integration from the community server of the new enterprise server. No problems. I think that is a great benefit. You can almost start with a small problem or data integration. >> In the past the other big companies maybe they had a little bit earlier start. Pentaho, the goal to come along the other players. I think in Europe, especially in Germany at the moment can be good. >> In Germany we have a situation over Pentaho user meeting or Pentaho community meetings but also other agencies come and ask why Pentaho and how did you do it? >> Is there an ongoing program of working with other federal agencies in Germany to share the best practices you've learned from using data at least to manage your agency's requirements? What could they learn from what you've done? >> The progress is starting now so the other come to us. We meet together and they want to take a look directly on our screens and want to see some cases. We play for them live and it's a very interesting situation. When they see eh, you have the same problems as I. It's interesting. >> And very important is also that we learned and we have learned from Pentaho that everything is possible. You need much less time for everything or for every kind of problem. We are very fast. Before we used to have another (foreign word), it's called Excel. It's crazy, it's good for statistics but we have no data quality. >> It's not possible to work with big data. (voice is muffled) >> Our data are actual, daily actual. Before we wait for one month or two months. >> Before we had exactly one day per month. At this day the data was correct only one day. And other other days we had to collect the data for the next month. >> It's a whole new world with Pentaho. Henry and Michael, thank you so much for coming on theCube. It was great having you on here. >> Thank you very much. >> We will have more from theCube's live coverage of PentahoWorld just after this. (upbeat digital music)

>>thank you so much that sake for inviting me to the Entity Research Summit. And I'm really excited to talk to all of them today. So I will be talking about achieving indistinguishability obfuscation from well founded assumptions. And this is really the result of a wonderful two year collaboration with But now it's standing. Graduate student I use chain will be graduating soon on my outstanding co author, Rachel Lynde from the University of Washington. So let me jump right into it. We all know that constructing indistinguishable the obfuscation. Constructing Io has been perhaps the most consequential open problem in the foundations of photography. For several years now, they've seen over 100 papers written that show how to use Iot to achieve a number of remarkable cryptographic goals. Um, that really expand the scope of cryptography in addition to doing just remarkable, really interesting new things. Unfortunately, however, until this work, I told the work I'm about to tell you about all known constructions of Iove. All required new hardness, assumptions, heart assumptions that were designed specifically to prove that Iowa secure. And unfortunately, uh, this has a torture of history. And many of the assumptions were actually broken, which led to just a lot of doubt and uncertainty about the status of Iot, whether it really exists or doesn't exist. And the work I'm about to tell you about today changes that state of affairs in the continental way in that we show how to build io from the combination of four well established topographic assumptions. Okay, let me jump right into it and tell you how we do it. So before this work that I'm about to tell you about over the last two years with Rachel and Ayush, we actually constructed a whole sequence of works that have looked at this question. And what we showed was that if we could just build a certain special object, then that would be sufficient for constructing Io, assuming well established assumptions like L W E P R g s and M C zero and the 68 assumption of a violin. Your mouths. Okay, So what is this object? The object first starts with a P. R G and >>S zero. In other words, of trg with constant locality that stretches end bits of seed to M bits of output where am is ended one plus Epsilon for any constant Epsilon zero. Yes, but in addition to this prg, we also have these l w we like samples. So as usual, we have an elder Bluey Secret s which is random vector z b two k, where K is the dimension of the secret, which is much smaller than any way also have this public about vectors ai which are also going to be okay. And now what is given out is are the elderly samples where the error is this X I that is just brilliant value. Uh, where these excise air Also the input to our prg. Okay, unfortunately, we needed to assume that these two things together, this y and Z together is actually pseudo random. But if you think about it, there is some sort of kind of strange assumption that assumes some kind of special leakage resilience, property of elderly, we where elderly samples, even with this sort of bizarre leakage on the errors from all debris, is still surround or still have some surrounding properties. And unfortunately, we had no idea how to prove that. And we still don't have any idea how to prove this. Actually, So this is just a assumption and we didn't know it's a new assumption. So far, it hasn't been broken, but that's pretty much it. That's all we knew about it. Um and that was it. If we could. If this is true, then we could actually build. I'll now to actually use this object. We needed additional property. We needed a special property that the output of this prg here can actually be computed. Every single bit of the output could be computed by a polynomial over the public. Elder Louise samples Why? And an additional secret w with the property that this additional secret w is actually quite small. It's only excise em to the one minus delta or some constant delta gradients. Barroso polynomial smaller from the output of the prg. And crucially, the degree of this polynomial is on Lee to its violin e er can this secret double that's where the bottle in your mouth will come. Okay. And in fact, this part we did not approve. So in this previous work, using various clever transformations, we were able to show that in fact we are able to construct this in a way to this Parliament has existed only degree to be short secret values. Double mhm. So now I'm gonna show you how using our new ideas were actually gonna build. That's a special object just like this from standard assumptions. We're just gonna be sufficient for building io, and we're gonna have to modify it a little bit. Okay? One of the things that makes me so excited is that actually, our ideas are extremely simple. I want to try to get that across today. Thanks. So the first idea is let's take thes elder movie samples that we have here and change them up a little bit when it changed them up. Start before I get to that in this talk, I want you to think of K the dimension of the secret here as something very small. Something like end of the excellent. That's only for the stock, not for the previous work. Okay. All right. So we have these elderly samples right from the previous work, but I'm going to change it up instead of computing them this way, as shown in the biggest slide on this line. Let's add some sparse hair. So let's replace this error x i with the air e i plus x I where e is very sparse. Almost all of these IIs or zero. But when the I is not zero is just completely random in all of Z, pizza just completely destroys all information. Okay, so first I just want to point out that the previous work that I already mentioned applies also to this case. So if we only want to compute P R g of X plus E, then that can still be computer the polynomial. That's degree to in a short W that's previous work the jail on Guess work from 2019. I'm not going to recall that you don't have time to tell you how you do it. It's very simple. Okay, so why are we doing this? Why are we adding the sparse error? The key observation is that even though I have changed the input of the PRG to the X Plus E because he is so sparse, prg of explosive is actually the same as P. R. G of X. In almost every outlet location. It's only a tiny, tiny fraction of the outputs that are actually corrupted by the sparse Arab. Okay, so for a moment Let's just pretend that in fact, we knew how to compute PRGF X with a degree to polynomial over a short seeking. We'll come back to this, I promise. But suppose for a moment we actually knew how to compute care to your ex, Not just scared of explosive in that case were essentially already done. And the reason is there's the L. P n over zp assumption that has been around for many years, which says that if you look at these sort of elderly like samples ai from the A, I s but plus a sparse air e I where you guys most zero open when it's not serious, completely random then In fact, these samples look pseudo random. They're indistinguishable from a I r r. I just completely uniform over ZP, okay? And this is a long history which I won't go because I don't have time, but it's just really nice or something. Okay, so let's see how we can use it. So again, suppose for the moment that we were able to compute, not just appeared you've explosive but appeared to you that well, the first operation that since we're adding the sparse R E I This part the the L P N part here is actually completely random by the LP an assumption so by L P and G. P, we can actually replace this entire term with just all right. And now, no, there is no more information about X present in the samples, The only place where as is being used in the input to the prg and as a result, we could just apply to sit around this of the prg and say this whole thing is pseudo random and that's it. We've now proven that this object that I wanted to construct it is actually surrounded, which is the main thing that was so bothering us and all this previous work. Now we get it like that just for the snap of our fingers just immediately from people. Okay, so the only thing that's missing that I haven't told you yet is Wait, how do we actually compute prg attacks? Right? Because we can compute p r g of X plus e. But there's still gonna be a few outputs. They're gonna be wrong. So how can we correct those few corrupted output positions to recover PRGF s? So, for the purpose of this talks because I don't have enough time. I'm gonna make sort of a crazy simplifying assumption. Let's just assume that in fact, Onley one out the position of P r g of X plus e was correct. So it's almost exactly what PR gox. There's only one position in prg of Ecstasy which needs to be corrected to get us back to PR gox. Okay, so how can we do that? The idea is again really, really simple. Okay, so the output of the PRG is an M. Becker and so Dimension and Becker. But let's actually just rearrange that into a spirit of them by spirit of them matrix. And as I mentioned, there's only one position in this matrix that actually needs to be corrected. So let's make this correction matrix, which is almost everywhere. Zero just in position. I j it contains a single correction factor. Why, right? And if you can add this matrix to prg of explosive, then we'll get PR dribbles. Okay, so now the Onley thing I need to do is to compute this extremely sparse matrix. And here the observation was almost trivia. Just I could take a spirit of em by one maker That just has why in position I and I could take a one by spirit of them matrix. I just have one in position J zero everywhere else. If I just take the tensor product was music the matrix product of these two of these two off this column vector in a row vector. Then I will get exactly this correction matrix. Right? And note that these two vectors that's called them you and be actually really, really swamped their only spirit of n dimensional way smaller than them. Right? So if I want to correct PRGF Expo see, all I have to do is add you, Tenzer V and I can add the individual vectors u and V to my short secret w it's still short. That's not gonna make W's any sufficiently bigger. And you chancery is only a degree to computation. So in this way, using a degree to computation, we can quickly, uh, correct our our computation to recover prg events. And now, of course, this was oversimplifying situation, uh, in general gonna have many more areas. We're not just gonna have one error, like as I mentioned, but it turns out that that is also easy to deal with, essentially the same way. It's again, just a very simple additional idea. Very, very briefly. The idea is that instead of just having one giant square to them by sort of a matrix, you can split up this matrix with lots of little sub matrices and with suitable concentration bound simple balls and pins arguments we can show that we could never Leslie this idea this you Tenzer v idea to correct all of the remaining yet. Okay, that's it. Just, you see, he's like, three simple >>ah ha moments. What kind of all that it took, um, that allowed >>us to achieve this result to get idol from standard assumptions. And, um, of course I'm presenting to you them to you in this very simple way. We just these three little ideas of which I told you to. Um, but of course, there were only made possible because of years of struggling with >>all the way that didn't work, that all that struggling and mapping out all the ways didn't work >>was what allowed us toe have these ideas. Um, and again, it yields the first I'll construction from well established cryptographic assumptions, namely Theo Elgon, assumption over zp learning with errors, assumption, existence of PR GS and then zero that is PR juice with constant death circuits and the SX th assumption over by linear notes, all of which have been used many years for a number of other applications, including such things as publicly inversion, something simple public inversion that's the That's the context in which the assumptions have been used so very far from the previous state of affairs where we had assumptions that were introduced on Lee Professor constructing my own. And with that I will conclude, uh and, uh, thank you for your attention. Thanks so much.

>> live from Boston, Massachusetts. It's the you covering your red hat. Some twenty nineteen >> rots. >> You buy bread >> hat, >> and we'LL go back here on the Cube as we continue our coverage here. Red Hat Summit day. One of three days of Walter Wall coverage coming to you exclusively here on the Q. I'm John Walls was too Millman. Thank you for joining us. And we're now joined by a couple of gentlemen. Guess the dynamic duo of the container World it at Red Hat. Scott McCarty is the principal product manager of Containers. That open shift and Forell. Scott. Good to see you, sir. >> You could see it >> and been. Bree are Who's the principal product? Manager of Containers and Koro s, Of course. Also it Red hat Been. Thank you for joining us. First off, just your thought about show. Obviously, there's a lot of educational programming going on up down, big crowds, a lot of buzz. Good activity day one, at least from our perspective. How are you guys seeing this so far? >> I love it. I mean, it's been great so far. We just had us. I just had a session, just got out of it. was completely full of people trying to get in that were lined up against a wall. So it's been very exciting so far. >> Yeah. Ben. So it's one of >> my favorite times of the year, right? It's so much energy. Everybody comes with the exchange of ideas, just feedback and everything is one of my favorites. >> Oh, good. Right now s o l e made available publicly today for the first time. We talked about that a lot so far on the program, I'd like to hear from >> your side of the fence. Then what does that mean to you in terms of the container world and the impact that you, you know, from here going forward, you've got a whole new world of concern, I would think Scott. >> Yeah. I mean, with the relic, it's it's >> exciting because we're releasing, uh, you know, a lot of new tools around containers, >> a ton of new operational, you know, management capabilities. I mean, it's just it's an exciting release, Ben. It's a It's a big step forward, right? Every single release is a big deal, and we look at the container space. It's evolved a lot in the past for five years right when we came out. Seven. So technology's matured, Really, it's Ah, it's a smooth, easy experience to get to the release. And if lots going into it a lot, >> Yeah, so, Scott, It's funny. I think back. Turn back. Five years ago, we had a lot of jokes about doctors. You mean the pants? Because container ization and, you know, limits, containers and everything. That was something most people hadn't heard about here. Twenty nineteen, You said, There's, you know, crowds trying to get in the door. And it's not what but there really digging in and understand the tools we give a little bit of. You know what? What's what's with the excitement these days? Where are the customers? And you know what? What do you digging into >> with them? Yeah, well ah, >> funny example. So I asked I asked this last session, You know, raise your hand if you've used containers. If you just even fired up a container before and everyone raise your hand. And now, five years ago, that was, like one person >> and then even last you worked for Google. Yeah. Even last >> year that it was still maybe forty percent of the people, and now it's one hundred percent when they come to a session. So I mean, it's it is it is definitely changed, a tremendous amount. And now it's about So I joked, You know, five years ago is about using a chef knife, you know, just like you cut everything with it, right? You cut it. Vegetables, meat, whatever. And there was like one thing, and you just figured out Doctor and Cooper names was even on the radar Yet now it's about refining all the tools and getting to a place where, like, it's really getting excited, cause now we have special paring knives and chef knife and, you know, hibachi, knife and all these different, more specialized >> tools. So it's getting saying >> You think it's easy to >> adopt now to write, because years ago everyone was hedging their bets on you know what orchestration am I going to use? What piece? Um, I'm gonna build my stack. We have >> now. It's much, much clear, well defined. You know, Cooper Netease is dominant factor, right? Mean, open shift is huge, huge growth for us in that space. So I mean, it's it's it's a lot easier for customers to get in that game now than it was, you know? Yeah, just a couple years ago. Yeah, just a couple years ago. All right, so let's let's sticking out security a little bit because that was one of the big question marks in the early days. And you know something? We talk about it all the shows. It's it's definitely a focus of the real late launch. So where were the container world today and anything new or nuance that the audience should understand? I think on the security side you've got I have three or four big points there. One is the container tools of worshipping. Today they basically inherit the full Lennox security model. Right? So no longer do you have ah, privilege socket. That is, I kind of that weak factor, if you will, that's gone on. Really? So that's a big That's a big win right there. Beyond that, we've got a new crystal policies. You can set a central policy for the O. S. And that works in the containers well, so of you and enforce a particular kind of floor, if you will, of crypto. You could do that with relate for the host way and images as well. That's a that's a big part of it. And then we also have new tools that you can build smaller containers because how did the security is what is in my container? So if you're putting less less packages and content in that image, that's a much smaller Becker as well. Soon. >> Yeah. So, um, from from a security perspective, too, you know, you know the fact that now we have, um, kind of we've got a set of tools now that we can do experiments with things like ruthless, for example. You know, we're tech preview release of ruthless contract, so historically have always ran them, you know, as route. That's just how it works. I mean, we kind of figured it out one way and did it, and it was cool. And then at a certain point, we went all right, we need these other use cases where want developers to build to do it. For example, I just talked to a customer that it has four two hundred. I'm sorry, developers that are all running instances on their laptops PM's with pod man and build a running and, you know, using these tools to actually build containers, and they want to do ruthless bad. They want to do it in all their essentially all their environment, so that people are really hungry for a lot of these security features that we're working on now and relate. And it's something that we're releasing even as a vato. >> How did the capabilities changed in terms of relate now and what you have to provide the support? So what's transformed? And then what will be the need in order to build on that toe work on that and to make it more secure stables on so >> far? Well, I think I think you kind of have to dig into, like, a selection of what tools we decided to go in. Relate you'LL see that it's pod man. Build a scope. Here are the three main lower level tools that we have, and those tools are built serving a Unix mindset where it's like you can pipe things together and do things and use them collaboratively together to go remotely inspect images, pull them, build them from scratch, you know, run them locally, not as roots run them as a non route, contains things like that way or not at, you know, we're not releasing doctrine. Relate. And so so the transition. There is probably the biggest transition for users. Kind of realizing. Okay, we're going kind of broken this apart into three little or tools that we can then use Todd Man being the main one you go to. And then and then it's got a command line that's very similar. And so it's very easy, tio kind of transition over. But then you start to again kind of my my chef knife reference. You realize once you transition from, say, Dr Pod man, you kind of that's your chef knife. You kind of know what? How to start doing things that way. But then you start to get more refined and start to dig deeper into, you know, like, you know, into building scope. You essentially teacher. Yeah. >> You're good there. Yeah. I don't know. All right. Whatever he says. Scott >> Universal base image. Something we've talked a little bit about to tell us how that this is going to impact, you know, talk about everybody building things on their laptop. Seems like that's an extension of where this fits. Help help us understand? >> Yeah, I can't hide my enthusiasm. One how excited I am by Eva, and I will admit Ivory had a couple people come to me and say, This is the most exciting thing for me at Summit period And I think that's interesting because it's not actually something new and that, you would say from a technology perspective, how exciting is that? I don't know, but like it allows a set of collaboration that we've never been able to like, really, really do with a well base image historically, and I think the real base image is the highest quality basement temperament out there. But the problem is, even if you had something really simple, like so you had one university and that created some kind of science experiment in a container, and then they want to push that out to a public registry, then pull it down a different university and share it. They couldn't do that under the terms of the rail base image. So that was that. Was that create a little bit of friction with the FBI? Now that's completely gone. You can now run it anywhere you want, distribute anywhere you want, just the distribution alone is exciting. It and the fact that when you >> run it on rail, you >> build on rail, run on relics completely supported Israel. But you can now push it out to a public registry and let it sit out there and other people can >> use it in an experiment. So is the, you know, coming together of container ization in that distribution is that would kind of is really new with this, as opposed to the ways that I used to be able to share lennox images in the past. >> Well, all I think I think the challenge was you'd have some people that would want to do something. They want to build a distributed anywhere they want have that freedom. But they still wanted the quality of the rail basement. Now that created friction, right? So then they'd have to make an unnatural choice between, like, a fedora or I use, you know, well, maybe how you sent to less and your lying and none of those have all the things that I want, right? It was like a card game trying to get all the components that you want. You want sport, ability of Raoul. You want the security of the performance center center. But you couldn't. You couldn't distribute anywhere, so that created friction where you make on natural choices on basement. Now you be. I just The name implies that universal use it for anything you want. >> Same for communities to write because they don't want to make one that could freely distribute and then another like supported variant. They have more to maintain its more cycles and everything so simple. Find that it is a big deal. Yeah, >> and migration between base images is a linen migration, so it's frustrating to do. You don't want to do it. You want to build on one thing. And then I thought I distribute that thing anywhere. Well, then it's >> interesting, you know, go back a few years. There was this big movement to do, like just enough OS. How do I slim down the core? Os was I don't need everything that you know Realm necessarily does. So have we gotten over that? And we now gotten with you know, the things like you be I down to like a nice unit that's easily terrible and distributed. It's a good question. It's a topic that we'LL never go away. I don't think we're still. It's just changing its form, right? It still exists on the host. It's still exists in images. It's still exist with unit colonels and everything. I >> think where we >> are today. That was a really good spot, right? We've got several footprints of FBI. If there's several footprints of Rehl, including well, Core OS, which is like bedded version of rail into open shift right for a small form factor container host. So where we are today is very strong, but it's going to continue to evolve and get better. So, yeah, >> and we I mean, we look at the future and we're we're looking at ways toe. Make it even smaller, you know, you're always looking at, but yeah, Ben, mention there's three footprints of you B i today. There's a minimal image. There's a standard image, and then there's even a little bit bigger images allows you run multiple services, but you know that's the selection today. But in the future, we're looking at making the minimal one more minimal. Were even looking at, you know, making the standard one more minimal. >> Yeah, we're not done. Yeah, we're not done. You're never done. I guess the last thing I have on this, you know, multi cloud is such you know where customers are today. You know, you're gonna have the CEO Microsoft up on stage today. Two years ago, when I was here, it was the partnership between Red had an eight of us was all the discussion. I spoke to the Red Hat team, the Cloud show recently. So how does the tooling that you have fit in tow all the clouds discussion that I have when I talked to users? You know, one of the biggest lock ins they have is the skill set and the understanding of different tools and knowledge. And so you know, where we standardize and where do we still have work to do in this space? That's a big question. So yeah, I guess way addressing a multiple levels right at the core. The center Israel. Right. So well ate right now today on all those cloud platforms that you just name, right. So same say maybe I level guarantee that ten years hard work everything. It's it's everywhere. It's pervasive today. Level up, right. You've got the container images and stuff same story. They're Goa level. You've got open shift that is pervasive everywhere. And now we're doing really cool things. And Cooper Net. He's like a machine, a p I and all these other things toe actually control those individual cloud infrastructures which abstracts all of the customers ations per for food for him, which is >> powerful. So I think, for me was the most exciting things is the open shift for paradigm shift that shift from managing individual nodes to ship to managing the cluster as a computer, which we've said for what, twenty years? The sun? I think you know the cluster is the computer, you know? But we're really there today. Like we have a single E p I. Ben mention the machine, the machine, a PR machine configure operator. There's there's essentially automation built into the chip platform now that allows you to appoint the same on any cloud. So eight of us azure, you know, open stack, even on VM, where even on, you know, even in liver gonna look a laptop. There's a way to deploy it in the identical, you know, in an identical configuration. To me, that's exciting, because now I have one set of things I could learn. And then again in the standard red hat way. If you feel locked in, you can go use a Okay, Daddy, you can use the upstream. So you're never locked into our product, Which that's something. Get a lot with Kat drives, right? Like if you're locked in there, you're you're locked in there. There's no there's no, you know, open source version of that to get out of that. >> So you've talked about growth opportunities? You said, No, we're not done yet. Making the joke about your own work. You've talked about a twenty year evolution, you know, Just refer to that. And if you could look, you know, whether it's three, four, five, whatever years down the road, where's the big leap? Where's that have to come? Where do you think it's going to come in terms of the capabilities that you want to work on and what you want to be able to deliver from where you are right? Now >> get my crystal ball. Yeah. >> Yeah, Well, I think you've got one. Yeah. Then I have a lot of confidence in you, but if you had to say okay, this is this is atleast where we're gonna be. We're gonna have to spend a lot of our time because this this is the area that we think I think needs most attention. A >> couple of things, right? People only scale so much. So automation is an area that's bulletproof going forward, and it's going to evolve and take many forms. Right now, our big push has been on the operator space and obviously technologies like answerable that's going to continue to evolve and make make people scale better. That's probably one of the biggest ones. And I >> think that's one of the biggest ones. I think I think for me, probably where my mind wanders, is around partners and building that ecosystem in the open ship space similar to what you see in the realm. Because system today I think three, four years from now you're going to see it really exploded at ABC that I already see it exploding. But by then you'LL see it maturing and you'LL really see. I think if you look at the operator paradigm, I'm very excited by that because it's kind of like the Emma science dollar that Microsoft invented. You know that kind of made that that ubiquitous that install experience. Except that operators make it you because they install and managed a too. So I think, like, kind of to his point of, like making that the install really simple and then the operation of it. Over time, I think you're going to see a lot of I think. I think you couldn't fill a room and ask him, Like what I in fact, I did. I asked what an operator was, you know, and they they weren't super aware of it yet. But I think in the next five years, that will become the big with this way of just installing software. >> All right, well, we're going to check back in five. We'LL see how it turns out and been by then. Bring that crystal ball back with wood. Ok, I'll do a good deal. Thanks, gentlemen. Thanks for the time you haven't put on the Cuba as we continue our coverage here. Red Hat Summit. We're in Boston back with more right after this

Yeah, I'm Aala, They're the co-founder back to back. This is the cube silicon angle.com, Silicon angle dot TV's production of the cube, our flagship telecasts. We go out to the event. That was a great conversation. I was really just, just cool. I could have, we could have probably hit on a few more things, obviously well read. Awesome. Co-founder of Cloudera a. You were, you did a good job teaming up with that co-founder, huh? Not bad on the cube, huh? He's not bad on the cube, isn't he? He, >>He reads the internet. >>That's what I'm saying. >>Anything is going on. >>He's a cube star, you know, And >>Technology. Jeff knows it. Yeah. >>We, we tell you, I'm smarter just by being in Cloudera all those years. And I actually was following what he was saying, Sad and didn't dust my brain. So, Okay, so you're back. So we were talking earlier with Michaels and about the relational database thing. So I kind of pick that up where we left off with you around, you know, he was really excited. It's like, you know, hey, we saw that relational database movement happen. He was part of that. Yeah, yeah. That generation. And then, but things were happening or kind of happening the same way in a similar way, still early. So I was trying to really peg with him, how early are we, like, so, you know, as the curve, you know, this is 1400, it's not the Javit Center yet. Maybe the Duke world, you know, next year might be at the Javit Center, 35,000 just don't go to Vegas. So I'm trying to figure out where we are on that curve. Yeah. And we on the upwards slope, you know, down here, not even hitting that, >>I think, I think, I think we're moving up quicker than previous waves. And actually if you, if you look for example, Oracle, I think it took them 15, 20 years until they, they really became a mature company, VM VMware, which started about, what, 12, 13 years ago. It took them about maybe eight years to, to be a big company, met your company, and I'm hoping we're gonna do it in five. So a couple more years. >>Highly accelerated. >>Yes. But yeah, we see, I mean, I'm, I'm, I've been surprised by the growth. I have been, Right? I've been told, warned about enterprise software and, and that it takes long for production to take place. >>But the consumerization trend is really changing that. I mean, it seems to be that, yeah, the enterprises always last. Why the shorter >>Cycle? I think the shorter cycle is coming from having the, the, the, the right solution for the right problem at the right time. I think that's a big part of it. So luck definitely is a big part of this. Now, in terms of why this is changing compared to a couple of dec decades ago, why the adoption is changing compared to a couple of decades ago. I, I think that's coming just because of how quickly the technology itself, the underlying hardware is evolving. So right now, the fact that you can buy a single server and it has eight cores to 16 cores has 12 hards to terabytes. Each is, is something that's just pushing the, the, the, the limits what you can do with the existing systems and hence making it more likely for new systems to disrupt them. >>Yeah. We can talk about a lot. It's very easy for people to actually start a, a big data >>Project. >>Yes. For >>Example. Yes. And the hardest part is, okay, what, what do I really, what problem do I need to solve? How am I gonna, how am I gonna monetize it? Right? Those are the hard parts. It's not the, not the underlying >>Technology. Yes, Yes, that's true. That's true. I mean, >>You're saying, eh, you're saying >>Because, because I'm seeing both so much. I'm, I'm seeing both. I'm seeing both. And like, I'm seeing cases where you're right. There's some companies that was like, Oh, this Hadoop thing is so cool. What problem can I solve with it? And I see other companies, like, I have this huge problem and, and, and they don't know that HA exists. It's so, And once they know, they just jump on it right away. It's like, we know when you have a headache and you're searching for the medicine in Espin. Wow. It >>Works. I was talking to Jeff Hiba before he came on stage and, and I didn't even get to it cuz we were so on a nice riff there. Right. Bunch of like a musicians playing the guitar together. But like he, we talked about the it and and dynamics and he said something that I thoughts right. On money and SAP is talking the same thing and said they're going to the lines of business. Yes. Because it is the gatekeeper that's, it's like selling mini computers to a mainframe selling client servers from a mini computer team. Yeah. >>There's not, we're seeing, we're seeing both as well. So more likely the, the former one meaning, meaning that yes, line of business and departments, they adopt the technology and then it comes in and they see there's already these five different departments having it and they think, okay, now we need to formalize this across the organization. >>So what happens then? What are you seeing out there? Like when that happens, that mean people get their hands on, Hey, we got a problem to solve. Yeah. Is that what it comes down to? Well, Hadoop exist. Go get Hadoop. Oh yeah. They plop it in there and I what does it do? They, >>So they pop it into their, in their own installation or on the, on the cloud and they show that this actually is working and solving the problem for them. Yeah. And when that happens, it's a very, it's a very easy adoption from there on because they just go tell it, We need this right now because it's solving this problem and it's gonna make, make us much >>More money moving it right in. Yes. No problems. >>Is is that another reason why the cycle's compressed? I mean, you know, you think client server, there was a lot of resistance from it and now it's more much, Same thing with mobile. I mean mobile is flipped, right? I mean, so okay, bring it in. We gotta deal with it. Yep. I would think the same thing. We, we have a data problem. Let's turn it into an >>Opportunity. Yeah. In my, and it goes back to what I said earlier, the right solution for the right problem at the right time. Like when they, when you have larger amounts of unstructured data, there isn't anything else out there that can even touch what had, can >>Do. So Amar, I need to just change gears here a minute. The gaming stuff. So we have, we we're featured on justin.tv right now on the front page. Oh wow. But the numbers aren't coming in because there's a competing stream of a recently released Modern Warfare three feature. Yes. Yes. So >>I was looking for, we >>Have to compete with Modern Warfare three. So can you, can we talk about Modern Warfare three for a minute and share the folks what you think of the current version, if any, if you played it. Yeah. So >>Unfortunately I'm waiting to get back home. I don't have my Xbox with me here. >>A little like a, I'm talking about >>My lines and business. >>Boom. Water warfares like a Christmas >>Tree here. Sorry. You know, I love, I'm a big gamer. I'm a big video gamer at Cloudera. We have every Thursday at five 30 end office, we, we play Call of of Beauty version four, which is modern world form one actually. And I challenge, I challenge people out there to come challenge our team. Just ping me on Twitter and we'll, we'll do a Cloudera versus >>Let's, let's, let's reframe that. Let team out. There am Abalas company. This is the geeks that invent the future. Jeff Haer Baer at Facebook now at Cloudera. Hammerer leading the charge. These guys are at gamers. So all the young gamers out there am are saying they're gonna challenge you. At which version? >>Modern Warfare one. >>Modern Warfare one. Yes. How do they fire in? Can you set up an >>External We'll >>We'll figure it out. We'll figure it out. Okay. >>Yeah. Just p me on Twitter and We'll, >>We can carry it live actually we can stream that. Yeah, >>That'd be great. >>Great. >>Yeah. So I'll tell you some of our best Hadooop committers and Hadoop developers pitch >>A picture. Modern Warfare >>Three going now Model Warfare three. Very excited about the game. I saw the, the trailers for it looks, graphics look just amazing. Graphics are amazing. I love the Sirius since the first one that came out. And I'm looking forward to getting back home to playing the game. >>I can't play, my son won't let me play. I'm such a fumbler with the Hub. I'm a keyboard controller. I can't work the Xbox controller. Oh, I have a coordination problem my age and I'm just a gluts and like, like Dad, sorry, Charity's over. I can I play with my friends? You the box. But I'm around big gamer. >>But, but in terms of, I mean, something I wanted to bring up is how to link up gaming with big data and analysis and so on. So like, I, I'm a big gamer. I love playing games, but at the same time, whenever I play games, I feel a little bit guilty because it's kind of like wasted time. So it's like, I mean, yeah, it's fun and I'm getting lots of enjoyment on it makes my life much more cheerful. But still, how can we harness all of this, all of these hours that gamers spend playing a game like Modern Warfare three, How can we, how can we collect instrument, all of the data that's coming from that and coming up, for example, with something useful with predicted. >>This is exactly, this is exactly the kind of application that's mainstream is gaming. Yeah. Yeah. Danny at Riot G is telling me, we saw him at Oracle Open World. He's up there for the Java one. He said that they, they don't really have a big data platform and their business is about understanding user behavior rep tons of data about user playing time, who they're playing with. Yeah, Yeah. How they want us to get into currency trading, You know, >>Buy, I can't, I can't mention the names, but some of the biggest giving companies out there are using Hadoop right now. And, and depending on CDH for doing exactly that kind of thing, creating >>A good user experience >>Today, they're doing it for the purpose of enhancing the user experience and improving retention. So they do track everything. Like every single bullet, you fire everything in best Ball Head, you get everything home run, you do. And, and, and in, in a three >>Type of game consecutive headshot, you get >>Everything, everything is being Yeah. Headshot you get and so on. But, but as you said, they are using that information today to sell more products and, and, and retain their users. Now what I'm suggesting is that how can you harness that energy for the good as well? I mean for making money, money is good and everything, but how can you harness that for doing something useful so that all of this entertainment time is also actually productive time as well. I think that'd be a holy grail in this, in this environment if we >>Can achieve that. Yeah. It used to be that corn used to be the telegraph of the future of about, of applications, but gaming really is, if you look at gaming, you know, you get the headset on. It's a collaborative environment. Oh yeah. You got unified communications. >>Yeah. And you see our teenager kids, how, how many hours they spend on these things. >>You got play as a play environments, very social collaborative. Yeah. You know, some say, you know, we we're saying, what I'm saying is that that's the, that's the future work environment with Skype evolving. We're our multiplayer game's called our job. Right? Yeah. You know, so I'm big on gaming. So all the gamers out there, a has challenged you. Yeah. Got a big data example. What else are we seeing? So let's talk about the, the software. So we, one of the things you were talking about that I really liked, you were going down the list. So on Mike's slide he had all the new features. So around the core, can you just go down the core and rattle off your version of what, what it means and what it is. So you start off with say H Base, we talked about that already. What are the other ones that are out there? >>So the projects that we have right there, >>The projects that are around those tools that are being built. Cause >>Yeah, so the foundational, the foundational one as we mentioned before, is sdfs for storage map use for processing. Yeah. And then the, the immediate layer above that is how to make MAP reduce easier for the masses. So how can, not everybody knows how to learn map, use Java, everybody knows sql, right? So, so one of the most successful projects right now that has the highest attach rate, meaning people usually when they install had do installed as well is Hive. So Hive takes sequel and so Jeff Harm Becker, my co-founder, when he was at Facebook, his team built the Hive system. Essentially Hive takes sql so you don't have to learn a new language, you already know sql. And then converts that into MAP use for you. That not only expands the developer base for how many people can use adu, but also makes it easier to integrate Hadoop through all DBC and JDBC integrated with BI tools like MicroStrategy and Tableau and Informatica, et cetera, et cetera. >>You mentioned R too. You mentioned R Program R >>As well. Yeah, R is one of our best partnerships. We're very, very happy with them. So that's, that's one of the very key projects is Hive assisted project to Hive ISS called Pig. A pig Latin is a language that ya invented that you have to learn the language. It's very easy, it's very easy to learn compared to map produce. But once you learn it, you can, you can specify very deep data pipelines, right? SQL is good for queries. It's not good for data pipelines because it becomes very convoluted. It becomes very hard for the, the human brain to understand it. So Pig is much more natural to the human. It's more like Pearl very similar to scripting kind of languages. So with Peggy can write very, very long data pipelines, again, very successful projects doing very, very well. Another key project is Edge Base, like you said. So Edge Base allows you to do low latencies. So you can do very, very quick lookups and also allows you to do transactions. So you can do updates in inserts and deletes. So one of the talks here that had World we try to recommend people watch when the videos come out is the Talk by Jonathan Gray from Facebook. And he talked about how they use Edge Base, >>Jonathan, something on here in the Cube later. Yeah. So >>Drill him on that. So they use Edge Base now for many, many things within Facebook. They have a big team now committed to building an improving edge base with us and with the community at large. And they're using it for doing their online messaging system. The live mail system in Facebook is powered by Edge Base right now. Again, Pro and eBay, The Casini project, they gave a keynote earlier today at the conference as well is using Edge Base as well. So Edge Base is definitely one of the projects that's growing very, very quickly right now within the Hudu system. Another key project that Jeff alluded to earlier when he was on here is Flum. So Flume is very instrumental because you have this nice system had, but Hadoop is useless unless you have data inside it. So how do you get the data inside do? >>So Flum essentially is this very nice framework for having these agents all over your infrastructure, inside your web servers, inside your application servers, inside your mobile devices, your network equipment that collects all of that data and then reliably and, and materializes it inside Hado. So Flum does that. Another good project is Uzi, so many of them, I dunno how, how long you want me to keep going here, But, but Uzi is great. Uzi is a workflow processing system. So Uzi allows you to define a series of jobs. Some of them in Pig, some of them in Hive, some of them in map use. You can define a series of them and then link them to each other and say, only start this job when these other jobs, two jobs finish because I'm waiting for the input from them before I can kick off and so on. >>So Uzi is a very nice framework that will will do that. We'll manage the whole graph of jobs for you and retry things when they fail, et cetera, et cetera. Another good project is where W H I R R and where allows you to very easily start ADU cluster on top of Amazon. Easy two on top of Rackspace, virtualized environ. It's more for kicking off, it's for kicking off Hadoop instances or edge based instances on any virtual infrastructure. Okay. VMware, vCloud. So that it supports all of the major vCloud, sorry, all of the me, all of the major virtualized infrastructure systems out there, Eucalyptus as well, and so on. So that's where W H I R R ARU is another key project. It's one, it's duck cutting's main kind of project right now. Don of that gut cutting came on stage with you guys has, So Aru ARO is a project about how do we encode with our files, the schema of these files, right? >>Because when you open up a text file and you don't know how to what the columns mean and how to pars it, it becomes very hard to work for it. So ARU allows you to do that much more easily. It's also useful for doing rrp. We call rtc remove procedure calls for having different services talk to each other. ARO is very useful for that as well. And the list keeps going on and on Maha. Yeah. Which we just, thanks for me for reminding me of my house. We just added Maha very recently actually. What is that >>Adam? I'm not >>Familiar with it. So Maha is a data mining library. So MAHA takes some of the most popular data mining algorithms for doing clustering and regression and statistical modeling and implements them using the map map with use model. >>They have, they have machine learning in it too or Yes, yes. So that's the machine learning. >>So, So yes. Stay vector to machines and so on. >>What Scoop? >>So Scoop, you know, all of them. Thanks for feeding me all the names. >>The ones I don't understand, >>But there's so many of them, right? I can't even remember all of them. So Scoop actually is a very interesting project, is short for SQL to Hadoop, hence the name Scoop, right? So SQ from SQL and Oops from Hadoop and also means Scoop as in scooping up stuff when you scoop up ice cream. Yeah. And the idea for Scoop is to make it easy to move data between relational systems like Oracle metadata and it is a vertical and so on and Hadoop. So you can very simply say, Scoop the name of the table inside the relation system, the name of the file inside Hadoop. And the, the table will be copied over to the file and Vice and Versa can say Scoop the name of the file in Hadoop, the name of the table over there, it'll move the table over there. So it's a connectivity tool between the relational world and the Hadoop world. >>Great, great tutorial. >>And all of these are Apache projects. They're all projects built. >>It's not part of your, your unique proprietary. >>Yes. But >>These are things that you've been contributing >>To, We're contributing to the whole ecosystem. Yes. >>And you understand very well. Yes. And >>And contribute to your knowledge of the marketplace >>And Absolutely. We collaborate with the, with the community on creating these projects. We employ committers and founders for many of these projects. Like Duck Cutting, the founder of He works in Cloudera, the founder for that UIE project. He works at Calera for zookeeper works at Calera. So we have a number of them on stuff >>Work. So we had Aroon from Horton Works. Yes. And and it was really good because I tell you, I walk away from that conversation and I gotta say for the folks out there, there really isn't a war going on in Apache. There isn't. And >>Apache, there isn't. I mean isn't but would be honest. Like, and in the developer community, we are friends, we're working together. We want to achieve the, there's >>No war. It's all Kumbaya. Everyone understands the rising tide floats, all boats are all playing nice in the same box. Yes. It's just a competitive landscape in Horton. Works >>In the business, >>Business business, competitive business, PR and >>Pr. We're trying to be friendly, as friendly as we can. >>Yeah, no, I mean they're, they're, they're hying it up. But he was like, he was cool. Like, Hey, you know, we know each other. Yes. We all know each other and we're just gonna offer free Yes. And charge with support. And so are they. And that's okay. And they got other things going on. Yes. But he brought up the question. He said they're, they're launching a management console. So I said, Tyler's got a significant lead. He kind of didn't really answer the question. So the question is, that's your core bread and butter, That's your yes >>And no. Yes and no. I mean if you look at, if you look at Cloudera Enterprise, and I mentioned this earlier and when we talked in the morning, it has two main things in it. Cloudera Enterprise has the management suite, but it also has the, the the the support and maintenance that we provide to our customers and all the experience that we have in our team part That subscription. Yes. For a description. And I, I wanna stress the point that the fact that I built a sports car doesn't mean that I'm good at running that sports car. The driver of the car usually is much better at driving the car than the guy who built the car, right? So yes, we have many people on staff that are helping build had, but we have many more people on stuff that helped run Hado at large scale, at at financial indu, financial industry, retail industry, telecom industry, media industry, health industry, et cetera, et cetera. So that's very, very important for our customer. All that experience that we bring in on how to run the system technically Yeah. Within these verticals. >>But their strategies clear. We're gonna create an open source project within Apache for a management consult. Yes. And we sell support too. Yes. So there'll be a free alternative to management. >>So we have to see, But I mean we look at the product, I mean our products, >>It's gotta come down to product differentiation. >>Our product has been in the market for two years, so they just started building their products. It's >>Alpha, It's just Alpha. The >>Product is Alpha in Alpha right now. Yeah. Okay. >>Well the Apache products, it is >>Apache, right? Yeah. The Apache project is out. So we'll see how it does it compare to ours. But I think ours is way, way ahead of anything else out there. Yeah. Essentially people to try that for themselves and >>See essentially, John, when I asked Arro why does the world need Hortonwork? You know, eventually the answer we got was, well it's free. It needs to be more open. Had needs to be more open. >>No, there's, >>It's going to be, That's not really the reason why Warton >>Works. >>No, they want, they want to go make money. >>Exactly. We wasn't >>Gonna say them you >>When I kept pushing and pushing and that's ultimately the closest we can get cuz you >>Just listens. Not gonna >>12 open source projects. Yes. >>I >>Mean, yeah, yeah. You can't get much more open. Yeah. Look >>At management >>Consult, but Airs not shooting on all those. I mean, I mean not only we are No, no, not >>No, no, we absolutely >>Are. No, you are contributing. You're not. But that's not all your projects. There's other people >>Involved. Yeah, we didn't start, we didn't start all of these projects. Yeah, that's >>True. You contributing heavily to all of them. >>Yes, we >>Are. And that's clear. Todd Lipkin said that, you know, he contributed his first patch to HPAC in 2008. Yes. So I mean, you go back through the ranks >>Of your people and Todd now is a committer on Edge base is a committer on had itself. So on a number >>Of you clearly the lead and, and you know, and, but >>There is a concern. But we, we've heard it and I wanna just ask you No, no. So there's a concern that if I build processes around a proprietary management console, Yes. I'm gonna end up being locked into that proprietary management CNA all over again. Now this is so far from ca Yes. >>Right. >>But that's a concern that some people have expressed. And, and, and I think one of the reasons why Port Works is getting so much attention. So Yes. >>Talk about that. It's, it's a very good, it's a very good observation to make. Actually, >>There there is two separate things here. There's the platform where all the data sets and then there's this management parcel beside the platform. Now why did we make the management console why the cloud didn't make the management console? Because it makes our job for supporting the customers much more achievable. When a customer calls in and says, We have a problem, help us fix this problem. When they go to our management console, there is a button they click that gives us a dump of the state, of the cluster. And that's what allows us to very quickly debug what's going on. And within minutes tell them you need to do this and you to do that. Yeah. Without that we just can't offer the support services. There's >>Real value there. >>Yes. So, so now a year from, But, but, but you have to keep in mind that the, the underlying platform is completely open source and free CBH is completely a hundred percent open source, a hundred percent free, a hundred percent Apache. So a year from now, when it comes time to renew with us, if the customer is not happy with our management suite is not happy with our support data, they can, they can go to work >>And works. People are afraid >>Of all they can go to ibm. >>The data, you can take the data that >>You don't even need to take the data. You're not gonna move the data. It's the same system, the same software. Every, everything in CDH is Apache. Right? We're not putting anything in cdh, which is not Apache. So a year from now, if you're not happy with our service to you and the value that we're providing, you can switch. There is no lock in. There is no lock. And >>Your, your argument would be the switching costs to >>The only lock in is happiness. The only lock in is which >>Happiness inspection customer delay. Which by, by the way, we just wrote a piece about those wars and we said the risk of lockin is low. We made that statement. We've got some heat for it. Yes. And >>This is sort of at scale though. What the, what the people are saying, they're throwing the tomatoes is saying if this is, again, in theory at scale, the customers are so comfortable with that, the console that they don't switch. Now my argument was >>Yes, but that means they're happy with it. That means they're satisfied and happy >>With it. >>And it's more economical for them than going and hiding people full-time on stuff. Yeah. >>So you're, you're always on check as, as long as the customer doesn't feel like Oracle. >>Yeah. See that's different. Oracle is very, Oracle >>Is like different, right? Yeah. Here it's like Cisco routers, they get nested into the environment, provide value. That's just good competitive product strategy. Yes. If it they're happy. Yeah. It's >>Called open washing with >>Oracle, >>I mean our number one core attribute on the company, the number one value for us is customer satisfaction. Keeping our people Yeah. Our customers happy with the service that we provide. >>So differentiate in the product. Yes. Keep the commanding lead. That's the strategist. That's the, that's what's happening. That's your goal. Yes. >>That's what's happening. >>Absolutely. Okay. Co-founder of Cloudera, Always a pleasure to have you on the cube. We really appreciate all the hospitality over the beer and a half. And wanna personally thank you for letting us sit in your office and we'll miss you >>And we'll miss you too. We'll >>See you at the, the Cube events off Swing by, thanks for coming on the cube and great to see you and congratulations on all your success. >>Thank >>You. And thanks for the review on Modern Warfare three. Yeah, yeah. >>Love me again. If there any gaming stuff, you know, I.