>> (Narrator) Live from Madrid, Spain. It's The Cube covering HP Discover Madrid 2017, brought to you by Hewlett Packard Enterprise. >> We're back at HPE Discover Madrid 2017. This is The Cube, the leader in live tech coverage. My name is Dave Vellante and I'm with my co-host for the week, Peter Burris. Cat Graves is here, she's a research scientist at Hewlett Packard Enterprises. And she's joined by Natalia Vassilieva. Cube alum, senior research manager at HPE. Both with the labs in Palo Alto. Thanks so much for coming on The Cube. >> Thank you for having us. >> You're welcome. So for decades this industry has marched to the cadence of Moore's Law, bowed down to Moore's Law, been subservient to Moore's Law. But that's changing, isn't it? >> Absolutely. >> What's going on? >> I can tell Moore's Law is changing. So we can't increase the number, of course, on the same chip and have the same space. We can't increase the density of the computer today. And from the software perspective, we need to analyze more and more data. We are now marching calls into the area of artificial intelligence when we need to train larger and larger models, we need more and more compute for that. And the only possible way today to speed up the training of those modules, to actually enable the AI, is to scale out. Because we can't put more cores on the chip. So we try to use more chips together But then communication bottlenecks come in. So we can't efficiently use all of those chips. So for us on the software side, on the part of people who works how to speed up the training, how to speed up the implementation of the algorithms, and the work of those algorithms, that's a problem. And that's where Cat can help us because she's working on a new hardware which will overcome those troubles. >> Yeah, so in our lab what we do is try and think of new ways of doing computation but also doing the computations that really matter. You know, what are the bottlenecks for the applications that Natalia is working on that are really preventing the performance from accelerating? Again exponentially like Moore's Law, right? We'd like to return to Moore's Law where we're in that sort of exponential growth in terms of what compute is really capable of. And so what we're doing in labs is leveraging novel devices so, you've heard of memristor in the past probably. But instead of using memristor for computer memory, non volatile memory for persistent memory driven computer systems, we're using these devices instead for doing computation itself in the analog domain. So one of our first target applications, and target core computations that we're going after is matrix multiplication. And that is a fundamental mathematical building block for a lot of different machine learning, deep learning, signal processing, you kind of name it, it's pretty broad in terms of where it's used today. >> So Dr. Tom Bradicich was talking about the dot product, and it sounds like it's related. Matrix multiplications, suddenly I start breaking out in hives but is that kind of related? >> That's exactly what it is. So, if you remember your linear algebra in college, a dot product is exactly a matrix multiplication. It's the dot in between the vector and the matrix. The two itself, so exactly right. Our hardware prototype is called the dot product engine. It's just cranking out those matrix multiplications. >> And can you explain how that addresses the problem that we're trying to solve with respect to Moore's Law? >> Yeah, let me. You mentioned the problem with Moore's Law. From me as a software person, the end of Moore's Law is a bad thing because I can't increase their compute power anymore on the single chip. But for Cat it's a good thing because it forced her to think what's unconventional. >> (Cat) It's an opportunity. >> It's an opportunity! >> It forced her to think, what are unconventional devices which she can come up with? And we also have to mention they understand that general purpose computing is not always a solution. Sometimes if you want to speed up the thing, you need to come up with a device which is designed specifically for the type of computation which you care about. And for machine learning technification, again as I've mentioned, these matrix-matrix multiplications matrix-vector multiplications, these are the core of it. Today if you want to do those AI type applications, you spend roughly 90% of the time doing exactly that computation. So if we can come up with a more power efficient and a more effective way of doing that, that will really help us, and that's what dot product engine is solving. >> Yes, an example some of our colleagues did in architectural work. Sort of taking the dot product engine as the core, and then saying, okay if I designed a computer architecture specifically for doing convolutional neural networks. So image classification, these kinds of applications. If I built this architecture, how would it perform? And how would it compare to GPUs? And we're seeing 10 to 100 X speed up over GPUs. And even 15 X speed up over if you had a custom-built, state of the art specialized digital Asic. Even comparing to the best that we can do today, we are seeing this potential for a huge amount of speed up and also energy savings as well. >> So follow up on that, if I may. So you're saying these alternative processors like GPUs, FGPAs, custom Asics, can I infer from that that that is a stop-gap architecturally, in your mind? Because you're seeing these alternative processors pop up all over the place. >> (Cat) Yes. >> Is that a fair assertion? >> I think that recent trends are obviously favoring a return to specialized hardware. >> (Dave) Yeah, for sure. Just look at INVIDIA, it's exploding. >> I think it really depends on the application and you have to look at what the requirements are. Especially in terms of where there's a lot of power limitations, right, GPUs have become a little bit tricky. So there's a lot of interest in the automotive industry, space, robotics, for more low power but still very high performance, highly efficient computation. >> Many years ago when I was actually thinking about doing computer science and realized pretty quickly that I didn't have the brain power to get there. But I remember thinking in terms of there's three ways of improving performance. You can do it architecturally, what do you do with an instruction? You can do it organizationally, how do you fit the various elements together? You can do it with technology, which is what's the clock speed, what's the underlying substrate? Moore's Law is focused on the technology. Risk, for example, focused on architecture. FPGAs, arm processors, GPUs focus on architecture. What we're talking about to get back to that doubling the performance every 18 months from a computing standpoint not just a chip standpoint, now we're talking about revealing and liberating, I presume, some of the organization elements. Ways of thinking about how to put these things together. So even if we can't get improvements that we've gotten out of technology, we can start getting more performance out of new architectures. But organizing how everything works together. And make it so that the software doesn't have to know, or the developer, doesn't have to know everything about the organization. Am I kind of getting there with this? >> Yes, I think you are right. And if we are talking about some of the architectural challenges of today's processors, not only we can't increase the power of a single device today, but even if we increase the power of a single device, then the challenge would be how do you bring the data fast enough to that device? So we will have problems with feeding that device. And again, what dot product engine does, it does computations in memory, inside. So you limit the number of data transfers between different chips and you don't face the problem of feeding their computation thing. >> So similar same technology, different architecture, and using a new organization to take advantage of that architecture. The dot product engine being kind of that combination. >> I would say that even technology is different. >> Yeah, my view of it we're actually thinking about it holistically. We have in labs software working with architects. >> I mean it's not just a clock speed issue. >> It's not just a clock speed issue. It's thinking about what computations actually matter, which ones you're actually doing, and how to perform them in different ways. And so one of the great things as well with the dot product engine and these kind of new computation accelerators, is with something like the memory driven computing architecture. We have now an ecosystem that is really favoring accelerators and encouraging the development of these specialized hardware pieces that can kind of slot in in the same architecture that can scale also in size. >> And you invoke that resource in an automated way, presumably. >> Yeah, exactly. >> What's the secret sauce behind that? Is that software that does that or an algorithm that chooses the algorithm? >> A gen z. >> A gen z's underlying protocol is to make the device talk to the data. But at the end of the system software, it's algorithms also which will make a decision at every particular point which compute device I should use to do a particular task. With memory driven computing, if all my data sits in the shared pool of memory and I have different heterogeneous compute devices, being able to see that data and to talk to that data, then it's up to the system management software to allocate the execution of a particular task to the device which does that the best. In a more power efficient way, in the fastest way, and everybody wins. >> So as a software person, you now with memory driven computing have been thinking about developing software in a completely different way. Is that correct? >> (Natalia) Yeah. You're not thinking about going through I/O stack anymore and waiting for a mechanical device and doing other things? >> It's not only the I/O stack. >> As I mentioned today, the only possibility for us to decrease the time of processing for the algorithms is to scale out. That means that I need to take into account the locality of the data. It's not only when you distribute the computation across multiple nodes, even if we have some number based which is we have different sockets in a single system. With local memory and the memory which is remote to that socket but which is local to another socket. Today as a software programmer, as a developer, I need to take into account where my data sits. Because I know in order to accept the data on a local memory it'll take me 100 seconds to accept my data. In the remote socket, it will take me longer. So when I developed the algorithm in order to prevent my computational course to stall and to wait for the data, I need to schedule that very carefully. With memory driven computing, giving an assumption that, again, all memory not only in the single pool, but it's also evenly accessible from every compute device. I don't need to care about that anymore. And you can't even imagine such a relief it is! (laughs) It makes our life so much easier. >> Yeah, because you're spending a lot of time previously trying to optimize your code >> Yes for that factor of the locality of the data. How much of your time was spent doing that menial task? >> Years! In the beginning of Moore's Law and the beginning of the traditional architectures, if you turn to the HPC applications, every HPC application device today needs to take care of data locality. >> And you hear about when a new GPU comes out or even just a slightly new generation. They have to take months to even redesign their algorithm to tune it to that specific hardware, right? And that's the same company, maybe even the same product sort of path lined. But just because that architecture has slightly changed changes exactly what Natalia is talking about. >> I'm interested in switching subjects here. I'd love to spend a minute on women in tech. How you guys got into this role. You're both obviously strong in math, computer backgrounds. But give us a little flavor of your background, Cat, and then, Natalia, you as well. >> Me or you? >> You start. >> Hm, I don't know. I was always interested in a lot of different things. I kind of wanted to study and do everything. And I got to the point in college where physics was something that still fascinated me. I felt like I didn't know nearly enough. I felt like there was still so much to learn and it was constantly challenging me. So I decided to pursue my Ph.D in that, and it's never boring, and you're always learning something new. Yeah, I don't know. >> Okay, and that led to a career in technology development. >> Yeah, and I actually did my Ph.D in kind of something that was pretty different. But towards the end of it, decided I really enjoyed research and was just always inspired by it. But I wanted to do that research on projects that I felt like might have more of an impact. And particularly an impact in my lifetime. My Ph.D work was kind of something that I knew would never actually be implemented in, maybe a couple hundred years or something we might get to that point. So there's not too many places, at least in my field in hardware, where you can be doing what feels like very cutting edge research, but be doing it in a place where you can see your ideas and your work be implemented. That's something that led me to labs. >> And Natalia, what's your passion? How did you arrive here? >> As a kid I always liked different math puzzles. I was into math and pretty soon it became obvious that I like solving those math problems much more than writing about anything. I think in middle school there was the first class on programming, I went right into that. And then the teacher told me that I should probably go to a specialized school and that led me to physics and mathematics lyceum and then mathematical department at the university so it was pretty straightforward for me since then. >> You're both obviously very comfortable in this role, extremely knowledgeable. You seem like great leaders. Why do you feel that more women don't pursue a career in technology. Do you have these discussions amongst yourselves? Is this something that you even think about? >> I think it starts very early. For me, both my parents are scientists, and so always had books around the house. Always was encouraged to think and pursue that path, and be curious. I think its something that happens at a very young age. And various academic institutions have done studies and shown when they do certain things, its surmountable. Carnegie Mellon has a very nice program for this, where they went for the percentage of women in their CS program went from 10% to 40% in five years. And there were a couple of strategies that they implemented. I'm not gonna get all of them, but one was peer to peer mentoring, when the freshmen came in, pairing them with a senior, feeling like you're not the only one doing what you're doing, or interested in what you're doing. It's like anything human, you want to feel like you belong and can relate to your group. So I think, yeah. (laughs) >> Let's have a last word. >> On that topic? >> Yeah sure, or any topic. But yes, I'm very interested in this topic because less than 20% of the tech business is women. Its 50W% of the population. >> I think for me its not the percentage which matters Just don't stay in the way of those who's interested in that. And give equal opportunities to everybody. And yes, the environment from the very childhood should be the proper one. >> Do you feel like the industry gives women equal opportunity? >> For me, my feeling would be yes. You also need to understand >> Because of your experience Because of my experience, but I also originally came from Russia, was born in St. Petersburg, and I do believe that ex-Soviet Union countries has much better history in that. Because the Soviet Union, we don't have man and woman. We have comrades. And after the Second World War, there was women who took all hard jobs. And we used to get moms at work. All moms of all my peers have been working. My mom was an engineer, my dad is an engineer. From that, there is no perception that the woman should stay at home, or the woman is taking care of kids. There is less of that. >> Interesting. So for me, yes. Now I think that industry going that direction. And that's right. >> Instructive, great. Well, listen, thanks very much for coming on the Cube. >> Sure. >> Sharing the stories, and good luck in lab, wherever you may end up. >> Thank you. >> Good to see you. >> Thank you very much. >> Alright, keep it right there everybody. We'll be back with our next guest, Dave Vallante for Peter Buress. We're live from Madrid, 2017, HPE Discover. This is the Cube.