>>From the cube studios in Palo Alto in Boston, connecting with thought leaders all around the world. This is a cute conversation. Welcome to this cube conversation. I'm Lisa Martin pleased to be joined once again by the co CEO of mem sequel, Raj Verma, Raj, welcome back to the program. >>Thank you very much, Lisa. Great to see you as always. >>It's great to see you as well. I always enjoy our conversations. So why don't you start off because something that's been in the news the last couple of months besides COVID is one of your competitors, snowflake confidentially filed IPO documents with the sec a couple months ago. Just wanted to get your perspective on from a market standpoint. What does that signify? >>Yeah. Firstly, congratulations to the snowflake team. Uh, you know, I've, I have a bunch of friends there, you know, John McMahon, my explosives on the board. And I remember having a conversation with him about seven years ago and it was just starting off and I'm just so glad for him and Bob Mobileye. And, and as I said, a bunch of my friends who are there, um, they're executed brilliantly and, uh, I'm thrilled for that. So, um, we are hearing as to what the outcomes are likely to be. And, uh, it just seems like, uh, you know, it's going to be a great help. Um, and I think what it signifies is firstly, if you have a bit technology and if you execute well, good things happen and there's enough room for innovation here. So that is one, the second aspect is I think, and I think more importantly, what it signifies is a change of thought in the database market. >>If you really see, um, and know if my memory serves me right in the last two decades or probably two and a half buckets, we just had one company go public in the database space and that was Mongo. And, um, and that was in, I think October, 2017 and then, uh, two and a half years. So three years we've seen on other ones and uh, from the industry that we know, um, you know, there are going to be a couple that are going to go out in the next 18 months, 24 months as well. So the fact is that we had a, the iron grip on the database market for almost, you know, more than two decades. It was Oracle, IBM that a bit of Sybase and SAP HANA. And now there are a bunch of companies which are helping solve the problems of tomorrow with the technology of the month. >>And, uh, and that is, um, that is snowflake is a primary example of that. Um, so that's a, that's good change. God is good. I do think the incumbents are gonna find it harder and harder going forward. And also if you really see the evolution of the database market, the first sort of workloads that moved to the cloud with the developer workloads and the big benefactor that that was the no secret movement and one company that executed in my opinion, the best was Mongol. And they were the big benefactor of that, that sort of movement to the cloud. The second was the very large, but Moisey database data warehouse market, and a big benefactor of that has been snowflake big queries, the other one as well. However, the biggest set of tsunami of data that's we are seeing move to the cloud is the operational data, which is the marriage of historical data with real time data to give you real time insights as, or what we call the now are now. >>And that's going to be much, much bigger than, uh, than both the, you know, sequel or the developer data movement and the data warehouse. And we hope to be a benefactor of that. And then the shake up that happens in the database market and the change that's happening there, isn't a vendor take on market anymore, and that's good because you don't then have the stranglehold that Oracle had and you know, some of the ways that are treated as customers and help them to run some, et cetera, um, yeah. And giving customers choice so that they can choose what's best for the business is going to be, it's going to be great. And me are going to see seven to 10 really good database companies in large, in the next decade. And we surely hope them secret as one of them of, we definitely have the, have the potential to be one of them. >>You have the market, we have the product, we have the customers. So, you know, as I tell my team, it's up to us as to what we make of it. And, um, you know, we don't worry that much about competition. You did mention snowflake being advantage station. We, yeah, sure. You know, we do compete on certain opportunities. However, their value proposition is a little more single-threaded than ours. So they are more than the Datavail house space are. Our vision of the board is that, uh, you know, you should have a single store for data, whether it's database house, whether it's developer data or whether it's operational data or DP data. And, uh, you know, watch this space from orders. We make somebody exciting announcements. >>So dig into that a little bit more because some of the news and the commentary Raj in the last, maybe six weeks since the snowflake, um, IPO confidential information was released was, is the enterprise data warehouse dead. And you just had a couple of interesting things we're talking about now, we're seeing this momentum, huge second database to go public in two and a half bigots. That's huge, but that's also signifying to a point you made earlier. There's, there's a shift. So memes SQL isn't, we're not talking about an EDW. We're talking about operational real time. How do you see that if you're not looking in the rear view mirror, those competitors, how do you see that market and the opportunities? >>Yeah, I, I don't think the data warehouse market is dead at thought. I think the very fact that, you know, smoke makers going out at whatever valuation they go out, which is, you know, tens of billions of dollars is, um, is a testimony to the fact that, you know, it's a fancy ad master. This is what it is. I mean, data warehouses have existed for decades and, uh, there is a better way of doing it. So it's a fancy of mousetrap and, and that's great. I mean, that's way to money and it's clearly been demonstrated. Now what we are saying is that I think that is a better way to manage the organization's data rather than having them categorized in buckets of, you know, data warehouse, data developer, data DP, or transactional data, you know, uh, analytical data. Is there a way to imagine the future where there is one single database that you can quit eat, or data warehouse workloads for operational workloads, for OLTB work acknowledge and gain insights. And that's not a fancier mousetrap that is a data strategy reimagine. And, uh, and that's our mission. That's our purpose in life right now and are very excited about it's going to be hard. It's not, it's not a given it's a hard problem to solve. Otherwise, if you can solve it before we have the, uh, we have the goods to deliver and the talent, the deliberate, and, um, we are, we are trying it out with some very, very marquee customers. So we've been very excited about, >>Well, changing of the guard, as you mentioned, is hard. The opposite is easy, the opposite, you know, ignoring and not wanting to get out of that comfort zone. That's taken the easy route in my opinion. So it seems like we've got in the market, this, this significant changing of the guard, not just in, you know, what some of your competition is doing, but also from a customer's perspective, how do you help customers, especially institutions that have been around for decades and decades and decades pivot quickly so that the changing of the guard doesn't wipe them out. >>Yeah. Um, I actually think slightly differently. I think changing of the guard, um, wiping out a customer is if they stick or are resistant to the fact that there is a change of God, you know, and if they, if they hold on to, as we said in our previous conversation, if you stick onto the decisions of yesterday, you will not see the Sundays of tomorrow. So I do think that, uh, you know, change, you have a, God is a, is a symbolism, not even a symbolism as a statement to our customers to say, there is a better way of doing, uh, what you are doing to solve tomorrow's problem. And then doesn't have to be the Oracles and the BB tools and the psychosis of the world. So that's, that's one aspect of it. The second thing is, as I've always said, you're not really that obsessed about, uh, competition. >>The competition will do what they do. Uh, we are really very focused on having an impact in the shortest period of time on our customers and, uh, hopefully a positive impact. And if you can't do it, then, you know, I've had conversations with a few of them saying, maybe be not the company for you. Uh, it's not as if I have to sort of, software's a good one. I supply to the successful customers in the bag to do the unsuccessful with customers. The fact is that, you know, in certain, certain places there isn't an organizational alignment and you don't succeed. However, we do have young, we have in the last 14 months or so made tremendous investments into really ease of use of flexibility of architecture, which is hybrid and tactile, and that shrinking the total time to value for our customers. Because if I, if I believe you, if you do these three things, you will have an impact, a positive impact on the customer, in the sharpest, uh, amount of time and your Lindy or yourself. And I think that is more important than worrying needlessly about competition. And then the competition will do what they do. But if you keep your customers happy by having a positive impact, um, successes, only amount of time, >>Customers and employees are essential to that. But I like that you talked about customer obsession because you see it all over the place. Many people use it as descriptors of themselves and their LinkedIn profiles, for example, but for it actually to be meaningful, you talked about the whole objective is to make an impact for your customers. How do you define that? So that it's not just, I don't want to say marketing term, but something that everyone says they're customer obsessed showing it right within the pudding. >>It's easy to say we are customer obsessed. I mean, this organization is going to say we don't care about our customer. So, you know, of course we all want our customers to be successful. How do you, that's easy, you know, having a cultural value that we put our customers first is, was easy, but we didn't choose to do that. What we said is how do you have an impact on your customer in the shortest amount of time, right? That is, that is what you have. I'm sequel and Lee have now designed every process in mem sequel to align with that word. If, if that is a decision that we have to make a B essentially lenses through the fact of what is in the best interest of our customer and what will get us to have an impact, a positive impact on the customer in the shortest amount of time, that is a decision, which is a buy decision for us to make. >>A lot of times it's more expensive. It's a, a lot duffel. It stresses the, um, the, the, the organization, um, and the people in it. But that's, uh, that's what you have to do if you are. Um, if you are, you know, as, as they say, customer obsessed, um, it is, it's just a term which is easy to use, but very difficult to put here too. And we want to be a tactic. It right to be, we are going to continue to learn. It's a, it's not a destination, it's a journey. And we continue to take decisions and refine our processes do, as I said, huh, impact on our customers in the shortest amount of time. Now, obsessiveness, a lot of times is seen as a negative in the current society that we live in. And there's a reason for that because the, they view view obsession, but I view obsession and aggression is that is a punishing expression, which is really akin to just being cruel, you know, leading by fear and all the rest of it, which is as no place in any organization. >>And I actually think that in society at large, nothing, I believe that doesn't have any place in society. And then there's something which I dumb as instrumentalists, which is, this is where we were. This is where we are. This is where we are going and how do we track our progress on a daily, weekly, monthly basis? And if we, aren't sort of getting to that level that we believe we should get to, if our customers, aren't seeing the value of dramas in the shortest amount of time, what is it that we need to do better? Um, is that obsession, our instrumental aggression is, is, is what we are all about. And that brings with it a level of intensity, which is not what everyone, but then when you are, you know, challenging the institutions which have, uh, you know, the also has to speak for naked, it's gonna take a Herculean effort to ask them. And, uh, you know, the, the basically believed that instrumental aggression in terms of the, uh, you know, having an impact on customer in the shop to smile at time is gonna get us there. And a, and B are glad to have people who actually believe in that. And, uh, and that's why we've made tremendous progress over the course of last, uh, two years. >>So instrumental aggression. Interesting. How you talked about that, it's a provocative statement, but the way that you talk about it almost seems it's a prescriptive, very strategic, well thought out type of moving the business forward, busting through the old guard. Cause let's face it, you know, the big guys, the Oracles they're there, they're not easy for customers to rip and replace, but instrumental aggression seems to kind of go hand in hand with the changing of the guard. You've got to embrace one to be able to deliver the other, right. >>Yeah. So ducks, I think even a fever inventing something new. Um, I mean, yeah, it just requires instrumental aggression, I believe is a, uh, uh, anchor core to most successful organizations, whether in IP or anywhere else. That is a, that is a site to that obsession. And not, I'm not talking about instrumental aggression here, but I'm really talking about the obsession to succeed, uh, which, uh, you know, gave rise to what I think someone called us brilliant jerks and all the rest of it, because that is the sort of negative side of off obsession. And I think the challenge of leadership in our times is how do you foster the positivity of obsession, which needs to change a garden? And that's the instrumental aggression as a, as a tool to, to go there. And how do you prevent the negative side of it, which says that the end justifies the means and, and that's just not true. >>Uh, there is, there is something that's right, and there's something that's wrong. And, uh, and if that is made very clear that the end does not justify the meanings, it creates a lot of trust between, um, Austin, our customers, also not employees. And when their inherent trust, um, happens, then you foster, as I said, the positive side of obsession and, um, get away from the negative side of obsession that you've seen in certain very, very large companies. Now, the one thing that instrumental aggression and obsession brings to a company is that, uh, it makes a lot of people uncomfortable, and this is what I continue to tell. Um, our, our employees and my audience is, um, you know, be comfortable being uncomfortable because what you're trying to do is odd. And it's going to take a, as I say, a Herculean effort. So let's, uh, let's be comfortable being uncomfortable, uh, and have fun doing it. If there's, uh, how many people get a chance to change, uh, industry, which was dominated by a few bears and have such a positive impact, not only on our estimates, but society at large. And, uh, I think it's a privilege. Pressure is a privilege. And, uh, I'm grateful for the opportunity that's been afforded to me and to my colleagues. And, uh, >>It's a great way. Sorry. That's a great way of looking at it. Pressure is a privilege. If you think about, I love what you said, I always say, get, you know, get comfortably uncomfortable. It is a heart in any aspect, whether it's your workouts or your discipline, you know, working from home, it's a hard thing to do to your point. There's a lot of positivity that can come from it. If we think of what's happening this week alone and the U S political climate changing of the old guard, we've got Kamala Harris as our first female VP nominee and how many years, but also from a diversity angle, from a women leadership perspective, blowing the door wide open. >>It's great to see that, um, you know, we have someone that my daughter's going to look up to and say that, uh, you know, yes, there is, there is a place for us in society and we can have a meaningful contribution to society. So I actually think that San Antonio versus nomination is, um, you know, it's a simple ism of change of God, for sure. Um, I have no political agendas, um, at all. Then you can see how it pans out in November, but the one thing is for sure, but it's going to make a lot of people uncomfortable, a change of God, or this makes a lot of people. And, and, uh, and you know, I was reflecting back on something else and in everything that I've actually achieved, which is, is something I'm proud of. I had to go through a zone, but I was extremely uncomfortable. >>Uh, Gould only happens when you have uncomfortable, um, girl to happens in your conference room. And, um, whether it's, um, you know, running them sequel, uh, or are having a society change, uh, if you stick to your comfort zone, you stick to your prejudices and viruses because it's just comfortable there, there's a, uh, wanting to be awkward. And, uh, and, and I think that that's that essential change of God. As I said, at the cost of repeating myself will make a lot of people uncomfortable, but I honestly believe will move the society forward. And, uh, yeah, I, um, I couldn't be more proud of, uh, having a California San Diego would be nominated and it's a, she brings diversity multicultural. And what I loved about it was, you know, we talk about culture and all the rest of it. And she, she was talking about how our parents who were both, uh, uh, at the Berkeley when she was growing up, we were picking up from and she be, you know, in our, in our prime going to protests and Valley. >>And so it was just, uh, it was ingrained in her to be able to challenge the status school and move the society forward. And, uh, you know, she was comfortable being uncomfortable when she was in that, you know, added that. And that's good. Maybe not. I think we sort of, uh, yeah, I, yeah, let's see, let's see what November brings to us, but, um, I think just a nomination has, uh, exchanged a lot of things and, uh, if it's not this time, it can be the next time, but at the time off the bat, but you're going to have a woman by woman president in my lifetime. Um, that's um, I minced about them, uh, and that's just great. >>Well, I should hope so too. And there's so many, I know we've got to wrap here, but so many different data points that show that that technology company actually, companies, excuse me, with women in leadership position are significantly 10, 20% more profitable. So the changing of the guard is hard as you said, but it's time to get uncomfortable. And this is a great example of that as well as the culture that you have at mem sequel Raja. It's always a pleasure and a philosophical time talking with you. I thank you for joining me on the cube today. >>Thank you me since I'm just stay safe, though. >>You as well for my guest, Raj Burma, I'm Lisa Martin. Thank you for watching this cube conversation.

>> Live from the MIT campus in Cambridge, Massachusetts, it's The Cube, covering the 12th annual MIT Chief Data Officer and Information Quality Symposium. Brought to you by Silicon Angle Media. >> Welcome back to The Cube's coverage of MIT CDOIQ, here in Cambridge, Massachusetts. I'm your host, Rebecca Knight, along with my cohost Peter Burris. We're joined by Ilana Golbin. She is the manager of artificial intelligence accelerator PWC... >> Hi. >> Based out of Los Angeles. Thanks so much for coming on the show! >> Thank you for having me. >> So I know you were on the main stage, giving a presentation, really talking about fears, unfounded or not, about how artificial intelligence will change the way companies do business. Lay out the problem for us. Tell our viewers a little bit about how you see the landscape right now. >> Yeah, so I think... We've really all experienced this, that we're generating more data than we ever have in the past. So there's all this data coming in. A few years ago that was the hot topic: big data. That big data's coming and how are we going to harness big data. And big data coupled with this increase in computing power has really enabled us to build stronger models that can provide more predictive power for a variety of use cases. So this is a good thing. The problem is that we're seeing these really cool models come out that are black box. Very difficult to understand how they're making decisions. And it's not just for us as end users, but also developers. We don't really know 100% why some models are making the decisions that they are. And that can be a problem for auditing. It can be a problem for regulation if that comes into play. And as end users for us to trust the model. Comes down to the use case, so why we're building these models. But ultimately we want to ensure that we're building models responsibly so the models are in line with our mission as business, and they also don't do any unintended harm. And so because of that, we need some additional layers to protect ourself. We need to build explainability into models and really understand what they're doing. >> You said two really interesting things. Let's take one and then the other. >> Of course. >> We need to better understand how we build models and we need to do a better job of articulating what those models are. Let's start with the building of models. What does it mean to do a better job of building models? Where are we in the adoption of better? >> So I think right now we're at the point where we just have a lot of data and we're very excited about it and we just want to throw it into whatever models we can and see what we can get that has the best performance. But we need to take a step back and look at the data that we're using. Is the data biased? Does the data match what we see in the real world? Do we have a variety of opinions in both the data collection process and also the model design process? Diversity is not just important for opinions in a room but it's also important for models. So we need to take a step back and make sure that we have that covered. Once we're sure that we have data that's sufficient for our use case and the bias isn't there or the bias is there to the extent that we want it to be, then we can go forward and build these better models. So I think we're at the point where we're really excited, and we're seeing what we can do, but businesses are starting to take a step back and see how they can do that better. >> Now the one B and the tooling, where is the tooling? >> The tooling... If you follow any of the literature, you'll see new publications come out sometimes every minute of the different applications for these really advanced models. Some of the hottest models on the market today are deep learning models and reinforcement learning models. They may not have an application for some businesses yet, but they definitely are building those types of applications, so the techniques themselves are continuing to advance, and I expect them to continue to do so. Mostly because the data is there and the processing power is there and there's so much investment coming in from various government institutions and governments in these types of models. >> And the way typically that these things work is the techniques and the knowledge of techniques advance and then we turn them into tools. So the tools are lagging a little bit still behind the techniques, but it's catching up. Would you agree? >> I would agree with that. Just because commercial tools can't keep up with the pace of academic environment, we wouldn't really expect them to, but once you've invested in a tool you want to try and improve that tool rather than reformat that tool with the best technique that came out yesterday. So there is some kind of iteration that will continue to happen to make sure that our commercially available tools match what we see in the academic space. >> So a second question is, now we've got the model, how do we declare the model? What is the state of the art in articulating metadata, what the model does, what its issues are? How are we doing a better job and what can we do better to characterize these models so they can be more applicable while at the same time maintaining fidelity that was originally intended and embedded? >> I think the first step is identifying your use case. The extent to which we want to explain a model really is dependent on this use case. For instance, if you have a model that is going to be navigating a self-driving car, you probably want to have a lot more rigor around how that model is developed than with a model that targets mailers. There's a lot of middle ground there, and most of the business applications fall into that middle ground, but there're still business risks that need to be considered. So to the extent to which we can clearly articulate and define the use case for an AI application, that will help inform what level of explainability or interpretability we need out of our tool. >> So are you thinking in terms of what it means, how do we successfully define use cases? Do you have templates that you're using at PWC? Or other approaches to ensure that you get the rigor in the definition or the characterization of the model that then can be applied both to a lesser, you know, who are you mailing, versus a life and death situation like, is the car behaving the way it's expected to? >> And yet the mailing, we have the example, the very famous Target example that outed a young teenage girl who was pregnant before. So these can have real life implications. >> And they can, but that's a very rare instance, right? And you could also argue that that's not the same as missing a stop sign and potentially injuring someone in a car. So there are always going to be extremes, but usually when we think about use cases we think about criticality, which is the extent to which someone could be harmed. And vulnerability, which is the willingness for an end user to accept a model and the decision that it makes. A high vulnerability use case could be... Like a few years ago or a year ago I was talking to a professor at UCSC, University of California San Diego, and he was talking to a medical devices company that manufactures devices for monitoring your blood sugar levels. So this could be a high vulnerability case. If you have an incorrect reading, someone's life could be in danger. This medical device was intended to read the blood sugar levels by noninvasive means, just by scanning your skin. But the metric that was used to calculate this blood sugar was correct, it just wasn't the same that an end user was expecting. Because that didn't match, these end users did not accept this device, even though it did operate very well. >> They abandoned it? >> They abandoned it. It didn't sell. And what this comes down to is this is a high vulnerability case. People want to make sure that their lives, the lives of their kids, whoever's using this devices is in good hands, and if they feel like they can't trust it, they're not going to use it. So the use case I do believe is very important, and when we think about use cases, we think of them on those two metrics: vulnerability and criticality. >> Vulnerability and criticality. >> And we're always evolving our thinking on this, but this is our current thinking, yeah. >> Where are we, in terms of the way in which... From your perspective, the way in which corporations are viewing this, do you believe that they have the right amount of trepidation? Or are they too trepidatious when it comes to this? What is the mindset? Speaking in general terms. >> I think everybody's still trying to figure it out. What I've been seeing, personally, is businesses taking a step back and saying, "You know we've been building all these proof of concepts, "or deploying these pilots, "but we haven't done anything enterprise-wide yet." Generally speaking. So what we're seeing are business coming back and saying, "Before we go any further, we need "a comprehensive AI strategy. "We need something central within our organization "that tells us, that defines how we're going to move forward "and build these future tools, so that we're not then "moving backwards and making sure everything aligns." So I think this is really the stage that businesses are in. Once they have a central AI strategy, I think it becomes much easier to evaluate regulatory risks or anything like that. Just because it all reports to a central entity. >> But I want to build on that notion. 'Cause generally we agree. But I want to build on that notion, though. We're doing a good job in the technology world of talking about how we're distributing processing power. We're doing a good job of describing how we're distributing data. And we're even doing a good job of just describing how we're distributing known process. We're not doing a particularly good job of what we call systems of agency. How we're distributing agency. In other words, the degree to which a model is made responsible for acting on behalf of the brand. Now in some domains, medical devices, there is a very clear relationship between what the device says it's going to do, and who ultimately is decided to be, who's culpable. But in the software world, we use copyright law. And copyright law is a speech act. How do we ensure that this notion of agency, we're distributing agency appropriately so that when something is being done on behalf of the brand, that there is a lineage of culpability, a lineage of obligations associated with that? Where are we? >> I think right now we're still... And I can't speak for most organizations, just my personal experience. I think that the companies or the instances I've seen, we're still really early on in that. Because AI is different from traditional software, but it still needs to be audited. So we're at the stage where we're taking a step back and we're saying, "We know we need a mechanism "to monitor and audit our AI." We need controls around this. We need to accurately provide auditing and assurance around our AI applications. But we recognize it's different from traditional software. For a variety of reasons. AI is adaptive. It's not static like traditional software. >> It's probabilistic and not categorical. >> Exactly. So there are a lot of other externalities that need to be considered. And so this is something that a lot of businesses are thinking about. One of the reasons why having a central AI strategy is really important, is that you can also define a central controls framework, some type of centralized assurance and auditing process that's mandated from a high level of the organization that everybody will follow. And that's really the best way to get AI widely adopted. Because otherwise, I think we'll be seeing a lot of challenges. >> So I've got one more question. And one question I have is, if you look out in the next three years, as someone who is working with customers, working with academics, trying to match the need to the expertise, what is the next conversation that's going to pop to the top of the stack in this world, in, say, within the next two years? >> Yeah what we'll we be talking about next year or five years from now, too, at the next CDOIQ? >> I think this topic of explainability will persist. Because I don't think we will necessarily tick all the boxes in the next year. I think we'll uncover new challenges and we'll have to think about new ways to explain how models are operating. Other than that, I think customers will want to see more transparency in the process itself. So not just the model and how it's making its decisions, but what data is feeding into that. How are you using my data to impact how a model is making decisions on my behalf? What is feeding into my credit score? And what can I do to improve it? Those are the types of conversations I think we'll be having in the next two years, for sure. >> Great, well Ilana, thanks so much for coming on The Cube. It was great having you. >> Thank you for having me. >> I'm Rebecca Knight for Peter Burris. We will have more from MIT Chief Data Officer Symposium 2018 just after this. (upbeat electronic music)