(bright upbeat music) >> In 2011, early Facebook employee and Cloudera co-founder Jeff Hammerbacher famously said, "The best minds of my generation are thinking about how to get people to click on ads, and that sucks!" Let's face it. More than a decade later, organizations continue to be frustrated with how difficult it is to get value from data and build a truly agile and data-driven enterprise. What does that even mean, you ask? Well, it means that everyone in the organization has the data they need when they need it in a context that's relevant to advance the mission of an organization. Now, that could mean cutting costs, could mean increasing profits, driving productivity, saving lives, accelerating drug discovery, making better diagnoses, solving supply chain problems, predicting weather disasters, simplifying processes, and thousands of other examples where data can completely transform people's lives beyond manipulating internet users to behave a certain way. We've heard the prognostications about the possibilities of data before and in fairness we've made progress, but the hard truth is the original promises of master data management, enterprise data warehouses, data marts, data hubs, and yes even data lakes were broken and left us wanting for more. Welcome to The Data Doesn't Lie... Or Does It? A series of conversations produced by theCUBE and made possible by Starburst Data. I'm your host, Dave Vellante, and joining me today are three industry experts. Justin Borgman is the co-founder and CEO of Starburst, Richard Jarvis is the CTO at EMIS Health, and Teresa Tung is cloud first technologist at Accenture. Today, we're going to have a candid discussion that will expose the unfulfilled, and yes, broken promises of a data past. We'll expose data lies: big lies, little lies, white lies, and hidden truths. And we'll challenge, age old data conventions and bust some data myths. We're debating questions like is the demise of a single source of truth inevitable? Will the data warehouse ever have feature parity with the data lake or vice versa? Is the so-called modern data stack simply centralization in the cloud, AKA the old guards model in new cloud close? How can organizations rethink their data architectures and regimes to realize the true promises of data? Can and will an open ecosystem deliver on these promises in our lifetimes? We're spanning much of the Western world today. Richard is in the UK, Teresa is on the West Coast, and Justin is in Massachusetts with me. I'm in theCUBE studios, about 30 miles outside of Boston. Folks, welcome to the program. Thanks for coming on. >> Thanks for having us. >> Okay, let's get right into it. You're very welcome. Now, here's the first lie. The most effective data architecture is one that is centralized with a team of data specialists serving various lines of business. What do you think Justin? >> Yeah, definitely a lie. My first startup was a company called Hadapt, which was an early SQL engine for IDU that was acquired by Teradata. And when I got to Teradata, of course, Teradata is the pioneer of that central enterprise data warehouse model. One of the things that I found fascinating was that not one of their customers had actually lived up to that vision of centralizing all of their data into one place. They all had data silos. They all had data in different systems. They had data on prem, data in the cloud. Those companies were acquiring other companies and inheriting their data architecture. So despite being the industry leader for 40 years, not one of their customers truly had everything in one place. So I think definitely history has proven that to be a lie. >> So Richard, from a practitioner's point of view, what are your thoughts? I mean, there's a lot of pressure to cut cost, keep things centralized, serve the business as best as possible from that standpoint. What does your experience show? >> Yeah, I mean, I think I would echo Justin's experience really that we as a business have grown up through acquisition, through storing data in different places sometimes to do information governance in different ways to store data in a platform that's close to data experts people who really understand healthcare data from pharmacies or from doctors. And so, although if you were starting from a greenfield site and you were building something brand new, you might be able to centralize all the data and all of the tooling and teams in one place. The reality is that businesses just don't grow up like that. And it's just really impossible to get that academic perfection of storing everything in one place. >> Teresa, I feel like Sarbanes-Oxley have kind of saved the data warehouse, right? (laughs) You actually did have to have a single version of the truth for certain financial data, but really for some of those other use cases I mentioned, I do feel like the industry has kind of let us down. What's your take on this? Where does it make sense to have that sort of centralized approach versus where does it make sense to maybe decentralize? >> I think you got to have centralized governance, right? So from the central team, for things like Sarbanes-Oxley, for things like security, for certain very core data sets having a centralized set of roles, responsibilities to really QA, right? To serve as a design authority for your entire data estate, just like you might with security, but how it's implemented has to be distributed. Otherwise, you're not going to be able to scale, right? So being able to have different parts of the business really make the right data investments for their needs. And then ultimately, you're going to collaborate with your partners. So partners that are not within the company, right? External partners. We're going to see a lot more data sharing and model creation. And so you're definitely going to be decentralized. >> So Justin, you guys last, jeez, I think it was about a year ago, had a session on data mesh. It was a great program. You invited Zhamak Dehghani. Of course, she's the creator of the data mesh. One of our fundamental premises is that you've got this hyper specialized team that you've got to go through if you want anything. But at the same time, these individuals actually become a bottleneck, even though they're some of the most talented people in the organization. So I guess, a question for you Richard. How do you deal with that? Do you organize so that there are a few sort of rock stars that build cubes and the like or have you had any success in sort of decentralizing with your constituencies that data model? >> Yeah. So we absolutely have got rockstar data scientists and data guardians, if you like. People who understand what it means to use this data, particularly the data that we use at EMIS is very private, it's healthcare information. And some of the rules and regulations around using the data are very complex and strict. So we have to have people who understand the usage of the data, then people who understand how to build models, how to process the data effectively. And you can think of them like consultants to the wider business because a pharmacist might not understand how to structure a SQL query, but they do understand how they want to process medication information to improve patient lives. And so that becomes a consulting type experience from a set of rock stars to help a more decentralized business who needs to understand the data and to generate some valuable output. >> Justin, what do you say to a customer or prospect that says, "Look, Justin. I got a centralized team and that's the most cost effective way to serve the business. Otherwise, I got duplication." What do you say to that? >> Well, I would argue it's probably not the most cost effective, and the reason being really twofold. I think, first of all, when you are deploying a enterprise data warehouse model, the data warehouse itself is very expensive, generally speaking. And so you're putting all of your most valuable data in the hands of one vendor who now has tremendous leverage over you for many, many years to come. I think that's the story at Oracle or Teradata or other proprietary database systems. But the other aspect I think is that the reality is those central data warehouse teams, as much as they are experts in the technology, they don't necessarily understand the data itself. And this is one of the core tenets of data mesh that Zhamak writes about is this idea of the domain owners actually know the data the best. And so by not only acknowledging that data is generally decentralized, and to your earlier point about Sarbanes-Oxley, maybe saving the data warehouse, I would argue maybe GDPR and data sovereignty will destroy it because data has to be decentralized for those laws to be compliant. But I think the reality is the data mesh model basically says data's decentralized and we're going to turn that into an asset rather than a liability. And we're going to turn that into an asset by empowering the people that know the data the best to participate in the process of curating and creating data products for consumption. So I think when you think about it that way, you're going to get higher quality data and faster time to insight, which is ultimately going to drive more revenue for your business and reduce costs. So I think that that's the way I see the two models comparing and contrasting. >> So do you think the demise of the data warehouse is inevitable? Teresa, you work with a lot of clients. They're not just going to rip and replace their existing infrastructure. Maybe they're going to build on top of it, but what does that mean? Does that mean the EDW just becomes less and less valuable over time or it's maybe just isolated to specific use cases? What's your take on that? >> Listen, I still would love all my data within a data warehouse. I would love it mastered, would love it owned by a central team, right? I think that's still what I would love to have. That's just not the reality, right? The investment to actually migrate and keep that up to date, I would say it's a losing battle. Like we've been trying to do it for a long time. Nobody has the budgets and then data changes, right? There's going to be a new technology that's going to emerge that we're going to want to tap into. There's going to be not enough investment to bring all the legacy, but still very useful systems into that centralized view. So you keep the data warehouse. I think it's a very, very valuable, very high performance tool for what it's there for, but you could have this new mesh layer that still takes advantage of the things I mentioned: the data products in the systems that are meaningful today, and the data products that actually might span a number of systems. Maybe either those that either source systems with the domains that know it best, or the consumer-based systems or products that need to be packaged in a way that'd be really meaningful for that end user, right? Each of those are useful for a different part of the business and making sure that the mesh actually allows you to use all of them. >> So, Richard, let me ask you. Take Zhamak's principles back to those. You got the domain ownership and data as product. Okay, great. Sounds good. But it creates what I would argue are two challenges: self-serve infrastructure, let's park that for a second, and then in your industry, one of the most regulated, most sensitive, computational governance. How do you automate and ensure federated governance in that mesh model that Teresa was just talking about? >> Well, it absolutely depends on some of the tooling and processes that you put in place around those tools to centralize the security and the governance of the data. And I think although a data warehouse makes that very simple 'cause it's a single tool, it's not impossible with some of the data mesh technologies that are available. And so what we've done at EMIS is we have a single security layer that sits on top of our data mesh, which means that no matter which user is accessing which data source, we go through a well audited, well understood security layer. That means that we know exactly who's got access to which data field, which data tables. And then everything that they do is audited in a very kind of standard way regardless of the underlying data storage technology. So for me, although storing the data in one place might not be possible, understanding where your source of truth is and securing that in a common way is still a valuable approach, and you can do it without having to bring all that data into a single bucket so that it's all in one place. And so having done that and investing quite heavily in making that possible has paid dividends in terms of giving wider access to the platform, and ensuring that only data that's available under GDPR and other regulations is being used by the data users. >> Yeah. So Justin, we always talk about data democratization, and up until recently, they really haven't been line of sight as to how to get there, but do you have anything to add to this because you're essentially doing analytic queries with data that's all dispersed all over. How are you seeing your customers handle this challenge? >> Yeah, I mean, I think data products is a really interesting aspect of the answer to that. It allows you to, again, leverage the data domain owners, the people who know the data the best, to create data as a product ultimately to be consumed. And we try to represent that in our product as effectively, almost eCommerce like experience where you go and discover and look for the data products that have been created in your organization, and then you can start to consume them as you'd like. And so really trying to build on that notion of data democratization and self-service, and making it very easy to discover and start to use with whatever BI tool you may like or even just running SQL queries yourself. >> Okay guys, grab a sip of water. After the short break, we'll be back to debate whether proprietary or open platforms are the best path to the future of data excellence. Keep it right there. (bright upbeat music)

>>In 2011, early Facebook employee and Cloudera co-founder Jeff Ocker famously said the best minds of my generation are thinking about how to get people to click on ads. And that sucks. Let's face it more than a decade later organizations continue to be frustrated with how difficult it is to get value from data and build a truly agile data-driven enterprise. What does that even mean? You ask? Well, it means that everyone in the organization has the data they need when they need it. In a context that's relevant to advance the mission of an organization. Now that could mean cutting cost could mean increasing profits, driving productivity, saving lives, accelerating drug discovery, making better diagnoses, solving, supply chain problems, predicting weather disasters, simplifying processes, and thousands of other examples where data can completely transform people's lives beyond manipulating internet users to behave a certain way. We've heard the prognostications about the possibilities of data before and in fairness we've made progress, but the hard truth is the original promises of master data management, enterprise data, warehouses, data marts, data hubs, and yes, even data lakes were broken and left us wanting from more welcome to the data doesn't lie, or doesn't a series of conversations produced by the cube and made possible by Starburst data. >>I'm your host, Dave Lanta and joining me today are three industry experts. Justin Borgman is this co-founder and CEO of Starburst. Richard Jarvis is the CTO at EMI health and Theresa tongue is cloud first technologist at Accenture. Today we're gonna have a candid discussion that will expose the unfulfilled and yes, broken promises of a data past we'll expose data lies, big lies, little lies, white lies, and hidden truths. And we'll challenge, age old data conventions and bust some data myths. We're debating questions like is the demise of a single source of truth. Inevitable will the data warehouse ever have featured parody with the data lake or vice versa is the so-called modern data stack, simply centralization in the cloud, AKA the old guards model in new cloud close. How can organizations rethink their data architectures and regimes to realize the true promises of data can and will and open ecosystem deliver on these promises in our lifetimes, we're spanning much of the Western world today. Richard is in the UK. Teresa is on the west coast and Justin is in Massachusetts with me. I'm in the cube studios about 30 miles outside of Boston folks. Welcome to the program. Thanks for coming on. Thanks for having us. Let's get right into it. You're very welcome. Now here's the first lie. The most effective data architecture is one that is centralized with a team of data specialists serving various lines of business. What do you think Justin? >>Yeah, definitely a lie. My first startup was a company called hit adapt, which was an early SQL engine for hit that was acquired by Teradata. And when I got to Teradata, of course, Teradata is the pioneer of that central enterprise data warehouse model. One of the things that I found fascinating was that not one of their customers had actually lived up to that vision of centralizing all of their data into one place. They all had data silos. They all had data in different systems. They had data on prem data in the cloud. You know, those companies were acquiring other companies and inheriting their data architecture. So, you know, despite being the industry leader for 40 years, not one of their customers truly had everything in one place. So I think definitely history has proven that to be a lie. >>So Richard, from a practitioner's point of view, you know, what, what are your thoughts? I mean, there, there's a lot of pressure to cut cost, keep things centralized, you know, serve the business as best as possible from that standpoint. What, what is your experience show? >>Yeah, I mean, I think I would echo Justin's experience really that we, as a business have grown up through acquisition, through storing data in different places sometimes to do information governance in different ways to store data in, in a platform that's close to data experts, people who really understand healthcare data from pharmacies or from, from doctors. And so, although if you were starting from a Greenfield site and you were building something brand new, you might be able to centralize all the data and all of the tooling and teams in one place. The reality is that that businesses just don't grow up like that. And, and it's just really impossible to get that academic perfection of, of storing everything in one place. >>Y you know, Theresa, I feel like Sarbanes Oxley kinda saved the data warehouse, you know, right. You actually did have to have a single version of the truth for certain financial data, but really for those, some of those other use cases, I, I mentioned, I, I do feel like the industry has kinda let us down. What's your take on this? Where does it make sense to have that sort of centralized approach versus where does it make sense to maybe decentralized? >>I, I think you gotta have centralized governance, right? So from the central team, for things like star Oxley, for things like security for certainly very core data sets, having a centralized set of roles, responsibilities to really QA, right. To serve as a design authority for your entire data estate, just like you might with security, but how it's implemented has to be distributed. Otherwise you're not gonna be able to scale. Right? So being able to have different parts of the business really make the right data investments for their needs. And then ultimately you're gonna collaborate with your partners. So partners that are not within the company, right. External partners, we're gonna see a lot more data sharing and model creation. And so you're definitely going to be decentralized. >>So, you know, Justin, you guys last, geez, I think it was about a year ago, had a session on, on data mesh. It was a great program. You invited Jamma, Dani, of course, she's the creator of the data mesh. And her one of our fundamental premises is that you've got this hyper specialized team that you've gotta go through. And if you want anything, but at the same time, these, these individuals actually become a bottleneck, even though they're some of the most talented people in the organization. So I guess question for you, Richard, how do you deal with that? Do you, do you organize so that there are a few sort of rock stars that, that, you know, build cubes and, and the like, and, and, and, or have you had any success in sort of decentralizing with, you know, your, your constituencies, that data model? >>Yeah. So, so we absolutely have got rockstar, data scientists and data guardians. If you like people who understand what it means to use this data, particularly as the data that we use at emos is very private it's healthcare information. And some of the, the rules and regulations around using the data are very complex and, and strict. So we have to have people who understand the usage of the data, then people who understand how to build models, how to process the data effectively. And you can think of them like consultants to the wider business, because a pharmacist might not understand how to structure a SQL query, but they do understand how they want to process medication information to improve patient lives. And so that becomes a, a consulting type experience from a, a set of rock stars to help a, a more decentralized business who needs to, to understand the data and to generate some valuable output. >>Justin, what do you say to a, to a customer or prospect that says, look, Justin, I'm gonna, I got a centralized team and that's the most cost effective way to serve the business. Otherwise I got, I got duplication. What do you say to that? >>Well, I, I would argue it's probably not the most cost effective and, and the reason being really twofold. I think, first of all, when you are deploying a enterprise data warehouse model, the, the data warehouse itself is very expensive, generally speaking. And so you're putting all of your most valuable data in the hands of one vendor who now has tremendous leverage over you, you know, for many, many years to come. I think that's the story at Oracle or Terra data or other proprietary database systems. But the other aspect I think is that the reality is those central data warehouse teams is as much as they are experts in the technology. They don't necessarily understand the data itself. And this is one of the core tenants of data mash that that jam writes about is this idea of the domain owners actually know the data the best. >>And so by, you know, not only acknowledging that data is generally decentralized and to your earlier point about SAR, brain Oxley, maybe saving the data warehouse, I would argue maybe GDPR and data sovereignty will destroy it because data has to be decentralized for, for those laws to be compliant. But I think the reality is, you know, the data mesh model basically says, data's decentralized, and we're gonna turn that into an asset rather than a liability. And we're gonna turn that into an asset by empowering the people that know the data, the best to participate in the process of, you know, curating and creating data products for, for consumption. So I think when you think about it, that way, you're going to get higher quality data and faster time to insight, which is ultimately going to drive more revenue for your business and reduce costs. So I think that that's the way I see the two, the two models comparing and contrasting. >>So do you think the demise of the data warehouse is inevitable? I mean, I mean, you know, there Theresa you work with a lot of clients, they're not just gonna rip and replace their existing infrastructure. Maybe they're gonna build on top of it, but what does that mean? Does that mean the E D w just becomes, you know, less and less valuable over time, or it's maybe just isolated to specific use cases. What's your take on that? >>Listen, I still would love all my data within a data warehouse would love it. Mastered would love it owned by essential team. Right? I think that's still what I would love to have. That's just not the reality, right? The investment to actually migrate and keep that up to date. I would say it's a losing battle. Like we've been trying to do it for a long time. Nobody has the budgets and then data changes, right? There's gonna be a new technology. That's gonna emerge that we're gonna wanna tap into. There's going to be not enough investment to bring all the legacy, but still very useful systems into that centralized view. So you keep the data warehouse. I think it's a very, very valuable, very high performance tool for what it's there for, but you could have this, you know, new mesh layer that still takes advantage of the things. I mentioned, the data products in the systems that are meaningful today and the data products that actually might span a number of systems, maybe either those that either source systems for the domains that know it best, or the consumer based systems and products that need to be packaged in a way that be really meaningful for that end user, right? Each of those are useful for a different part of the business and making sure that the mesh actually allows you to use all of them. >>So, Richard, let me ask you, you take, take Gemma's principles back to those. You got to, you know, domain ownership and, and, and data as product. Okay, great. Sounds good. But it creates what I would argue are two, you know, challenges, self-serve infrastructure let's park that for a second. And then in your industry, the one of the high, most regulated, most sensitive computational governance, how do you automate and ensure federated governance in that mesh model that Theresa was just talking about? >>Well, it absolutely depends on some of the tooling and processes that you put in place around those tools to be, to centralize the security and the governance of the data. And I think, although a data warehouse makes that very simple, cause it's a single tool, it's not impossible with some of the data mesh technologies that are available. And so what we've done at emus is we have a single security layer that sits on top of our data match, which means that no matter which user is accessing, which data source, we go through a well audited well understood security layer. That means that we know exactly who's got access to which data field, which data tables. And then everything that they do is, is audited in a very kind of standard way, regardless of the underlying data storage technology. So for me, although storing the data in one place might not be possible understanding where your source of truth is and securing that in a common way is still a valuable approach and you can do it without having to bring all that data into a single bucket so that it's all in one place. And, and so having done that and investing quite heavily in making that possible has paid dividends in terms of giving wider access to the platform and ensuring that only data that's available under GDPR and other regulations is being used by, by the data users. >>Yeah. So Justin, I mean, Democrat, we always talk about data democratization and you know, up until recently, they really haven't been line of sight as to how to get there. But do you have anything to add to this because you're essentially taking, you know, do an analytic queries and with data that's all dispersed all over the, how are you seeing your customers handle this, this challenge? >>Yeah. I mean, I think data products is a really interesting aspect of the answer to that. It allows you to, again, leverage the data domain owners, people know the data, the best to, to create, you know, data as a product ultimately to be consumed. And we try to represent that in our product as effectively a almost eCommerce like experience where you go and discover and look for the data products that have been created in your organization. And then you can start to consume them as, as you'd like. And so really trying to build on that notion of, you know, data democratization and self-service, and making it very easy to discover and, and start to use with whatever BI tool you, you may like, or even just running, you know, SQL queries yourself, >>Okay. G guys grab a sip of water. After this short break, we'll be back to debate whether proprietary or open platforms are the best path to the future of data excellence, keep it right there. >>Your company has more data than ever, and more people trying to understand it, but there's a problem. Your data is stored across multiple systems. It's hard to access and that delays analytics and ultimately decisions. The old method of moving all of your data into a single source of truth is slow and definitely not built for the volume of data we have today or where we are headed while your data engineers spent over half their time, moving data, your analysts and data scientists are left, waiting, feeling frustrated, unproductive, and unable to move the needle for your business. But what if you could spend less time moving or copying data? What if your data consumers could analyze all your data quickly? >>Starburst helps your teams run fast queries on any data source. We help you create a single point of access to your data, no matter where it's stored. And we support high concurrency, we solve for speed and scale, whether it's fast, SQL queries on your data lake or faster queries across multiple data sets, Starburst helps your teams run analytics anywhere you can't afford to wait for data to be available. Your team has questions that need answers. Now with Starburst, the wait is over. You'll have faster access to data with enterprise level security, easy connectivity, and 24 7 support from experts, organizations like Zolando Comcast and FINRA rely on Starburst to move their businesses forward. Contact our Trino experts to get started. >>We're back with Jess Borgman of Starburst and Richard Jarvis of EVAs health. Okay, we're gonna get to lie. Number two, and that is this an open source based platform cannot give you the performance and control that you can get with a proprietary system. Is that a lie? Justin, the enterprise data warehouse has been pretty dominant and has evolved and matured. Its stack has mature over the years. Why is it not the default platform for data? >>Yeah, well, I think that's become a lie over time. So I, I think, you know, if we go back 10 or 12 years ago with the advent of the first data lake really around Hudu, that probably was true that you couldn't get the performance that you needed to run fast, interactive, SQL queries in a data lake. Now a lot's changed in 10 or 12 years. I remember in the very early days, people would say, you you'll never get performance because you need to be column there. You need to store data in a column format. And then, you know, column formats we're introduced to, to data apes, you have Parque ORC file in aro that were created to ultimately deliver performance out of that. So, okay. We got, you know, largely over the performance hurdle, you know, more recently people will say, well, you don't have the ability to do updates and deletes like a traditional data warehouse. >>And now we've got the creation of new data formats, again like iceberg and Delta and Hodi that do allow for updates and delete. So I think the data lake has continued to mature. And I remember a, a quote from, you know, Kurt Monash many years ago where he said, you know, know it takes six or seven years to build a functional database. I think that's that's right. And now we've had almost a decade go by. So, you know, these technologies have matured to really deliver very, very close to the same level performance and functionality of, of cloud data warehouses. So I think the, the reality is that's become a line and now we have large giant hyperscale internet companies that, you know, don't have the traditional data warehouse at all. They do all of their analytics in a data lake. So I think we've, we've proven that it's very much possible today. >>Thank you for that. And so Richard, talk about your perspective as a practitioner in terms of what open brings you versus, I mean, look closed is it's open as a moving target. I remember Unix used to be open systems and so it's, it is an evolving, you know, spectrum, but, but from your perspective, what does open give you that you can't get from a proprietary system where you are fearful of in a proprietary system? >>I, I suppose for me open buys us the ability to be unsure about the future, because one thing that's always true about technology is it evolves in a, a direction, slightly different to what people expect. And what you don't want to end up is done is backed itself into a corner that then prevents it from innovating. So if you have chosen a technology and you've stored trillions of records in that technology and suddenly a new way of processing or machine learning comes out, you wanna be able to take advantage and your competitive edge might depend upon it. And so I suppose for us, we acknowledge that we don't have perfect vision of what the future might be. And so by backing open storage technologies, we can apply a number of different technologies to the processing of that data. And that gives us the ability to remain relevant, innovate on our data storage. And we have bought our way out of the, any performance concerns because we can use cloud scale infrastructure to scale up and scale down as we need. And so we don't have the concerns that we don't have enough hardware today to process what we want to do, want to achieve. We can just scale up when we need it and scale back down. So open source has really allowed us to maintain the being at the cutting edge. >>So Jess, let me play devil's advocate here a little bit, and I've talked to Shaak about this and you know, obviously her vision is there's an open source that, that the data meshes open source, an open source tooling, and it's not a proprietary, you know, you're not gonna buy a data mesh. You're gonna build it with, with open source toolings and, and vendors like you are gonna support it, but to come back to sort of today, you can get to market with a proprietary solution faster. I'm gonna make that statement. You tell me if it's a lie and then you can say, okay, we support Apache iceberg. We're gonna support open source tooling, take a company like VMware, not really in the data business, but how, the way they embraced Kubernetes and, and you know, every new open source thing that comes along, they say, we do that too. Why can't proprietary systems do that and be as effective? >>Yeah, well, I think at least with the, within the data landscape saying that you can access open data formats like iceberg or, or others is, is a bit dis disingenuous because really what you're selling to your customer is a certain degree of performance, a certain SLA, and you know, those cloud data warehouses that can reach beyond their own proprietary storage drop all the performance that they were able to provide. So it is, it reminds me kind of, of, again, going back 10 or 12 years ago when everybody had a connector to Haddo and that they thought that was the solution, right? But the reality was, you know, a connector was not the same as running workloads in Haddo back then. And I think similarly, you know, being able to connect to an external table that lives in an open data format, you know, you're, you're not going to give it the performance that your customers are accustomed to. And at the end of the day, they're always going to be predisposed. They're always going to be incentivized to get that data ingested into the data warehouse, cuz that's where they have control. And you know, the bottom line is the database industry has really been built around vendor lockin. I mean, from the start, how, how many people love Oracle today, but our customers, nonetheless, I think, you know, lockin is, is, is part of this industry. And I think that's really what we're trying to change with open data formats. >>Well, that's interesting reminded when I, you know, I see the, the gas price, the tees or gas price I, I drive up and then I say, oh, that's the cash price credit card. I gotta pay 20 cents more, but okay. But so the, the argument then, so let me, let me come back to you, Justin. So what's wrong with saying, Hey, we support open data formats, but yeah, you're gonna get better performance if you, if you keep it into our closed system, are you saying that long term that's gonna come back and bite you cuz you're gonna end up, you mentioned Oracle, you mentioned Teradata. Yeah. That's by, by implication, you're saying that's where snowflake customers are headed. >>Yeah, absolutely. I think this is a movie that, you know, we've all seen before. At least those of us who've been in the industry long enough to, to see this movie play over a couple times. So I do think that's the future. And I think, you know, I loved what Richard said. I actually wrote it down. Cause I thought it was an amazing quote. He said, it buys us the ability to be unsure of the future. Th that that pretty much says it all the, the future is unknowable and the reality is using open data formats. You remain interoperable with any technology you want to utilize. If you want to use spark to train a machine learning model and you want to use Starbust to query via sequel, that's totally cool. They can both work off the same exact, you know, data, data sets by contrast, if you're, you know, focused on a proprietary model, then you're kind of locked in again to that model. I think the same applies to data, sharing to data products, to a wide variety of, of aspects of the data landscape that a proprietary approach kind of closes you in and locks you in. >>So I, I would say this Richard, I'd love to get your thoughts on it. Cause I talked to a lot of Oracle customers, not as many te data customers, but, but a lot of Oracle customers and they, you know, they'll admit, yeah, you know, they're jamming us on price and the license cost they give, but we do get value out of it. And so my question to you, Richard, is, is do the, let's call it data warehouse systems or the proprietary systems. Are they gonna deliver a greater ROI sooner? And is that in allure of, of that customers, you know, are attracted to, or can open platforms deliver as fast in ROI? >>I think the answer to that is it can depend a bit. It depends on your businesses skillset. So we are lucky that we have a number of proprietary teams that work in databases that provide our operational data capability. And we have teams of analytics and big data experts who can work with open data sets and open data formats. And so for those different teams, they can get to an ROI more quickly with different technologies for the business though, we can't do better for our operational data stores than proprietary databases. Today we can back off very tight SLAs to them. We can demonstrate reliability from millions of hours of those databases being run at enterprise scale, but for an analytics workload where increasing our business is growing in that direction, we can't do better than open data formats with cloud-based data mesh type technologies. And so it's not a simple answer. That one will always be the right answer for our business. We definitely have times when proprietary databases provide a capability that we couldn't easily represent or replicate with open technologies. >>Yeah. Richard, stay with you. You mentioned, you know, you know, some things before that, that strike me, you know, the data brick snowflake, you know, thing is, oh, is a lot of fun for analysts like me. You've got data bricks coming at it. Richard, you mentioned you have a lot of rockstar, data engineers, data bricks coming at it from a data engineering heritage. You get snowflake coming at it from an analytics heritage. Those two worlds are, are colliding people like PJI Mohan said, you know what? I think it's actually harder to play in the data engineering. So I E it's easier to for data engineering world to go into the analytics world versus the reverse, but thinking about up and coming engineers and developers preparing for this future of data engineering and data analytics, how, how should they be thinking about the future? What, what's your advice to those young people? >>So I think I'd probably fall back on general programming skill sets. So the advice that I saw years ago was if you have open source technologies, the pythons and Javas on your CV, you commander 20% pay, hike over people who can only do proprietary programming languages. And I think that's true of data technologies as well. And from a business point of view, that makes sense. I'd rather spend the money that I save on proprietary licenses on better engineers, because they can provide more value to the business that can innovate us beyond our competitors. So I think I would my advice to people who are starting here or trying to build teams to capitalize on data assets is begin with open license, free capabilities, because they're very cheap to experiment with. And they generate a lot of interest from people who want to join you as a business. And you can make them very successful early, early doors with, with your analytics journey. >>It's interesting. Again, analysts like myself, we do a lot of TCO work and have over the last 20 plus years. And in world of Oracle, you know, normally it's the staff, that's the biggest nut in total cost of ownership, not an Oracle. It's the it's the license cost is by far the biggest component in the, in the blame pie. All right, Justin, help us close out this segment. We've been talking about this sort of data mesh open, closed snowflake data bricks. Where does Starburst sort of as this engine for the data lake data lake house, the data warehouse fit in this, in this world? >>Yeah. So our view on how the future ultimately unfolds is we think that data lakes will be a natural center of gravity for a lot of the reasons that we described open data formats, lowest total cost of ownership, because you get to choose the cheapest storage available to you. Maybe that's S3 or Azure data lake storage, or Google cloud storage, or maybe it's on-prem object storage that you bought at a, at a really good price. So ultimately storing a lot of data in a deal lake makes a lot of sense, but I think what makes our perspective unique is we still don't think you're gonna get everything there either. We think that basically centralization of all your data assets is just an impossible endeavor. And so you wanna be able to access data that lives outside of the lake as well. So we kind of think of the lake as maybe the biggest place by volume in terms of how much data you have, but to, to have comprehensive analytics and to truly understand your business and understand it holistically, you need to be able to go access other data sources as well. And so that's the role that we wanna play is to be a single point of access for our customers, provide the right level of fine grained access controls so that the right people have access to the right data and ultimately make it easy to discover and consume via, you know, the creation of data products as well. >>Great. Okay. Thanks guys. Right after this quick break, we're gonna be back to debate whether the cloud data model that we see emerging and the so-called modern data stack is really modern, or is it the same wine new bottle? When it comes to data architectures, you're watching the cube, the leader in enterprise and emerging tech coverage. >>Your data is capable of producing incredible results, but data consumers are often left in the dark without fast access to the data they need. Starers makes your data visible from wherever it lives. Your company is acquiring more data in more places, more rapidly than ever to rely solely on a data centralization strategy. Whether it's in a lake or a warehouse is unrealistic. A single source of truth approach is no longer viable, but disconnected data silos are often left untapped. We need a new approach. One that embraces distributed data. One that enables fast and secure access to any of your data from anywhere with Starburst, you'll have the fastest query engine for the data lake that allows you to connect and analyze your disparate data sources no matter where they live Starburst provides the foundational technology required for you to build towards the vision of a decentralized data mesh Starburst enterprise and Starburst galaxy offer enterprise ready, connectivity, interoperability, and security features for multiple regions, multiple clouds and everchanging global regulatory requirements. The data is yours. And with Starburst, you can perform analytics anywhere in light of your world. >>Okay. We're back with Justin Boardman. CEO of Starbust Richard Jarvis is the CTO of EMI health and Theresa tongue is the cloud first technologist from Accenture. We're on July number three. And that is the claim that today's modern data stack is actually modern. So I guess that's the lie it's it is it's is that it's not modern. Justin, what do you say? >>Yeah. I mean, I think new isn't modern, right? I think it's the, it's the new data stack. It's the cloud data stack, but that doesn't necessarily mean it's modern. I think a lot of the components actually are exactly the same as what we've had for 40 years, rather than Terra data. You have snowflake rather than Informatica you have five trend. So it's the same general stack, just, you know, a cloud version of it. And I think a lot of the challenges that it plagued us for 40 years still maintain. >>So lemme come back to you just, but okay. But, but there are differences, right? I mean, you can scale, you can throw resources at the problem. You can separate compute from storage. You really, you know, there's a lot of money being thrown at that by venture capitalists and snowflake, you mentioned it's competitors. So that's different. Is it not, is that not at least an aspect of, of modern dial it up, dial it down. So what, what do you say to that? >>Well, it, it is, it's certainly taking, you know, what the cloud offers and taking advantage of that, but it's important to note that the cloud data warehouses out there are really just separating their compute from their storage. So it's allowing them to scale up and down, but your data still stored in a proprietary format. You're still locked in. You still have to ingest the data to get it even prepared for analysis. So a lot of the same sort of structural constraints that exist with the old enterprise data warehouse model OnPrem still exist just yes, a little bit more elastic now because the cloud offers that. >>So Theresa, let me go to you cuz you have cloud first in your, in your, your title. So what's what say you to this conversation? >>Well, even the cloud providers are looking towards more of a cloud continuum, right? So the centralized cloud, as we know it, maybe data lake data warehouse in the central place, that's not even how the cloud providers are looking at it. They have news query services. Every provider has one that really expands those queries to be beyond a single location. And if we look at a lot of where our, the future goes, right, that that's gonna very much fall the same thing. There was gonna be more edge. There's gonna be more on premise because of data sovereignty, data gravity, because you're working with different parts of the business that have already made major cloud investments in different cloud providers. Right? So there's a lot of reasons why the modern, I guess, the next modern generation of the data staff needs to be much more federated. >>Okay. So Richard, how do you deal with this? You you've obviously got, you know, the technical debt, the existing infrastructure it's on the books. You don't wanna just throw it out. A lot of, lot of conversation about modernizing applications, which a lot of times is a, you know, a microservices layer on top of leg legacy apps. How do you think about the modern data stack? >>Well, I think probably the first thing to say is that the stack really has to include the processes and people around the data as well is all well and good changing the technology. But if you don't modernize how people use that technology, then you're not going to be able to, to scale because just cuz you can scale CPU and storage doesn't mean you can get more people to use your data, to generate you more, more value for the business. And so what we've been looking at is really changing in very much aligned to data products and, and data mesh. How do you enable more people to consume the service and have the stack respond in a way that keeps costs low? Because that's important for our customers consuming this data, but also allows people to occasionally run enormous queries and then tick along with smaller ones when required. And it's a good job we did because during COVID all of a sudden we had enormous pressures on our data platform to answer really important life threatening queries. And if we couldn't scale both our data stack and our teams, we wouldn't have been able to answer those as quickly as we had. So I think the stack needs to support a scalable business, not just the technology itself. >>Well thank you for that. So Justin let's, let's try to break down what the critical aspects are of the modern data stack. So you think about the past, you know, five, seven years cloud obviously has given a different pricing model. De-risked experimentation, you know that we talked about the ability to scale up scale down, but it's, I'm, I'm taking away that that's not enough based on what Richard just said. The modern data stack has to serve the business and enable the business to build data products. I, I buy that. I'm a big fan of the data mesh concepts, even though we're early days. So what are the critical aspects if you had to think about, you know, paying, maybe putting some guardrails and definitions around the modern data stack, what does that look like? What are some of the attributes and, and principles there >>Of, of how it should look like or, or how >>It's yeah. What it should be. >>Yeah. Yeah. Well, I think, you know, in, in Theresa mentioned this in, in a previous segment about the data warehouse is not necessarily going to disappear. It just becomes one node, one element of the overall data mesh. And I, I certainly agree with that. So by no means, are we suggesting that, you know, snowflake or Redshift or whatever cloud data warehouse you may be using is going to disappear, but it's, it's not going to become the end all be all. It's not the, the central single source of truth. And I think that's the paradigm shift that needs to occur. And I think it's also worth noting that those who were the early adopters of the modern data stack were primarily digital, native born in the cloud young companies who had the benefit of, of idealism. They had the benefit of it was starting with a clean slate that does not reflect the vast majority of enterprises. >>And even those companies, as they grow up mature out of that ideal state, they go buy a business. Now they've got something on another cloud provider that has a different data stack and they have to deal with that heterogeneity that is just change and change is a part of life. And so I think there is an element here that is almost philosophical. It's like, do you believe in an absolute ideal where I can just fit everything into one place or do I believe in reality? And I think the far more pragmatic approach is really what data mesh represents. So to answer your question directly, I think it's adding, you know, the ability to access data that lives outside of the data warehouse, maybe living in open data formats in a data lake or accessing operational systems as well. Maybe you want to directly access data that lives in an Oracle database or a Mongo database or, or what have you. So creating that flexibility to really Futureproof yourself from the inevitable change that you will, you won't encounter over time. >>So thank you. So there, based on what Justin just said, I, my takeaway there is it's inclusive, whether it's a data Mar data hub, data lake data warehouse, it's a, just a node on the mesh. Okay. I get that. Does that include there on Preem data? O obviously it has to, what are you seeing in terms of the ability to, to take that data mesh concept on Preem? I mean, most implementations I've seen in data mesh, frankly really aren't, you know, adhering to the philosophy. They're maybe, maybe it's data lake and maybe it's using glue. You look at what JPMC is doing. Hello, fresh, a lot of stuff happening on the AWS cloud in that, you know, closed stack, if you will. What's the answer to that Theresa? >>I mean, I, I think it's a killer case for data. Me, the fact that you have valuable data sources, OnPrem, and then yet you still wanna modernize and take the best of cloud cloud is still, like we mentioned, there's a lot of great reasons for it around the economics and the way ability to tap into the innovation that the cloud providers are giving around data and AI architecture. It's an easy button. So the mesh allows you to have the best of both worlds. You can start using the data products on-prem or in the existing systems that are working already. It's meaningful for the business. At the same time, you can modernize the ones that make business sense because it needs better performance. It needs, you know, something that is, is cheaper or, or maybe just tap into better analytics to get better insights, right? So you're gonna be able to stretch and really have the best of both worlds. That, again, going back to Richard's point, that is meaningful by the business. Not everything has to have that one size fits all set a tool. >>Okay. Thank you. So Richard, you know, talking about data as product, wonder if we could give us your perspectives here, what are the advantages of treating data as a product? What, what role do data products have in the modern data stack? We talk about monetizing data. What are your thoughts on data products? >>So for us, one of the most important data products that we've been creating is taking data that is healthcare data across a wide variety of different settings. So information about patients' demographics about their, their treatment, about their medications and so on, and taking that into a standards format that can be utilized by a wide variety of different researchers because misinterpreting that data or having the data not presented in the way that the user is expecting means that you generate the wrong insight. And in any business, that's clearly not a desirable outcome, but when that insight is so critical, as it might be in healthcare or some security settings, you really have to have gone to the trouble of understanding the data, presenting it in a format that everyone can clearly agree on. And then letting people consume in a very structured, managed way, even if that data comes from a variety of different sources in, in, in the first place. And so our data product journey has really begun by standardizing data across a number of different silos through the data mesh. So we can present out both internally and through the right governance externally to, to researchers. >>So that data product through whatever APIs is, is accessible, it's discoverable, but it's obviously gotta be governed as well. You mentioned you, you appropriately provided to internally. Yeah. But also, you know, external folks as well. So the, so you've, you've architected that capability today >>We have, and because the data is standard, it can generate value much more quickly and we can be sure of the security and, and, and value that that's providing because the data product isn't just about formatting the data into the correct tables, it's understanding what it means to redact the data or to remove certain rows from it or to interpret what a date actually means. Is it the start of the contract or the start of the treatment or the date of birth of a patient? These things can be lost in the data storage without having the proper product management around the data to say in a very clear business context, what does this data mean? And what does it mean to process this data for a particular use case? >>Yeah, it makes sense. It's got the context. If the, if the domains own the data, you, you gotta cut through a lot of the, the, the centralized teams, the technical teams that, that data agnostic, they don't really have that context. All right. Let's send Justin, how does Starburst fit into this modern data stack? Bring us home. >>Yeah. So I think for us, it's really providing our customers with, you know, the flexibility to operate and analyze data that lives in a wide variety of different systems. Ultimately giving them that optionality, you know, and optionality provides the ability to reduce costs, store more in a data lake rather than data warehouse. It provides the ability for the fastest time to insight to access the data directly where it lives. And ultimately with this concept of data products that we've now, you know, incorporated into our offering as well, you can really create and, and curate, you know, data as a product to be shared and consumed. So we're trying to help enable the data mesh, you know, model and make that an appropriate compliment to, you know, the, the, the modern data stack that people have today. >>Excellent. Hey, I wanna thank Justin Theresa and Richard for joining us today. You guys are great. I big believers in the, in the data mesh concept, and I think, you know, we're seeing the future of data architecture. So thank you. Now, remember, all these conversations are gonna be available on the cube.net for on-demand viewing. You can also go to starburst.io. They have some great content on the website and they host some really thought provoking interviews and, and, and they have awesome resources, lots of data mesh conversations over there, and really good stuff in, in the resource section. So check that out. Thanks for watching the data doesn't lie or does it made possible by Starburst data? This is Dave Valante for the cube, and we'll see you next time. >>The explosion of data sources has forced organizations to modernize their systems and architecture and come to terms with one size does not fit all for data management today. Your teams are constantly moving and copying data, which requires time management. And in some cases, double paying for compute resources. Instead, what if you could access all your data anywhere using the BI tools and SQL skills your users already have. And what if this also included enterprise security and fast performance with Starburst enterprise, you can provide your data consumers with a single point of secure access to all of your data, no matter where it lives with features like strict, fine grained, access control, end to end data encryption and data masking Starburst meets the security standards of the largest companies. Starburst enterprise can easily be deployed anywhere and managed with insights where data teams holistically view their clusters operation and query execution. So they can reach meaningful business decisions faster, all this with the support of the largest team of Trino experts in the world, delivering fully tested stable releases and available to support you 24 7 to unlock the value in all of your data. You need a solution that easily fits with what you have today and can adapt to your architecture. Tomorrow. Starbust enterprise gives you the fastest path from big data to better decisions, cuz your team can't afford to wait. Trino was created to empower analytics anywhere and Starburst enterprise was created to give you the enterprise grade performance, connectivity, security management, and support your company needs organizations like Zolando Comcast and FINRA rely on Starburst to move their businesses forward. Contact us to get started.

>>In 2011, early Facebook employee and Cloudera co-founder Jeff Ocker famously said the best minds of my generation are thinking about how to get people to click on ads. And that sucks. Let's face it more than a decade later organizations continue to be frustrated with how difficult it is to get value from data and build a truly agile data-driven enterprise. What does that even mean? You ask? Well, it means that everyone in the organization has the data they need when they need it. In a context that's relevant to advance the mission of an organization. Now that could mean cutting cost could mean increasing profits, driving productivity, saving lives, accelerating drug discovery, making better diagnoses, solving, supply chain problems, predicting weather disasters, simplifying processes, and thousands of other examples where data can completely transform people's lives beyond manipulating internet users to behave a certain way. We've heard the prognostications about the possibilities of data before and in fairness we've made progress, but the hard truth is the original promises of master data management, enterprise data, warehouses, data marts, data hubs, and yes, even data lakes were broken and left us wanting from more welcome to the data doesn't lie, or doesn't a series of conversations produced by the cube and made possible by Starburst data. >>I'm your host, Dave Lanta and joining me today are three industry experts. Justin Borgman is this co-founder and CEO of Starburst. Richard Jarvis is the CTO at EMI health and Theresa tongue is cloud first technologist at Accenture. Today we're gonna have a candid discussion that will expose the unfulfilled and yes, broken promises of a data past we'll expose data lies, big lies, little lies, white lies, and hidden truths. And we'll challenge, age old data conventions and bust some data myths. We're debating questions like is the demise of a single source of truth. Inevitable will the data warehouse ever have featured parody with the data lake or vice versa is the so-called modern data stack, simply centralization in the cloud, AKA the old guards model in new cloud close. How can organizations rethink their data architectures and regimes to realize the true promises of data can and will and open ecosystem deliver on these promises in our lifetimes, we're spanning much of the Western world today. Richard is in the UK. Teresa is on the west coast and Justin is in Massachusetts with me. I'm in the cube studios about 30 miles outside of Boston folks. Welcome to the program. Thanks for coming on. Thanks for having us. Let's get right into it. You're very welcome. Now here's the first lie. The most effective data architecture is one that is centralized with a team of data specialists serving various lines of business. What do you think Justin? >>Yeah, definitely a lie. My first startup was a company called hit adapt, which was an early SQL engine for hit that was acquired by Teradata. And when I got to Teradata, of course, Teradata is the pioneer of that central enterprise data warehouse model. One of the things that I found fascinating was that not one of their customers had actually lived up to that vision of centralizing all of their data into one place. They all had data silos. They all had data in different systems. They had data on prem data in the cloud. You know, those companies were acquiring other companies and inheriting their data architecture. So, you know, despite being the industry leader for 40 years, not one of their customers truly had everything in one place. So I think definitely history has proven that to be a lie. >>So Richard, from a practitioner's point of view, you know, what, what are your thoughts? I mean, there, there's a lot of pressure to cut cost, keep things centralized, you know, serve the business as best as possible from that standpoint. What, what is your experience show? >>Yeah, I mean, I think I would echo Justin's experience really that we, as a business have grown up through acquisition, through storing data in different places sometimes to do information governance in different ways to store data in, in a platform that's close to data experts, people who really understand healthcare data from pharmacies or from, from doctors. And so, although if you were starting from a Greenfield site and you were building something brand new, you might be able to centralize all the data and all of the tooling and teams in one place. The reality is that that businesses just don't grow up like that. And, and it's just really impossible to get that academic perfection of, of storing everything in one place. >>Y you know, Theresa, I feel like Sarbanes Oxley kinda saved the data warehouse, you know, right. You actually did have to have a single version of the truth for certain financial data, but really for those, some of those other use cases, I, I mentioned, I, I do feel like the industry has kinda let us down. What's your take on this? Where does it make sense to have that sort of centralized approach versus where does it make sense to maybe decentralized? >>I, I think you gotta have centralized governance, right? So from the central team, for things like star Oxley, for things like security for certainly very core data sets, having a centralized set of roles, responsibilities to really QA, right. To serve as a design authority for your entire data estate, just like you might with security, but how it's implemented has to be distributed. Otherwise you're not gonna be able to scale. Right? So being able to have different parts of the business really make the right data investments for their needs. And then ultimately you're gonna collaborate with your partners. So partners that are not within the company, right. External partners, we're gonna see a lot more data sharing and model creation. And so you're definitely going to be decentralized. >>So, you know, Justin, you guys last, geez, I think it was about a year ago, had a session on, on data mesh. It was a great program. You invited Jamma, Dani, of course, she's the creator of the data mesh. And her one of our fundamental premises is that you've got this hyper specialized team that you've gotta go through. And if you want anything, but at the same time, these, these individuals actually become a bottleneck, even though they're some of the most talented people in the organization. So I guess question for you, Richard, how do you deal with that? Do you, do you organize so that there are a few sort of rock stars that, that, you know, build cubes and, and the like, and, and, and, or have you had any success in sort of decentralizing with, you know, your, your constituencies, that data model? >>Yeah. So, so we absolutely have got rockstar, data scientists and data guardians. If you like people who understand what it means to use this data, particularly as the data that we use at emos is very private it's healthcare information. And some of the, the rules and regulations around using the data are very complex and, and strict. So we have to have people who understand the usage of the data, then people who understand how to build models, how to process the data effectively. And you can think of them like consultants to the wider business, because a pharmacist might not understand how to structure a SQL query, but they do understand how they want to process medication information to improve patient lives. And so that becomes a, a consulting type experience from a, a set of rock stars to help a, a more decentralized business who needs to, to understand the data and to generate some valuable output. >>Justin, what do you say to a, to a customer or prospect that says, look, Justin, I'm gonna, I got a centralized team and that's the most cost effective way to serve the business. Otherwise I got, I got duplication. What do you say to that? >>Well, I, I would argue it's probably not the most cost effective and, and the reason being really twofold. I think, first of all, when you are deploying a enterprise data warehouse model, the, the data warehouse itself is very expensive, generally speaking. And so you're putting all of your most valuable data in the hands of one vendor who now has tremendous leverage over you, you know, for many, many years to come. I think that's the story at Oracle or Terra data or other proprietary database systems. But the other aspect I think is that the reality is those central data warehouse teams is as much as they are experts in the technology. They don't necessarily understand the data itself. And this is one of the core tenants of data mash that that jam writes about is this idea of the domain owners actually know the data the best. >>And so by, you know, not only acknowledging that data is generally decentralized and to your earlier point about SAR, brain Oxley, maybe saving the data warehouse, I would argue maybe GDPR and data sovereignty will destroy it because data has to be decentralized for, for those laws to be compliant. But I think the reality is, you know, the data mesh model basically says, data's decentralized, and we're gonna turn that into an asset rather than a liability. And we're gonna turn that into an asset by empowering the people that know the data, the best to participate in the process of, you know, curating and creating data products for, for consumption. So I think when you think about it, that way, you're going to get higher quality data and faster time to insight, which is ultimately going to drive more revenue for your business and reduce costs. So I think that that's the way I see the two, the two models comparing and contrasting. >>So do you think the demise of the data warehouse is inevitable? I mean, I mean, you know, there Theresa you work with a lot of clients, they're not just gonna rip and replace their existing infrastructure. Maybe they're gonna build on top of it, but what does that mean? Does that mean the E D w just becomes, you know, less and less valuable over time, or it's maybe just isolated to specific use cases. What's your take on that? >>Listen, I still would love all my data within a data warehouse would love it. Mastered would love it owned by essential team. Right? I think that's still what I would love to have. That's just not the reality, right? The investment to actually migrate and keep that up to date. I would say it's a losing battle. Like we've been trying to do it for a long time. Nobody has the budgets and then data changes, right? There's gonna be a new technology. That's gonna emerge that we're gonna wanna tap into. There's going to be not enough investment to bring all the legacy, but still very useful systems into that centralized view. So you keep the data warehouse. I think it's a very, very valuable, very high performance tool for what it's there for, but you could have this, you know, new mesh layer that still takes advantage of the things. I mentioned, the data products in the systems that are meaningful today and the data products that actually might span a number of systems, maybe either those that either source systems for the domains that know it best, or the consumer based systems and products that need to be packaged in a way that be really meaningful for that end user, right? Each of those are useful for a different part of the business and making sure that the mesh actually allows you to use all of them. >>So, Richard, let me ask you, you take, take Gemma's principles back to those. You got to, you know, domain ownership and, and, and data as product. Okay, great. Sounds good. But it creates what I would argue are two, you know, challenges, self-serve infrastructure let's park that for a second. And then in your industry, the one of the high, most regulated, most sensitive computational governance, how do you automate and ensure federated governance in that mesh model that Theresa was just talking about? >>Well, it absolutely depends on some of the tooling and processes that you put in place around those tools to be, to centralize the security and the governance of the data. And I think, although a data warehouse makes that very simple, cause it's a single tool, it's not impossible with some of the data mesh technologies that are available. And so what we've done at emus is we have a single security layer that sits on top of our data match, which means that no matter which user is accessing, which data source, we go through a well audited well understood security layer. That means that we know exactly who's got access to which data field, which data tables. And then everything that they do is, is audited in a very kind of standard way, regardless of the underlying data storage technology. So for me, although storing the data in one place might not be possible understanding where your source of truth is and securing that in a common way is still a valuable approach and you can do it without having to bring all that data into a single bucket so that it's all in one place. And, and so having done that and investing quite heavily in making that possible has paid dividends in terms of giving wider access to the platform and ensuring that only data that's available under GDPR and other regulations is being used by, by the data users. >>Yeah. So Justin, I mean, Democrat, we always talk about data democratization and you know, up until recently, they really haven't been line of sight as to how to get there. But do you have anything to add to this because you're essentially taking, you know, do an analytic queries and with data that's all dispersed all over the, how are you seeing your customers handle this, this challenge? >>Yeah. I mean, I think data products is a really interesting aspect of the answer to that. It allows you to, again, leverage the data domain owners, people know the data, the best to, to create, you know, data as a product ultimately to be consumed. And we try to represent that in our product as effectively a almost eCommerce like experience where you go and discover and look for the data products that have been created in your organization. And then you can start to consume them as, as you'd like. And so really trying to build on that notion of, you know, data democratization and self-service, and making it very easy to discover and, and start to use with whatever BI tool you, you may like, or even just running, you know, SQL queries yourself, >>Okay. G guys grab a sip of water. After this short break, we'll be back to debate whether proprietary or open platforms are the best path to the future of data excellence, keep it right there. >>Your company has more data than ever, and more people trying to understand it, but there's a problem. Your data is stored across multiple systems. It's hard to access and that delays analytics and ultimately decisions. The old method of moving all of your data into a single source of truth is slow and definitely not built for the volume of data we have today or where we are headed while your data engineers spent over half their time, moving data, your analysts and data scientists are left, waiting, feeling frustrated, unproductive, and unable to move the needle for your business. But what if you could spend less time moving or copying data? What if your data consumers could analyze all your data quickly? >>Starburst helps your teams run fast queries on any data source. We help you create a single point of access to your data, no matter where it's stored. And we support high concurrency, we solve for speed and scale, whether it's fast, SQL queries on your data lake or faster queries across multiple data sets, Starburst helps your teams run analytics anywhere you can't afford to wait for data to be available. Your team has questions that need answers. Now with Starburst, the wait is over. You'll have faster access to data with enterprise level security, easy connectivity, and 24 7 support from experts, organizations like Zolando Comcast and FINRA rely on Starburst to move their businesses forward. Contact our Trino experts to get started. >>We're back with Jess Borgman of Starburst and Richard Jarvis of EVAs health. Okay, we're gonna get to lie. Number two, and that is this an open source based platform cannot give you the performance and control that you can get with a proprietary system. Is that a lie? Justin, the enterprise data warehouse has been pretty dominant and has evolved and matured. Its stack has mature over the years. Why is it not the default platform for data? >>Yeah, well, I think that's become a lie over time. So I, I think, you know, if we go back 10 or 12 years ago with the advent of the first data lake really around Hudu, that probably was true that you couldn't get the performance that you needed to run fast, interactive, SQL queries in a data lake. Now a lot's changed in 10 or 12 years. I remember in the very early days, people would say, you you'll never get performance because you need to be column there. You need to store data in a column format. And then, you know, column formats we're introduced to, to data apes, you have Parque ORC file in aro that were created to ultimately deliver performance out of that. So, okay. We got, you know, largely over the performance hurdle, you know, more recently people will say, well, you don't have the ability to do updates and deletes like a traditional data warehouse. >>And now we've got the creation of new data formats, again like iceberg and Delta and Hodi that do allow for updates and delete. So I think the data lake has continued to mature. And I remember a, a quote from, you know, Kurt Monash many years ago where he said, you know, know it takes six or seven years to build a functional database. I think that's that's right. And now we've had almost a decade go by. So, you know, these technologies have matured to really deliver very, very close to the same level performance and functionality of, of cloud data warehouses. So I think the, the reality is that's become a line and now we have large giant hyperscale internet companies that, you know, don't have the traditional data warehouse at all. They do all of their analytics in a data lake. So I think we've, we've proven that it's very much possible today. >>Thank you for that. And so Richard, talk about your perspective as a practitioner in terms of what open brings you versus, I mean, look closed is it's open as a moving target. I remember Unix used to be open systems and so it's, it is an evolving, you know, spectrum, but, but from your perspective, what does open give you that you can't get from a proprietary system where you are fearful of in a proprietary system? >>I, I suppose for me open buys us the ability to be unsure about the future, because one thing that's always true about technology is it evolves in a, a direction, slightly different to what people expect. And what you don't want to end up is done is backed itself into a corner that then prevents it from innovating. So if you have chosen a technology and you've stored trillions of records in that technology and suddenly a new way of processing or machine learning comes out, you wanna be able to take advantage and your competitive edge might depend upon it. And so I suppose for us, we acknowledge that we don't have perfect vision of what the future might be. And so by backing open storage technologies, we can apply a number of different technologies to the processing of that data. And that gives us the ability to remain relevant, innovate on our data storage. And we have bought our way out of the, any performance concerns because we can use cloud scale infrastructure to scale up and scale down as we need. And so we don't have the concerns that we don't have enough hardware today to process what we want to do, want to achieve. We can just scale up when we need it and scale back down. So open source has really allowed us to maintain the being at the cutting edge. >>So Jess, let me play devil's advocate here a little bit, and I've talked to Shaak about this and you know, obviously her vision is there's an open source that, that the data meshes open source, an open source tooling, and it's not a proprietary, you know, you're not gonna buy a data mesh. You're gonna build it with, with open source toolings and, and vendors like you are gonna support it, but to come back to sort of today, you can get to market with a proprietary solution faster. I'm gonna make that statement. You tell me if it's a lie and then you can say, okay, we support Apache iceberg. We're gonna support open source tooling, take a company like VMware, not really in the data business, but how, the way they embraced Kubernetes and, and you know, every new open source thing that comes along, they say, we do that too. Why can't proprietary systems do that and be as effective? >>Yeah, well, I think at least with the, within the data landscape saying that you can access open data formats like iceberg or, or others is, is a bit dis disingenuous because really what you're selling to your customer is a certain degree of performance, a certain SLA, and you know, those cloud data warehouses that can reach beyond their own proprietary storage drop all the performance that they were able to provide. So it is, it reminds me kind of, of, again, going back 10 or 12 years ago when everybody had a connector to Haddo and that they thought that was the solution, right? But the reality was, you know, a connector was not the same as running workloads in Haddo back then. And I think similarly, you know, being able to connect to an external table that lives in an open data format, you know, you're, you're not going to give it the performance that your customers are accustomed to. And at the end of the day, they're always going to be predisposed. They're always going to be incentivized to get that data ingested into the data warehouse, cuz that's where they have control. And you know, the bottom line is the database industry has really been built around vendor lockin. I mean, from the start, how, how many people love Oracle today, but our customers, nonetheless, I think, you know, lockin is, is, is part of this industry. And I think that's really what we're trying to change with open data formats. >>Well, that's interesting reminded when I, you know, I see the, the gas price, the tees or gas price I, I drive up and then I say, oh, that's the cash price credit card. I gotta pay 20 cents more, but okay. But so the, the argument then, so let me, let me come back to you, Justin. So what's wrong with saying, Hey, we support open data formats, but yeah, you're gonna get better performance if you, if you keep it into our closed system, are you saying that long term that's gonna come back and bite you cuz you're gonna end up, you mentioned Oracle, you mentioned Teradata. Yeah. That's by, by implication, you're saying that's where snowflake customers are headed. >>Yeah, absolutely. I think this is a movie that, you know, we've all seen before. At least those of us who've been in the industry long enough to, to see this movie play over a couple times. So I do think that's the future. And I think, you know, I loved what Richard said. I actually wrote it down. Cause I thought it was an amazing quote. He said, it buys us the ability to be unsure of the future. Th that that pretty much says it all the, the future is unknowable and the reality is using open data formats. You remain interoperable with any technology you want to utilize. If you want to use spark to train a machine learning model and you want to use Starbust to query via sequel, that's totally cool. They can both work off the same exact, you know, data, data sets by contrast, if you're, you know, focused on a proprietary model, then you're kind of locked in again to that model. I think the same applies to data, sharing to data products, to a wide variety of, of aspects of the data landscape that a proprietary approach kind of closes you in and locks you in. >>So I, I would say this Richard, I'd love to get your thoughts on it. Cause I talked to a lot of Oracle customers, not as many te data customers, but, but a lot of Oracle customers and they, you know, they'll admit, yeah, you know, they're jamming us on price and the license cost they give, but we do get value out of it. And so my question to you, Richard, is, is do the, let's call it data warehouse systems or the proprietary systems. Are they gonna deliver a greater ROI sooner? And is that in allure of, of that customers, you know, are attracted to, or can open platforms deliver as fast in ROI? >>I think the answer to that is it can depend a bit. It depends on your businesses skillset. So we are lucky that we have a number of proprietary teams that work in databases that provide our operational data capability. And we have teams of analytics and big data experts who can work with open data sets and open data formats. And so for those different teams, they can get to an ROI more quickly with different technologies for the business though, we can't do better for our operational data stores than proprietary databases. Today we can back off very tight SLAs to them. We can demonstrate reliability from millions of hours of those databases being run at enterprise scale, but for an analytics workload where increasing our business is growing in that direction, we can't do better than open data formats with cloud-based data mesh type technologies. And so it's not a simple answer. That one will always be the right answer for our business. We definitely have times when proprietary databases provide a capability that we couldn't easily represent or replicate with open technologies. >>Yeah. Richard, stay with you. You mentioned, you know, you know, some things before that, that strike me, you know, the data brick snowflake, you know, thing is, oh, is a lot of fun for analysts like me. You've got data bricks coming at it. Richard, you mentioned you have a lot of rockstar, data engineers, data bricks coming at it from a data engineering heritage. You get snowflake coming at it from an analytics heritage. Those two worlds are, are colliding people like PJI Mohan said, you know what? I think it's actually harder to play in the data engineering. So I E it's easier to for data engineering world to go into the analytics world versus the reverse, but thinking about up and coming engineers and developers preparing for this future of data engineering and data analytics, how, how should they be thinking about the future? What, what's your advice to those young people? >>So I think I'd probably fall back on general programming skill sets. So the advice that I saw years ago was if you have open source technologies, the pythons and Javas on your CV, you commander 20% pay, hike over people who can only do proprietary programming languages. And I think that's true of data technologies as well. And from a business point of view, that makes sense. I'd rather spend the money that I save on proprietary licenses on better engineers, because they can provide more value to the business that can innovate us beyond our competitors. So I think I would my advice to people who are starting here or trying to build teams to capitalize on data assets is begin with open license, free capabilities, because they're very cheap to experiment with. And they generate a lot of interest from people who want to join you as a business. And you can make them very successful early, early doors with, with your analytics journey. >>It's interesting. Again, analysts like myself, we do a lot of TCO work and have over the last 20 plus years. And in world of Oracle, you know, normally it's the staff, that's the biggest nut in total cost of ownership, not an Oracle. It's the it's the license cost is by far the biggest component in the, in the blame pie. All right, Justin, help us close out this segment. We've been talking about this sort of data mesh open, closed snowflake data bricks. Where does Starburst sort of as this engine for the data lake data lake house, the data warehouse fit in this, in this world? >>Yeah. So our view on how the future ultimately unfolds is we think that data lakes will be a natural center of gravity for a lot of the reasons that we described open data formats, lowest total cost of ownership, because you get to choose the cheapest storage available to you. Maybe that's S3 or Azure data lake storage, or Google cloud storage, or maybe it's on-prem object storage that you bought at a, at a really good price. So ultimately storing a lot of data in a deal lake makes a lot of sense, but I think what makes our perspective unique is we still don't think you're gonna get everything there either. We think that basically centralization of all your data assets is just an impossible endeavor. And so you wanna be able to access data that lives outside of the lake as well. So we kind of think of the lake as maybe the biggest place by volume in terms of how much data you have, but to, to have comprehensive analytics and to truly understand your business and understand it holistically, you need to be able to go access other data sources as well. And so that's the role that we wanna play is to be a single point of access for our customers, provide the right level of fine grained access controls so that the right people have access to the right data and ultimately make it easy to discover and consume via, you know, the creation of data products as well. >>Great. Okay. Thanks guys. Right after this quick break, we're gonna be back to debate whether the cloud data model that we see emerging and the so-called modern data stack is really modern, or is it the same wine new bottle? When it comes to data architectures, you're watching the cube, the leader in enterprise and emerging tech coverage. >>Your data is capable of producing incredible results, but data consumers are often left in the dark without fast access to the data they need. Starers makes your data visible from wherever it lives. Your company is acquiring more data in more places, more rapidly than ever to rely solely on a data centralization strategy. Whether it's in a lake or a warehouse is unrealistic. A single source of truth approach is no longer viable, but disconnected data silos are often left untapped. We need a new approach. One that embraces distributed data. One that enables fast and secure access to any of your data from anywhere with Starburst, you'll have the fastest query engine for the data lake that allows you to connect and analyze your disparate data sources no matter where they live Starburst provides the foundational technology required for you to build towards the vision of a decentralized data mesh Starburst enterprise and Starburst galaxy offer enterprise ready, connectivity, interoperability, and security features for multiple regions, multiple clouds and everchanging global regulatory requirements. The data is yours. And with Starburst, you can perform analytics anywhere in light of your world. >>Okay. We're back with Justin Boardman. CEO of Starbust Richard Jarvis is the CTO of EMI health and Theresa tongue is the cloud first technologist from Accenture. We're on July number three. And that is the claim that today's modern data stack is actually modern. So I guess that's the lie it's it is it's is that it's not modern. Justin, what do you say? >>Yeah. I mean, I think new isn't modern, right? I think it's the, it's the new data stack. It's the cloud data stack, but that doesn't necessarily mean it's modern. I think a lot of the components actually are exactly the same as what we've had for 40 years, rather than Terra data. You have snowflake rather than Informatica you have five trend. So it's the same general stack, just, you know, a cloud version of it. And I think a lot of the challenges that it plagued us for 40 years still maintain. >>So lemme come back to you just, but okay. But, but there are differences, right? I mean, you can scale, you can throw resources at the problem. You can separate compute from storage. You really, you know, there's a lot of money being thrown at that by venture capitalists and snowflake, you mentioned it's competitors. So that's different. Is it not, is that not at least an aspect of, of modern dial it up, dial it down. So what, what do you say to that? >>Well, it, it is, it's certainly taking, you know, what the cloud offers and taking advantage of that, but it's important to note that the cloud data warehouses out there are really just separating their compute from their storage. So it's allowing them to scale up and down, but your data still stored in a proprietary format. You're still locked in. You still have to ingest the data to get it even prepared for analysis. So a lot of the same sort of structural constraints that exist with the old enterprise data warehouse model OnPrem still exist just yes, a little bit more elastic now because the cloud offers that. >>So Theresa, let me go to you cuz you have cloud first in your, in your, your title. So what's what say you to this conversation? >>Well, even the cloud providers are looking towards more of a cloud continuum, right? So the centralized cloud, as we know it, maybe data lake data warehouse in the central place, that's not even how the cloud providers are looking at it. They have news query services. Every provider has one that really expands those queries to be beyond a single location. And if we look at a lot of where our, the future goes, right, that that's gonna very much fall the same thing. There was gonna be more edge. There's gonna be more on premise because of data sovereignty, data gravity, because you're working with different parts of the business that have already made major cloud investments in different cloud providers. Right? So there's a lot of reasons why the modern, I guess, the next modern generation of the data staff needs to be much more federated. >>Okay. So Richard, how do you deal with this? You you've obviously got, you know, the technical debt, the existing infrastructure it's on the books. You don't wanna just throw it out. A lot of, lot of conversation about modernizing applications, which a lot of times is a, you know, a microservices layer on top of leg legacy apps. How do you think about the modern data stack? >>Well, I think probably the first thing to say is that the stack really has to include the processes and people around the data as well is all well and good changing the technology. But if you don't modernize how people use that technology, then you're not going to be able to, to scale because just cuz you can scale CPU and storage doesn't mean you can get more people to use your data, to generate you more, more value for the business. And so what we've been looking at is really changing in very much aligned to data products and, and data mesh. How do you enable more people to consume the service and have the stack respond in a way that keeps costs low? Because that's important for our customers consuming this data, but also allows people to occasionally run enormous queries and then tick along with smaller ones when required. And it's a good job we did because during COVID all of a sudden we had enormous pressures on our data platform to answer really important life threatening queries. And if we couldn't scale both our data stack and our teams, we wouldn't have been able to answer those as quickly as we had. So I think the stack needs to support a scalable business, not just the technology itself. >>Well thank you for that. So Justin let's, let's try to break down what the critical aspects are of the modern data stack. So you think about the past, you know, five, seven years cloud obviously has given a different pricing model. De-risked experimentation, you know that we talked about the ability to scale up scale down, but it's, I'm, I'm taking away that that's not enough based on what Richard just said. The modern data stack has to serve the business and enable the business to build data products. I, I buy that. I'm a big fan of the data mesh concepts, even though we're early days. So what are the critical aspects if you had to think about, you know, paying, maybe putting some guardrails and definitions around the modern data stack, what does that look like? What are some of the attributes and, and principles there >>Of, of how it should look like or, or how >>It's yeah. What it should be. >>Yeah. Yeah. Well, I think, you know, in, in Theresa mentioned this in, in a previous segment about the data warehouse is not necessarily going to disappear. It just becomes one node, one element of the overall data mesh. And I, I certainly agree with that. So by no means, are we suggesting that, you know, snowflake or Redshift or whatever cloud data warehouse you may be using is going to disappear, but it's, it's not going to become the end all be all. It's not the, the central single source of truth. And I think that's the paradigm shift that needs to occur. And I think it's also worth noting that those who were the early adopters of the modern data stack were primarily digital, native born in the cloud young companies who had the benefit of, of idealism. They had the benefit of it was starting with a clean slate that does not reflect the vast majority of enterprises. >>And even those companies, as they grow up mature out of that ideal state, they go buy a business. Now they've got something on another cloud provider that has a different data stack and they have to deal with that heterogeneity that is just change and change is a part of life. And so I think there is an element here that is almost philosophical. It's like, do you believe in an absolute ideal where I can just fit everything into one place or do I believe in reality? And I think the far more pragmatic approach is really what data mesh represents. So to answer your question directly, I think it's adding, you know, the ability to access data that lives outside of the data warehouse, maybe living in open data formats in a data lake or accessing operational systems as well. Maybe you want to directly access data that lives in an Oracle database or a Mongo database or, or what have you. So creating that flexibility to really Futureproof yourself from the inevitable change that you will, you won't encounter over time. >>So thank you. So there, based on what Justin just said, I, my takeaway there is it's inclusive, whether it's a data Mar data hub, data lake data warehouse, it's a, just a node on the mesh. Okay. I get that. Does that include there on Preem data? O obviously it has to, what are you seeing in terms of the ability to, to take that data mesh concept on Preem? I mean, most implementations I've seen in data mesh, frankly really aren't, you know, adhering to the philosophy. They're maybe, maybe it's data lake and maybe it's using glue. You look at what JPMC is doing. Hello, fresh, a lot of stuff happening on the AWS cloud in that, you know, closed stack, if you will. What's the answer to that Theresa? >>I mean, I, I think it's a killer case for data. Me, the fact that you have valuable data sources, OnPrem, and then yet you still wanna modernize and take the best of cloud cloud is still, like we mentioned, there's a lot of great reasons for it around the economics and the way ability to tap into the innovation that the cloud providers are giving around data and AI architecture. It's an easy button. So the mesh allows you to have the best of both worlds. You can start using the data products on-prem or in the existing systems that are working already. It's meaningful for the business. At the same time, you can modernize the ones that make business sense because it needs better performance. It needs, you know, something that is, is cheaper or, or maybe just tap into better analytics to get better insights, right? So you're gonna be able to stretch and really have the best of both worlds. That, again, going back to Richard's point, that is meaningful by the business. Not everything has to have that one size fits all set a tool. >>Okay. Thank you. So Richard, you know, talking about data as product, wonder if we could give us your perspectives here, what are the advantages of treating data as a product? What, what role do data products have in the modern data stack? We talk about monetizing data. What are your thoughts on data products? >>So for us, one of the most important data products that we've been creating is taking data that is healthcare data across a wide variety of different settings. So information about patients' demographics about their, their treatment, about their medications and so on, and taking that into a standards format that can be utilized by a wide variety of different researchers because misinterpreting that data or having the data not presented in the way that the user is expecting means that you generate the wrong insight. And in any business, that's clearly not a desirable outcome, but when that insight is so critical, as it might be in healthcare or some security settings, you really have to have gone to the trouble of understanding the data, presenting it in a format that everyone can clearly agree on. And then letting people consume in a very structured, managed way, even if that data comes from a variety of different sources in, in, in the first place. And so our data product journey has really begun by standardizing data across a number of different silos through the data mesh. So we can present out both internally and through the right governance externally to, to researchers. >>So that data product through whatever APIs is, is accessible, it's discoverable, but it's obviously gotta be governed as well. You mentioned you, you appropriately provided to internally. Yeah. But also, you know, external folks as well. So the, so you've, you've architected that capability today >>We have, and because the data is standard, it can generate value much more quickly and we can be sure of the security and, and, and value that that's providing because the data product isn't just about formatting the data into the correct tables, it's understanding what it means to redact the data or to remove certain rows from it or to interpret what a date actually means. Is it the start of the contract or the start of the treatment or the date of birth of a patient? These things can be lost in the data storage without having the proper product management around the data to say in a very clear business context, what does this data mean? And what does it mean to process this data for a particular use case? >>Yeah, it makes sense. It's got the context. If the, if the domains own the data, you, you gotta cut through a lot of the, the, the centralized teams, the technical teams that, that data agnostic, they don't really have that context. All right. Let's send Justin, how does Starburst fit into this modern data stack? Bring us home. >>Yeah. So I think for us, it's really providing our customers with, you know, the flexibility to operate and analyze data that lives in a wide variety of different systems. Ultimately giving them that optionality, you know, and optionality provides the ability to reduce costs, store more in a data lake rather than data warehouse. It provides the ability for the fastest time to insight to access the data directly where it lives. And ultimately with this concept of data products that we've now, you know, incorporated into our offering as well, you can really create and, and curate, you know, data as a product to be shared and consumed. So we're trying to help enable the data mesh, you know, model and make that an appropriate compliment to, you know, the, the, the modern data stack that people have today. >>Excellent. Hey, I wanna thank Justin Theresa and Richard for joining us today. You guys are great. I big believers in the, in the data mesh concept, and I think, you know, we're seeing the future of data architecture. So thank you. Now, remember, all these conversations are gonna be available on the cube.net for on-demand viewing. You can also go to starburst.io. They have some great content on the website and they host some really thought provoking interviews and, and, and they have awesome resources, lots of data mesh conversations over there, and really good stuff in, in the resource section. So check that out. Thanks for watching the data doesn't lie or does it made possible by Starburst data? This is Dave Valante for the cube, and we'll see you next time. >>The explosion of data sources has forced organizations to modernize their systems and architecture and come to terms with one size does not fit all for data management today. Your teams are constantly moving and copying data, which requires time management. And in some cases, double paying for compute resources. Instead, what if you could access all your data anywhere using the BI tools and SQL skills your users already have. And what if this also included enterprise security and fast performance with Starburst enterprise, you can provide your data consumers with a single point of secure access to all of your data, no matter where it lives with features like strict, fine grained, access control, end to end data encryption and data masking Starburst meets the security standards of the largest companies. Starburst enterprise can easily be deployed anywhere and managed with insights where data teams holistically view their clusters operation and query execution. So they can reach meaningful business decisions faster, all this with the support of the largest team of Trino experts in the world, delivering fully tested stable releases and available to support you 24 7 to unlock the value in all of your data. You need a solution that easily fits with what you have today and can adapt to your architecture. Tomorrow. Starbust enterprise gives you the fastest path from big data to better decisions, cuz your team can't afford to wait. Trino was created to empower analytics anywhere and Starburst enterprise was created to give you the enterprise grade performance, connectivity, security management, and support your company needs organizations like Zolando Comcast and FINRA rely on Starburst to move their businesses forward. Contact us to get started.

>>In 2011, early Facebook employee and Cloudera co-founder Jeff Ocker famously said the best minds of my generation are thinking about how to get people to click on ads. And that sucks. Let's face it more than a decade later organizations continue to be frustrated with how difficult it is to get value from data and build a truly agile data driven enterprise. What does that even mean? You ask? Well, it means that everyone in the organization has the data they need when they need it. In a context that's relevant to advance the mission of an organization. Now that could mean cutting costs could mean increasing profits, driving productivity, saving lives, accelerating drug discovery, making better diagnoses, solving, supply chain problems, predicting weather disasters, simplifying processes, and thousands of other examples where data can completely transform people's lives beyond manipulating internet users to behave a certain way. We've heard the prognostications about the possibilities of data before and in fairness we've made progress, but the hard truth is the original promises of master data management, enterprise data, warehouses, data, Mars, data hubs, and yes, even data lakes were broken and left us wanting for more welcome to the data doesn't lie, or does it a series of conversations produced by the cube and made possible by Starburst data. >>I'm your host, Dave Lanta and joining me today are three industry experts. Justin Borgman is this co-founder and CEO of Starburst. Richard Jarvis is the CTO at EMI health and Theresa tongue is cloud first technologist at Accenture. Today we're gonna have a candid discussion that will expose the unfulfilled and yes, broken promises of a data past we'll expose data lies, big lies, little lies, white lies, and hidden truths. And we'll challenge, age old data conventions and bust some data myths. We're debating questions like is the demise of a single source of truth. Inevitable will the data warehouse ever have feature parody with the data lake or vice versa is the so-called modern data stack simply centralization in the cloud, AKA the old guards model in new cloud close. How can organizations rethink their data architectures and regimes to realize the true promises of data can and will and open ecosystem deliver on these promises in our lifetimes, we're spanning much of the Western world today. Richard is in the UK. Teresa is on the west coast and Justin is in Massachusetts with me. I'm in the cube studios about 30 miles outside of Boston folks. Welcome to the program. Thanks for coming on. Thanks for having us. Let's get right into it. You're very welcome. Now here's the first lie. The most effective data architecture is one that is centralized with a team of data specialists serving various lines of business. What do you think Justin? >>Yeah, definitely a lie. My first startup was a company called hit adapt, which was an early SQL engine for IDU that was acquired by Teradata. And when I got to Teradata, of course, Terada is the pioneer of that central enterprise data warehouse model. One of the things that I found fascinating was that not one of their customers had actually lived up to that vision of centralizing all of their data into one place. They all had data silos. They all had data in different systems. They had data on-prem data in the cloud. You know, those companies were acquiring other companies and inheriting their data architecture. So, you know, despite being the industry leader for 40 years, not one of their customers truly had everything in one place. So I think definitely history has proven that to be a lie. >>So Richard, from a practitioner's point of view, you know, what, what are your thoughts? I mean, there, there's a lot of pressure to cut cost, keep things centralized, you know, serve the business as best as possible from that standpoint. What, what is your experience, Joe? >>Yeah, I mean, I think I would echo Justin's experience really that we, as a business have grown up through acquisition, through storing data in different places sometimes to do information governance in different ways to store data in, in a platform that's close to data experts, people who really understand healthcare data from pharmacies or from, from doctors. And so, although if you were starting from a Greenfield site and you were building something brand new, you might be able to centralize all the data and all of the tooling and teams in one place. The reality is that that businesses just don't grow up like that. And, and it's just really impossible to get that academic perfection of, of storing everything in one place. >>Y you know, Theresa, I feel like Sarbanes Oxley kinda saved the data warehouse, you know? Right. But you actually did have to have a single version of the truth for certain financial data, but really for those, some of those other use cases, I, I mentioned, I, I do feel like the industry has kinda let us down. What's your take on this? Where does it make sense to have that sort of centralized approach versus where does it make sense to maybe decentralized? >>I, I think you gotta have centralized governance, right? So from the central team, for things like swans Oxley, for things like security, for certain very core data sets, having a centralized set of roles, responsibilities to really QA, right. To serve as a design authority for your entire data estate, just like you might with security, but how it's implemented has to be distributed. Otherwise you're not gonna be able to scale. Right? So being able to have different parts of the business really make the right data investments for their needs. And then ultimately you're gonna collaborate with your partners. So partners that are not within the company, right. External partners, we're gonna see a lot more data sharing and model creation. And so you're definitely going to be decentralized. >>So, you know, Justin, you guys last, geez, I think it was about a year ago, had a session on, on data mesh. It was a great program. You invited JAK, Dani, of course, she's the creator of the data mesh. And her one of our fundamental premises is that you've got this hyper specialized team that you've gotta go through. And if you want anything, but at the same time, these, these individuals actually become a bottleneck, even though they're some of the most talented people in the organization. So I guess question for you, Richard, how do you deal with that? Do you, do you organize so that there are a few sort of rock stars that, that, you know, build cubes and, and the like, and, and, and, or have you had any success in sort of decentralizing with, you know, your, your constituencies, that data model? >>Yeah. So, so we absolutely have got rockstar, data scientists and data guardians. If you like people who understand what it means to use this data, particularly as the data that we use at emos is very private it's healthcare information. And some of the, the rules and regulations around using the data are very complex and, and strict. So we have to have people who understand the usage of the data, then people who understand how to build models, how to process the data effectively. And you can think of them like consultants to the wider business, because a pharmacist might not understand how to structure a SQL query, but they do understand how they want to process medication information to improve patient lives. And so that becomes a, a consulting type experience from a, a set of rock stars to help a, a more decentralized business who needs to, to understand the data and to generate some valuable output. >>Justin, what do you say to a, to a customer or prospect that says, look, Justin, I'm gonna, I got a centralized team and that's the most cost effective way to serve the business. Otherwise I got, I got duplication. What do you say to that? >>Well, I, I would argue it's probably not the most cost effective and, and the reason being really twofold. I think, first of all, when you are deploying a enterprise data warehouse model, the, the data warehouse itself is very expensive, generally speaking. And so you're putting all of your most valuable data in the hands of one vendor who now has tremendous leverage over you, you know, for many, many years to come, I think that's the story of Oracle or Terra data or other proprietary database systems. But the other aspect I think is that the reality is those central data warehouse teams is as much as they are experts in the technology. They don't necessarily understand the data itself. And this is one of the core tenets of data mash that that jam writes about is this idea of the domain owners actually know the data the best. >>And so by, you know, not only acknowledging that data is generally decentralized and to your earlier point about, so Oxley, maybe saving the data warehouse, I would argue maybe GDPR and data sovereignty will destroy it because data has to be decentralized for, for those laws to be compliant. But I think the reality is, you know, the data mesh model basically says, data's decentralized, and we're gonna turn that into an asset rather than a liability. And we're gonna turn that into an asset by empowering the people that know the data, the best to participate in the process of, you know, curating and creating data products for, for consumption. So I think when you think about it, that way, you're going to get higher quality data and faster time to insight, which is ultimately going to drive more revenue for your business and reduce costs. So I think that that's the way I see the two, the two models comparing and con contrasting. >>So do you think the demise of the data warehouse is inevitable? I mean, I mean, you know, there Theresa you work with a lot of clients, they're not just gonna rip and replace their existing infrastructure. Maybe they're gonna build on top of it, but the, what does that mean? Does that mean the ed w just becomes, you know, less and less valuable over time, or it's maybe just isolated to specific use cases. What's your take on that? >>Listen, I still would love all my data within a data warehouse would love it. Mastered would love it owned by essential team. Right? I think that's still what I would love to have. That's just not the reality, right? The investment to actually migrate and keep that up to date. I would say it's a losing battle. Like we've been trying to do it for a long time. Nobody has the budgets and then data changes, right? There's gonna be a new technology. That's gonna emerge that we're gonna wanna tap into. There's gonna be not enough investment to bring all the legacy, but still very useful systems into that centralized view. So you keep the data warehouse. I think it's a very, very valuable, very high performance tool for what it's there for, but you could have this, you know, new mesh layer that still takes advantage of the things. I mentioned, the data products in the systems that are meaningful today and the data products that actually might span a number of systems. Maybe either those that either source systems, the domains that know it best, or the consumer based systems and products that need to be packaged in a way that be really meaningful for that end user, right? Each of those are useful for a different part of the business and making sure that the mesh actually allows you to lose all of them. >>So, Richard, let me ask you, you take, take Gemma's principles back to those. You got, you know, the domain ownership and, and, and data as product. Okay, great. Sounds good. But it creates what I would argue or two, you know, challenges self-serve infrastructure let's park that for a second. And then in your industry, one of the high, most regulated, most sensitive computational governance, how do you automate and ensure federated governance in that mesh model that Theresa was just talking about? >>Well, it absolutely depends on some of the tooling and processes that you put in place around those tools to be, to centralize the security and the governance of the data. And, and I think, although a data warehouse makes that very simple, cause it's a single tool, it's not impossible with some of the data mesh technologies that are available. And so what we've done at EMI is we have a single security layer that sits on top of our data mesh, which means that no matter which user is accessing, which data source, we go through a well audited well understood security layer. That means that we know exactly who's got access to which data field, which data tables. And then everything that they do is, is audited in a very kind of standard way, regardless of the underlying data storage technology. So for me, although storing the data in one place might not be possible understanding where your source of truth is and securing that in a common way is still a valuable approach and you can do it without having to bring all that data into a single bucket so that it's all in one place. >>And, and so having done that and investing quite heavily in making that possible has paid dividends in terms of giving wider access to the platform and ensuring that only data that's available under GDPR and other regulations is being used by, by the data users. >>Yeah. So Justin mean Democrat, we always talk about data democratization and you know, up until recently, they really haven't been line of sight as to how to get there. But do you have anything to add to this because you're essentially taking, you know, doing analytic queries and with data, that's all dispersed all over the, how are you seeing your customers handle this, this challenge? >>Yeah, I mean, I think data products is a really interesting aspect of the answer to that. It allows you to, again, leverage the data domain owners, people know the data, the best to, to create, you know, data as a product ultimately to be consumed. And we try to represent that in our product as effectively, almost eCommerce, like experience where you go and discover and look for the data products that have been created in your organization. And then you can start to consume them as, as you'd like. And so really trying to build on that notion of, you know, data democratization and self-service, and making it very easy to discover and, and start to use with whatever BI tool you, you may like, or even just running, you know, SQL queries yourself. >>Okay. G guys grab a sip of water. After the short break, we'll be back to debate whether proprietary or open platforms are the best path to the future of data excellence. Keep it right there.

(upbeat music) >> Welcome back, everyone. It's theCUBE's coverage here in Las Vegas. I'm John Furrier, your host of theCUBE with re:MARS. Amazon re:MARS stands for machine learning, automation, robotics, and space. Lot of great content, accomplishment. AI meets meets robotics and space, industrial IoT, all things data. And we've got two great guests here to unpack the AI side of it. Manoj Suvarna, Managing Director at AI Ecosystem at Deloitte and Arte Merritt, Conversational AI Lead at AWS. Manoj, it's great to see you CUBE alumni. Art, welcome to theCUBE. >> Thanks for having me. I appreciate it. >> So AI's the big theme. Actually, the big disconnect in the industry has been the industrial OT versus IT, and that's happening. Now you've got space and robotics meets what we know is machine learning and AI which we've been covering. This is the confluence of the new IoT market. >> It absolutely is. >> What's your opinion on that? >> Yeah, so actually it's taking IoT beyond the art of possible. One area that we have been working very closely with AWS. We're strategic alliance with them. And for the past six years, we have been investing a lot in transformations. Transformation as it relate to the cloud, transformation as it relate to data modernization. The new edge is essentially on AI and machine learning. And just this week, we announced a new solution which is more focused around enhancing contact center intelligence. So think about the edge of the contact center, where we all have experiences around dealing with customer service and how to really take that to the next level, challenges that clients are facing in every part of that business. So clearly. >> Well, Conversational AI is a good topic. Talk about the relationship with Deloitte and Amazon for a second around AI because you guys have some great projects going on right now. That's well ahead of the curve on solving the scale problem 'cause there's a scale and problem, practical problem and then scale. What's the relationship with Amazon and Deloitte? >> We have a great alliance and relationship. Deloitte brings that expertise to help folks build high quality, highly effective conversational AI and enterprises are implementing these solutions to really try to improve the overall customer experience. So they want to help agents improve productivity, gain insights into the reasons why folks are calling but it's really to provide that better user experience being available 24/7 on channels users prefer to interact. And the solutions that Deloitte is building are highly advanced, super exciting. Like when we show demos of them to potential customers, the eyes light up and they want those solutions. >> John: Give an example when their eyes light up. What are you showing there? >> One solution, it's called multimodal interfaces. So what this is, is when you're call into like a voice IVR, Deloitte's solution will send the folks say a mobile app or a website. So the person can interact with both the phone touching on the screen and the voice and it's all kept in sync. So imagine you call the doctor's office or say I was calling a airline and I want to change my flight or sorry, change the seat. If they were to say, seat 20D is available. Well, I don't know what that means, but if you see the map while you're talking, you can say, oh, 20D is the aisle. I'm going to select that. So Deloitte's doing those kind of experiences. It's incredible. >> Manoj, this is where the magic comes into play when you bring data together and you have integration like this. Asynchronously or synchronously, it's all coming together. You have different platforms, phone, voice, silo databases potentially, the old way. Now, the new ways integrating. What makes it all work? What's the key to success? >> Yeah, it's certainly not a trivial feat. Bringing together all of these ecosystems of relationships, technologies all put together. We cannot do it alone. This is where we partner with AWS with some of our other partners like Salesforce and OneReach and really trying to bring a symphony of some of these solutions to bear. When you think about, going back to the example of contact center, the challenges that the pandemic posed in the last couple of years was the fact that who's a humongous rise in volume of number of calls. You can imagine people calling in asking for all kinds of different things, whether it's airlines whether it is doctor's office and retail. And then couple with that is the fact that there's the labor shortage. And how do you train agents to get them to be productive enough to be able to address hundreds or thousands of these calls? And so that's where we have been starting to, we have invested in those solutions bringing those technologies together to address real client problems, not just slideware but actual production environments. And that's where we launched this solution called TrueServe as of this week, which is really a multimodal solution that is built with preconceived notions of technologies and libraries where we can then be industry agnostic and be able to deliver those experiences to our clients based on whatever vertical or industry they're in. >> Take me through the client's engagement here because I can imagine they want to get a practical solution. They're going to want to have it up and running, not like a just a chatbot, but like they completely integrated system. What's the challenge and what's the outcome first set of milestones that you see that they do first? Do they just get the data together? Are they deploying a software solution? What's the use cases? >> There's a couple different use cases. We see there's the self-service component that we're talking about with the chatbots or voice IVR solutions. There's also use cases for helping the agents, so real-time agent assist. So you call into a contact center, it's transcribed in real time, run through some sort of knowledge base to give the agents possible answers to help the user out, tying in, say the Salesforce data, CRM data, to know more about the user. Like if I was to call the airline, it's going to say, "Are you calling about your flight to San Francisco tomorrow?" It knows who I am. It leverages that stuff. And then the key piece is the analytics knowing why folks are calling, not just your metrics around, length of calls or deflections, but what were the reasons people were calling in because you can use that data to improve your underlying products or services. These are the things that enterprise are looking for and this is where someone like Deloitte comes in, brings that expertise, speeds up the time to market and really helps the customers. >> Manoj, what was the solution you mentioned that you guys announced? >> Yeah, so this is called Deloitte TrueServe. And essentially, it's a combination of multiple different solutions combinations from AWS, from Salesforce, from OneReach. All put together with our joint engineering and really delivering that capability. Enhancing on that is the analytics component, which is really critical, especially because when you think about the average contact center, less than 10% of the data gets analyzed today, and how do you then extract value out of that data and be able to deliver business outcomes. >> I was just talking to some of the other day about Zoom. Everyone records their zoom meetings, and no one watches them. I mean, who's going to wade through that. Call center is even more high volume. We're talking about massive data. And so will you guys automate that? Do you go through every single piece of data, every call and bring it down? Is that how it works? >> Go ahead. >> There's just some of the things you can do. Analyze the calls for common themes, like figuring out like topic modeling, what are the reasons people are calling in. Summarizing that stuff so you can see what those underlying issues are. And so that could be, like I was mentioning, improving the product or service. It could also be for helping train the agents. So here's how to answer that question. And it could even be reinforcing positive experiences maybe an agent had a particular great call and that could be a reference for other folks. >> Yeah, and also during the conversation, when you think about within 60 to 90 seconds, how do you identify the intonation, the sentiments of the client customer calling in and be able to respond in real time for the challenges that they might be facing and the ability to authenticate the customer at the same time be able to respond to them. I think that is the advancements that we are seeing in the market. >> I think also your point about the data having residual values also excellent because this is a long tail of value in this data, like for predictions and stuff. So NASA was just on before you guys came on, talking about the Artemis project and all the missions and they have to run massive amounts of simulations. And this is where I've kind of seen the dots connect here. You can run with AI, run all the heavy lifting without human touching it to get that first ingestion or analysis, and then iterating on the data based upon what else happens. >> Manoj: Absolutely. >> This is now the new normal, right? Is this? >> It is. And it's transverse towards across multiple domains. So the example we gave you was around Conversational AI. We're now looking at that for doing predictive analytics. Those are some examples that we are doing jointly with AWS SageMaker. We are working on things like computer vision with some of the capabilities and what computer vision has to offer. And so when you think about the continuum of possibilities of what we can bring together from a tools, technology, services perspective, really the sky is the limit in terms of delivering these real experiences to our clients. >> So take me through a customer. Pretending I'm a customer, I get it. I got to do this. It's a competitive advantage. What are the outcomes that they are envisioning? What are some of the patterns you're seeing with customers? What outcomes are they expecting and what kind of high level upside you see them envisioning coming out of the data? >> So when you think about the CxOs today and the board, a lot of them are thinking about, okay, how do you build more efficiency in those system? How do you enable a technology or solution for them to not only increase their top line but as well as their bottom line? How do you enhance the customer experience, which in this case is spot on because when you think about, when customers go repeat to a vendor, it's based on quality, it's based on price. Customer experience is now topping that where your first experience, whether it's through a chat or a virtual assistant or a phone call is going to determine the longevity of that customer with you as a vendor. And so clearly, when you think about how clients are becoming AI fuel, this is where we are bringing in new technologies, new solutions to really push the art to the limit and the art of possible. >> You got a playbook too to do this? >> Yeah, yeah, absolutely. We have done that. And in fact, we are now taking that to the next level up. So something that I've mentioned about this before, which is how do you trust an AI system as it's building up. >> Hold on, I need to plug in. >> Yeah, absolutely. >> I put this here for a reason to remind me. No, but also trust is a big thing. Just put that trustworthy. This is an AI ethics question. >> Arte: It's a big. >> Let's get into it. This is huge. Data's data. Data can be biased from coming in >> Part of it, there are concerns you have to look at the bias in the data. It's also how you communicate through these automated channels, being empathetic, building trust with the customer, being concise in the answers and being accessible to all sorts of different folks and how they might communicate. So it's definitely a big area. >> I mean, you think about just normal life. We all lived situations where we got a text message from a friend or someone close to us where, what the hell, what are you saying? And they had no contextual bad feelings about it or, well, there's misunderstandings 'cause the context isn't there 'cause you're rapid fire them on the subway. I'm riding my bike. I stop and text, okay, I'm okay. Church response could mean I'm busy or I'm angry. Like this is now what you said about empathy. This is now a new dynamic in here. >> Oh, the empathy is huge, especially if you're say a financial institution or building that trust with folks and being empathetic. If someone's reaching out to a contact center, there's a good chance they're upset about something. So you have to take that. >> John: Calm them down first. >> Yeah, and not being like false like platitude kind of things, like really being empathetic, being inclusive in the language. Those are things that you have conversation designers and linguistics folks that really look into that. That's why having domain expertise from folks like Deloitte come in to help with that. 'Cause maybe if you're just building the chat on your own, you might not think of those things. But the folks with the domain expertise will say like, Hey, this is how you script it. It's the power of words, getting that message across clearly. >> The linguistics matter? >> Yeah, yeah. >> It does. >> By vertical too, I mean, you could pick any the tribe, whatever orientation and age, demographics, genders. >> All of those things that we take for granted as a human. When you think about trust, when you think about bias, when you think about ethics, it just gets amplified. Because now you're dealing with millions and millions of data points that may or may not be the right direction in terms of somebody's calling in depending on what age group they're in. Some questions might not be relevant for that age group. Now a human can determine that, but a bot cannot. And so how do you make sure that when you look at this data coming in, how do you build models that are ethically aware of the contextual algorithms and the alignment with it and also enabling that experience to be much enhanced than taking it backwards, and that's really. >> I can imagine it getting better with as people get scaled up a bit 'cause then you're going to have to start having AI to watch the AI at some point, as they say. Where are we in the progress in the industry right now? Because I know there's been a lot of news stories around, ethics and AI and bias and it's a moving train actually, but still problems are going to be solved. Are we at the tipping point yet? Are we still walking in before we crawl or crawling before we walk? I should say, I mean, where are we? >> I think we are in between a crawling or walk phase. And the reason for that is because it varies depending on whether you're regulated industry or unregulated. In the regulated industry, there are compliance regulations requirements, whether it's government whether it's banking, financial institutions where they have to meet Sarbanes-Oxley and all kinds of compliance requirements, whereas an unregulated industry like retail and consumer, it is anybody's gain. And so the reality of it is that there is more of an awareness now. And that's one of the reasons why we've been promoting this jointly with AWS. We have a framework that we have established where there are multiple pillars of trust, bias, privacy, and security that companies and organizations need to think about. Our data scientists, ML engineers need to be familiar with it, but because while they're super great in terms of model building and development, when it comes to the business, when it comes to the client or a customer, it is super important for them to trust this platform, this algorithm. And that is where we are trying to build that momentum, bring that awareness. One of my colleagues has written this book "Trustworthy AI". We're trying to take the message out to the market to say, there is a framework. We can help you get there. And certainly that's what we are doing. >> Just call Deloitte up and you're going to take care of them. >> Manoj: Yeah. >> On the Amazon side, Amazon Web Services. I always interview Swami every year at re:Invent and he always get the updates. He's been bullish on this for a long time on this Conversational AI. What's the update on the AWS side? Where are you guys at? What's the current trends that you're riding? What wave are you riding right now? >> So some of the trends we see in customer interest, there's a couple of things. One is the multimodal interfaces we we're just chatting about where the voice IVA is synced with like a web or mobile experience, so you take that full advantage of the device. The other is adding additional AI into the Conversational AI. So one example is a customer that included intelligent document processing as part of the chatbot. So instead of typing your name and address, take a photo of your driver's license. It was an insurance onboarding chatbot, so you could take a photo of your existing insurance policy. It'll extract that information to build the new insurance policy. So folks get excited about that. And the third area we see interest is what's called multi-bot orchestration. And this is where you can have one main chatbot. Marshall user across different sub-chatbots based on the use case persona or even language. So those things get people really excited and then AWS is launching all sorts of new features. I don't know which one is coming out. >> I know something's coming out tomorrow. He's right at corner. He's big smile on his face. He wouldn't tell me. It's good. >> We have for folks like the closer alliance relationships, we we're able to get previews. So there a preview of all the new stuff. And I don't know what I could, it's pretty exciting stuff. >> You get in trouble if you spill the beans here. Don't, be careful. I'll watch you. We'll talk off camera. All exciting stuff. >> Yeah, yeah. I think the orchestrator bot is interesting. Having the ability to orchestrate across different contextual datasets is interesting. >> One of the areas where it's particularly interesting is in financial services. Imagine a bank could have consumer accounts, merchant accounts, investment banking accounts. So if you were to chat with the chatbot and say I want to open account, well, which account do you mean? And so it's able to figure out that context to navigate folks to those sub-chatbots behind the scenes. And so it's pretty interesting style. >> Awesome. Manoj while we're here, take a minute to quickly give a plug for Deloitte. What your program's about? What customers should expect if they work with you guys on this project? Give a quick commercial for Deloitte. >> Yeah, no, absolutely. I mean, Deloitte has been continuing to lead the AI field organization effort across our client base. If you think about all the Fortune 100, Fortune 500, Fortune 2000 clients, we certainly have them where they are in advanced stages of multiple deployments for AI. And we look at it all the way from strategy to implementation to operational models. So clients don't have to do it alone. And we are continuing to build our ecosystem of relationships, partnerships like the alliances that we have with AWS, building the ecosystem of relationships with other emerging startups, to your point about how do you continue to innovate and bring those technologies to your clients in a trustworthy environment so that we can deliver it in production scale. That is essentially what we're driving. >> Well, Arte, there's a great conversation and the AI will take over from here as we end the segment. I see a a bot coming on theCUBE later and there might be CUBE be replaced with robots. >> Right, right, right, exactly. >> I'm John Furrier, calling from Palo Alto. >> Someday, CUBE bot. >> You can just say, Alexa do my demo for me or whatever it is. >> Or digital twin for John. >> We're going to have a robot on earlier do a CUBE interview and that's Dave Vellante. He'd just pipe his voice in and be fun. Well, thanks for coming on, great conversation. >> Thank you. Thanks for having us. >> CUBE coverage here at re:MARS in Las Vegas. Back to the event circle. We're back in the line. Got re:Inforce and don't forget re:Invent at the end of the year. CUBE coverage of this exciting show here. Machine learning, automation, robotics, space. That's MARS, it's re:MARS. I'm John Furrier. Thanks for watching. (gentle music)

(bright music) >> In the 2010s, organizations became keenly aware that data would become the key ingredient to driving competitive advantage, differentiation, and growth. But to this day, putting data to work remains a difficult challenge for many, if not most organizations. Now, as the cloud matures, it has become a game changer for data practitioners by making cheap storage and massive processing power readily accessible. We've also seen better tooling in the form of data workflows, streaming, machine intelligence, AI, developer tools, security, observability, automation, new databases and the like. These innovations they accelerate data proficiency, but at the same time, they add complexity for practitioners. Data lakes, data hubs, data warehouses, data marts, data fabrics, data meshes, data catalogs, data oceans are forming, they're evolving and exploding onto the scene. So in an effort to bring perspective to the sea of optionality, we've brought together the brightest minds in the data analyst community to discuss how data management is morphing and what practitioners should expect in 2022 and beyond. Hello everyone, my name is Dave Velannte with theCUBE, and I'd like to welcome you to a special Cube presentation, analysts predictions 2022: the future of data management. We've gathered six of the best analysts in data and data management who are going to present and discuss their top predictions and trends for 2022 in the first half of this decade. Let me introduce our six power panelists. Sanjeev Mohan is former Gartner Analyst and Principal at SanjMo. Tony Baer, principal at dbInsight, Carl Olofson is well-known Research Vice President with IDC, Dave Menninger is Senior Vice President and Research Director at Ventana Research, Brad Shimmin, Chief Analyst, AI Platforms, Analytics and Data Management at Omdia and Doug Henschen, Vice President and Principal Analyst at Constellation Research. Gentlemen, welcome to the program and thanks for coming on theCUBE today. >> Great to be here. >> Thank you. >> All right, here's the format we're going to use. I as moderator, I'm going to call on each analyst separately who then will deliver their prediction or mega trend, and then in the interest of time management and pace, two analysts will have the opportunity to comment. If we have more time, we'll elongate it, but let's get started right away. Sanjeev Mohan, please kick it off. You want to talk about governance, go ahead sir. >> Thank you Dave. I believe that data governance which we've been talking about for many years is now not only going to be mainstream, it's going to be table stakes. And all the things that you mentioned, you know, the data, ocean data lake, lake houses, data fabric, meshes, the common glue is metadata. If we don't understand what data we have and we are governing it, there is no way we can manage it. So we saw Informatica went public last year after a hiatus of six. I'm predicting that this year we see some more companies go public. My bet is on Culebra, most likely and maybe Alation we'll see go public this year. I'm also predicting that the scope of data governance is going to expand beyond just data. It's not just data and reports. We are going to see more transformations like spark jawsxxxxx, Python even Air Flow. We're going to see more of a streaming data. So from Kafka Schema Registry, for example. We will see AI models become part of this whole governance suite. So the governance suite is going to be very comprehensive, very detailed lineage, impact analysis, and then even expand into data quality. We already seen that happen with some of the tools where they are buying these smaller companies and bringing in data quality monitoring and integrating it with metadata management, data catalogs, also data access governance. So what we are going to see is that once the data governance platforms become the key entry point into these modern architectures, I'm predicting that the usage, the number of users of a data catalog is going to exceed that of a BI tool. That will take time and we already seen that trajectory. Right now if you look at BI tools, I would say there a hundred users to BI tool to one data catalog. And I see that evening out over a period of time and at some point data catalogs will really become the main way for us to access data. Data catalog will help us visualize data, but if we want to do more in-depth analysis, it'll be the jumping off point into the BI tool, the data science tool and that is the journey I see for the data governance products. >> Excellent, thank you. Some comments. Maybe Doug, a lot of things to weigh in on there, maybe you can comment. >> Yeah, Sanjeev I think you're spot on, a lot of the trends the one disagreement, I think it's really still far from mainstream. As you say, we've been talking about this for years, it's like God, motherhood, apple pie, everyone agrees it's important, but too few organizations are really practicing good governance because it's hard and because the incentives have been lacking. I think one thing that deserves mention in this context is ESG mandates and guidelines, these are environmental, social and governance, regs and guidelines. We've seen the environmental regs and guidelines and posts in industries, particularly the carbon-intensive industries. We've seen the social mandates, particularly diversity imposed on suppliers by companies that are leading on this topic. We've seen governance guidelines now being imposed by banks on investors. So these ESGs are presenting new carrots and sticks, and it's going to demand more solid data. It's going to demand more detailed reporting and solid reporting, tighter governance. But we're still far from mainstream adoption. We have a lot of, you know, best of breed niche players in the space. I think the signs that it's going to be more mainstream are starting with things like Azure Purview, Google Dataplex, the big cloud platform players seem to be upping the ante and starting to address governance. >> Excellent, thank you Doug. Brad, I wonder if you could chime in as well. >> Yeah, I would love to be a believer in data catalogs. But to Doug's point, I think that it's going to take some more pressure for that to happen. I recall metadata being something every enterprise thought they were going to get under control when we were working on service oriented architecture back in the nineties and that didn't happen quite the way we anticipated. And so to Sanjeev's point it's because it is really complex and really difficult to do. My hope is that, you know, we won't sort of, how do I put this? Fade out into this nebula of domain catalogs that are specific to individual use cases like Purview for getting data quality right or like data governance and cybersecurity. And instead we have some tooling that can actually be adaptive to gather metadata to create something. And I know its important to you, Sanjeev and that is this idea of observability. If you can get enough metadata without moving your data around, but understanding the entirety of a system that's running on this data, you can do a lot. So to help with the governance that Doug is talking about. >> So I just want to add that, data governance, like any other initiatives did not succeed even AI went into an AI window, but that's a different topic. But a lot of these things did not succeed because to your point, the incentives were not there. I remember when Sarbanes Oxley had come into the scene, if a bank did not do Sarbanes Oxley, they were very happy to a million dollar fine. That was like, you know, pocket change for them instead of doing the right thing. But I think the stakes are much higher now. With GDPR, the flood gates opened. Now, you know, California, you know, has CCPA but even CCPA is being outdated with CPRA, which is much more GDPR like. So we are very rapidly entering a space where pretty much every major country in the world is coming up with its own compliance regulatory requirements, data residents is becoming really important. And I think we are going to reach a stage where it won't be optional anymore. So whether we like it or not, and I think the reason data catalogs were not successful in the past is because we did not have the right focus on adoption. We were focused on features and these features were disconnected, very hard for business to adopt. These are built by IT people for IT departments to take a look at technical metadata, not business metadata. Today the tables have turned. CDOs are driving this initiative, regulatory compliances are beating down hard, so I think the time might be right. >> Yeah so guys, we have to move on here. But there's some real meat on the bone here, Sanjeev. I like the fact that you called out Culebra and Alation, so we can look back a year from now and say, okay, he made the call, he stuck it. And then the ratio of BI tools to data catalogs that's another sort of measurement that we can take even though with some skepticism there, that's something that we can watch. And I wonder if someday, if we'll have more metadata than data. But I want to move to Tony Baer, you want to talk about data mesh and speaking, you know, coming off of governance. I mean, wow, you know the whole concept of data mesh is, decentralized data, and then governance becomes, you know, a nightmare there, but take it away, Tony. >> We'll put this way, data mesh, you know, the idea at least as proposed by ThoughtWorks. You know, basically it was at least a couple of years ago and the press has been almost uniformly almost uncritical. A good reason for that is for all the problems that basically Sanjeev and Doug and Brad we're just speaking about, which is that we have all this data out there and we don't know what to do about it. Now, that's not a new problem. That was a problem we had in enterprise data warehouses, it was a problem when we had over DoOP data clusters, it's even more of a problem now that data is out in the cloud where the data is not only your data lake, is not only us three, it's all over the place. And it's also including streaming, which I know we'll be talking about later. So the data mesh was a response to that, the idea of that we need to bait, you know, who are the folks that really know best about governance? It's the domain experts. So it was basically data mesh was an architectural pattern and a process. My prediction for this year is that data mesh is going to hit cold heart reality. Because if you do a Google search, basically the published work, the articles on data mesh have been largely, you know, pretty uncritical so far. Basically loading and is basically being a very revolutionary new idea. I don't think it's that revolutionary because we've talked about ideas like this. Brad now you and I met years ago when we were talking about so and decentralizing all of us, but it was at the application level. Now we're talking about it at the data level. And now we have microservices. So there's this thought of have we managed if we're deconstructing apps in cloud native to microservices, why don't we think of data in the same way? My sense this year is that, you know, this has been a very active search if you look at Google search trends, is that now companies, like enterprise are going to look at this seriously. And as they look at it seriously, it's going to attract its first real hard scrutiny, it's going to attract its first backlash. That's not necessarily a bad thing. It means that it's being taken seriously. The reason why I think that you'll start to see basically the cold hearted light of day shine on data mesh is that it's still a work in progress. You know, this idea is basically a couple of years old and there's still some pretty major gaps. The biggest gap is in the area of federated governance. Now federated governance itself is not a new issue. Federated governance decision, we started figuring out like, how can we basically strike the balance between getting let's say between basically consistent enterprise policy, consistent enterprise governance, but yet the groups that understand the data and know how to basically, you know, that, you know, how do we basically sort of balance the two? There's a huge gap there in practice and knowledge. Also to a lesser extent, there's a technology gap which is basically in the self-service technologies that will help teams essentially govern data. You know, basically through the full life cycle, from develop, from selecting the data from, you know, building the pipelines from, you know, determining your access control, looking at quality, looking at basically whether the data is fresh or whether it's trending off course. So my prediction is that it will receive the first harsh scrutiny this year. You are going to see some organization and enterprises declare premature victory when they build some federated query implementations. You going to see vendors start with data mesh wash their products anybody in the data management space that they are going to say that where this basically a pipelining tool, whether it's basically ELT, whether it's a catalog or federated query tool, they will all going to get like, you know, basically promoting the fact of how they support this. Hopefully nobody's going to call themselves a data mesh tool because data mesh is not a technology. We're going to see one other thing come out of this. And this harks back to the metadata that Sanjeev was talking about and of the catalog just as he was talking about. Which is that there's going to be a new focus, every renewed focus on metadata. And I think that's going to spur interest in data fabrics. Now data fabrics are pretty vaguely defined, but if we just take the most elemental definition, which is a common metadata back plane, I think that if anybody is going to get serious about data mesh, they need to look at the data fabric because we all at the end of the day, need to speak, you know, need to read from the same sheet of music. >> So thank you Tony. Dave Menninger, I mean, one of the things that people like about data mesh is it pretty crisply articulate some of the flaws in today's organizational approaches to data. What are your thoughts on this? >> Well, I think we have to start by defining data mesh, right? The term is already getting corrupted, right? Tony said it's going to see the cold hard light of day. And there's a problem right now that there are a number of overlapping terms that are similar but not identical. So we've got data virtualization, data fabric, excuse me for a second. (clears throat) Sorry about that. Data virtualization, data fabric, data federation, right? So I think that it's not really clear what each vendor means by these terms. I see data mesh and data fabric becoming quite popular. I've interpreted data mesh as referring primarily to the governance aspects as originally intended and specified. But that's not the way I see vendors using it. I see vendors using it much more to mean data fabric and data virtualization. So I'm going to comment on the group of those things. I think the group of those things is going to happen. They're going to happen, they're going to become more robust. Our research suggests that a quarter of organizations are already using virtualized access to their data lakes and another half, so a total of three quarters will eventually be accessing their data lakes using some sort of virtualized access. Again, whether you define it as mesh or fabric or virtualization isn't really the point here. But this notion that there are different elements of data, metadata and governance within an organization that all need to be managed collectively. The interesting thing is when you look at the satisfaction rates of those organizations using virtualization versus those that are not, it's almost double, 68% of organizations, I'm sorry, 79% of organizations that were using virtualized access express satisfaction with their access to the data lake. Only 39% express satisfaction if they weren't using virtualized access. >> Oh thank you Dave. Sanjeev we just got about a couple of minutes on this topic, but I know you're speaking or maybe you've always spoken already on a panel with (indistinct) who sort of invented the concept. Governance obviously is a big sticking point, but what are your thoughts on this? You're on mute. (panelist chuckling) >> So my message to (indistinct) and to the community is as opposed to what they said, let's not define it. We spent a whole year defining it, there are four principles, domain, product, data infrastructure, and governance. Let's take it to the next level. I get a lot of questions on what is the difference between data fabric and data mesh? And I'm like I can't compare the two because data mesh is a business concept, data fabric is a data integration pattern. How do you compare the two? You have to bring data mesh a level down. So to Tony's point, I'm on a warpath in 2022 to take it down to what does a data product look like? How do we handle shared data across domains and governance? And I think we are going to see more of that in 2022, or is "operationalization" of data mesh. >> I think we could have a whole hour on this topic, couldn't we? Maybe we should do that. But let's corner. Let's move to Carl. So Carl, you're a database guy, you've been around that block for a while now, you want to talk about graph databases, bring it on. >> Oh yeah. Okay thanks. So I regard graph database as basically the next truly revolutionary database management technology. I'm looking forward for the graph database market, which of course we haven't defined yet. So obviously I have a little wiggle room in what I'm about to say. But this market will grow by about 600% over the next 10 years. Now, 10 years is a long time. But over the next five years, we expect to see gradual growth as people start to learn how to use it. The problem is not that it's not useful, its that people don't know how to use it. So let me explain before I go any further what a graph database is because some of the folks on the call may not know what it is. A graph database organizes data according to a mathematical structure called a graph. The graph has elements called nodes and edges. So a data element drops into a node, the nodes are connected by edges, the edges connect one node to another node. Combinations of edges create structures that you can analyze to determine how things are related. In some cases, the nodes and edges can have properties attached to them which add additional informative material that makes it richer, that's called a property graph. There are two principle use cases for graph databases. There's semantic property graphs, which are use to break down human language texts into the semantic structures. Then you can search it, organize it and answer complicated questions. A lot of AI is aimed at semantic graphs. Another kind is the property graph that I just mentioned, which has a dazzling number of use cases. I want to just point out as I talk about this, people are probably wondering, well, we have relation databases, isn't that good enough? So a relational database defines... It supports what I call definitional relationships. That means you define the relationships in a fixed structure. The database drops into that structure, there's a value, foreign key value, that relates one table to another and that value is fixed. You don't change it. If you change it, the database becomes unstable, it's not clear what you're looking at. In a graph database, the system is designed to handle change so that it can reflect the true state of the things that it's being used to track. So let me just give you some examples of use cases for this. They include entity resolution, data lineage, social media analysis, Customer 360, fraud prevention. There's cybersecurity, there's strong supply chain is a big one actually. There is explainable AI and this is going to become important too because a lot of people are adopting AI. But they want a system after the fact to say, how do the AI system come to that conclusion? How did it make that recommendation? Right now we don't have really good ways of tracking that. Machine learning in general, social network, I already mentioned that. And then we've got, oh gosh, we've got data governance, data compliance, risk management. We've got recommendation, we've got personalization, anti money laundering, that's another big one, identity and access management, network and IT operations is already becoming a key one where you actually have mapped out your operation, you know, whatever it is, your data center and you can track what's going on as things happen there, root cause analysis, fraud detection is a huge one. A number of major credit card companies use graph databases for fraud detection, risk analysis, tracking and tracing turn analysis, next best action, what if analysis, impact analysis, entity resolution and I would add one other thing or just a few other things to this list, metadata management. So Sanjeev, here you go, this is your engine. Because I was in metadata management for quite a while in my past life. And one of the things I found was that none of the data management technologies that were available to us could efficiently handle metadata because of the kinds of structures that result from it, but graphs can, okay? Graphs can do things like say, this term in this context means this, but in that context, it means that, okay? Things like that. And in fact, logistics management, supply chain. And also because it handles recursive relationships, by recursive relationships I mean objects that own other objects that are of the same type. You can do things like build materials, you know, so like parts explosion. Or you can do an HR analysis, who reports to whom, how many levels up the chain and that kind of thing. You can do that with relational databases, but yet it takes a lot of programming. In fact, you can do almost any of these things with relational databases, but the problem is, you have to program it. It's not supported in the database. And whenever you have to program something, that means you can't trace it, you can't define it. You can't publish it in terms of its functionality and it's really, really hard to maintain over time. >> Carl, thank you. I wonder if we could bring Brad in, I mean. Brad, I'm sitting here wondering, okay, is this incremental to the market? Is it disruptive and replacement? What are your thoughts on this phase? >> It's already disrupted the market. I mean, like Carl said, go to any bank and ask them are you using graph databases to get fraud detection under control? And they'll say, absolutely, that's the only way to solve this problem. And it is frankly. And it's the only way to solve a lot of the problems that Carl mentioned. And that is, I think it's Achilles heel in some ways. Because, you know, it's like finding the best way to cross the seven bridges of Koenigsberg. You know, it's always going to kind of be tied to those use cases because it's really special and it's really unique and because it's special and it's unique, it's still unfortunately kind of stands apart from the rest of the community that's building, let's say AI outcomes, as a great example here. Graph databases and AI, as Carl mentioned, are like chocolate and peanut butter. But technologically, you think don't know how to talk to one another, they're completely different. And you know, you can't just stand up SQL and query them. You've got to learn, know what is the Carl? Specter special. Yeah, thank you to, to actually get to the data in there. And if you're going to scale that data, that graph database, especially a property graph, if you're going to do something really complex, like try to understand you know, all of the metadata in your organization, you might just end up with, you know, a graph database winter like we had the AI winter simply because you run out of performance to make the thing happen. So, I think it's already disrupted, but we need to like treat it like a first-class citizen in the data analytics and AI community. We need to bring it into the fold. We need to equip it with the tools it needs to do the magic it does and to do it not just for specialized use cases, but for everything. 'Cause I'm with Carl. I think it's absolutely revolutionary. >> Brad identified the principal, Achilles' heel of the technology which is scaling. When these things get large and complex enough that they spill over what a single server can handle, you start to have difficulties because the relationships span things that have to be resolved over a network and then you get network latency and that slows the system down. So that's still a problem to be solved. >> Sanjeev, any quick thoughts on this? I mean, I think metadata on the word cloud is going to be the largest font, but what are your thoughts here? >> I want to (indistinct) So people don't associate me with only metadata, so I want to talk about something slightly different. dbengines.com has done an amazing job. I think almost everyone knows that they chronicle all the major databases that are in use today. In January of 2022, there are 381 databases on a ranked list of databases. The largest category is RDBMS. The second largest category is actually divided into two property graphs and IDF graphs. These two together make up the second largest number databases. So talking about Achilles heel, this is a problem. The problem is that there's so many graph databases to choose from. They come in different shapes and forms. To Brad's point, there's so many query languages in RDBMS, in SQL. I know the story, but here We've got cipher, we've got gremlin, we've got GQL and then we're proprietary languages. So I think there's a lot of disparity in this space. >> Well, excellent. All excellent points, Sanjeev, if I must say. And that is a problem that the languages need to be sorted and standardized. People need to have a roadmap as to what they can do with it. Because as you say, you can do so many things. And so many of those things are unrelated that you sort of say, well, what do we use this for? And I'm reminded of the saying I learned a bunch of years ago. And somebody said that the digital computer is the only tool man has ever device that has no particular purpose. (panelists chuckle) >> All right guys, we got to move on to Dave Menninger. We've heard about streaming. Your prediction is in that realm, so please take it away. >> Sure. So I like to say that historical databases are going to become a thing of the past. By that I don't mean that they're going to go away, that's not my point. I mean, we need historical databases, but streaming data is going to become the default way in which we operate with data. So in the next say three to five years, I would expect that data platforms and we're using the term data platforms to represent the evolution of databases and data lakes, that the data platforms will incorporate these streaming capabilities. We're going to process data as it streams into an organization and then it's going to roll off into historical database. So historical databases don't go away, but they become a thing of the past. They store the data that occurred previously. And as data is occurring, we're going to be processing it, we're going to be analyzing it, we're going to be acting on it. I mean we only ever ended up with historical databases because we were limited by the technology that was available to us. Data doesn't occur in patches. But we processed it in patches because that was the best we could do. And it wasn't bad and we've continued to improve and we've improved and we've improved. But streaming data today is still the exception. It's not the rule, right? There are projects within organizations that deal with streaming data. But it's not the default way in which we deal with data yet. And so that's my prediction is that this is going to change, we're going to have streaming data be the default way in which we deal with data and how you label it and what you call it. You know, maybe these databases and data platforms just evolved to be able to handle it. But we're going to deal with data in a different way. And our research shows that already, about half of the participants in our analytics and data benchmark research, are using streaming data. You know, another third are planning to use streaming technologies. So that gets us to about eight out of 10 organizations need to use this technology. And that doesn't mean they have to use it throughout the whole organization, but it's pretty widespread in its use today and has continued to grow. If you think about the consumerization of IT, we've all been conditioned to expect immediate access to information, immediate responsiveness. You know, we want to know if an item is on the shelf at our local retail store and we can go in and pick it up right now. You know, that's the world we live in and that's spilling over into the enterprise IT world We have to provide those same types of capabilities. So that's my prediction, historical databases become a thing of the past, streaming data becomes the default way in which we operate with data. >> All right thank you David. Well, so what say you, Carl, the guy who has followed historical databases for a long time? >> Well, one thing actually, every database is historical because as soon as you put data in it, it's now history. They'll no longer reflect the present state of things. But even if that history is only a millisecond old, it's still history. But I would say, I mean, I know you're trying to be a little bit provocative in saying this Dave 'cause you know, as well as I do that people still need to do their taxes, they still need to do accounting, they still need to run general ledger programs and things like that. That all involves historical data. That's not going to go away unless you want to go to jail. So you're going to have to deal with that. But as far as the leading edge functionality, I'm totally with you on that. And I'm just, you know, I'm just kind of wondering if this requires a change in the way that we perceive applications in order to truly be manifested and rethinking the way applications work. Saying that an application should respond instantly, as soon as the state of things changes. What do you say about that? >> I think that's true. I think we do have to think about things differently. It's not the way we designed systems in the past. We're seeing more and more systems designed that way. But again, it's not the default. And I agree 100% with you that we do need historical databases you know, that's clear. And even some of those historical databases will be used in conjunction with the streaming data, right? >> Absolutely. I mean, you know, let's take the data warehouse example where you're using the data warehouse as its context and the streaming data as the present and you're saying, here's the sequence of things that's happening right now. Have we seen that sequence before? And where? What does that pattern look like in past situations? And can we learn from that? >> So Tony Baer, I wonder if you could comment? I mean, when you think about, you know, real time inferencing at the edge, for instance, which is something that a lot of people talk about, a lot of what we're discussing here in this segment, it looks like it's got a great potential. What are your thoughts? >> Yeah, I mean, I think you nailed it right. You know, you hit it right on the head there. Which is that, what I'm seeing is that essentially. Then based on I'm going to split this one down the middle is that I don't see that basically streaming is the default. What I see is streaming and basically and transaction databases and analytics data, you know, data warehouses, data lakes whatever are converging. And what allows us technically to converge is cloud native architecture, where you can basically distribute things. So you can have a node here that's doing the real-time processing, that's also doing... And this is where it leads in or maybe doing some of that real time predictive analytics to take a look at, well look, we're looking at this customer journey what's happening with what the customer is doing right now and this is correlated with what other customers are doing. So the thing is that in the cloud, you can basically partition this and because of basically the speed of the infrastructure then you can basically bring these together and kind of orchestrate them sort of a loosely coupled manner. The other parts that the use cases are demanding, and this is part of it goes back to what Dave is saying. Is that, you know, when you look at Customer 360, when you look at let's say Smart Utility products, when you look at any type of operational problem, it has a real time component and it has an historical component. And having predictive and so like, you know, my sense here is that technically we can bring this together through the cloud. And I think the use case is that we can apply some real time sort of predictive analytics on these streams and feed this into the transactions so that when we make a decision in terms of what to do as a result of a transaction, we have this real-time input. >> Sanjeev, did you have a comment? >> Yeah, I was just going to say that to Dave's point, you know, we have to think of streaming very different because in the historical databases, we used to bring the data and store the data and then we used to run rules on top, aggregations and all. But in case of streaming, the mindset changes because the rules are normally the inference, all of that is fixed, but the data is constantly changing. So it's a completely reversed way of thinking and building applications on top of that. >> So Dave Menninger, there seem to be some disagreement about the default. What kind of timeframe are you thinking about? Is this end of decade it becomes the default? What would you pin? >> I think around, you know, between five to 10 years, I think this becomes the reality. >> I think its... >> It'll be more and more common between now and then, but it becomes the default. And I also want Sanjeev at some point, maybe in one of our subsequent conversations, we need to talk about governing streaming data. 'Cause that's a whole nother set of challenges. >> We've also talked about it rather in two dimensions, historical and streaming, and there's lots of low latency, micro batch, sub-second, that's not quite streaming, but in many cases its fast enough and we're seeing a lot of adoption of near real time, not quite real-time as good enough for many applications. (indistinct cross talk from panelists) >> Because nobody's really taking the hardware dimension (mumbles). >> That'll just happened, Carl. (panelists laughing) >> So near real time. But maybe before you lose the customer, however we define that, right? Okay, let's move on to Brad. Brad, you want to talk about automation, AI, the pipeline people feel like, hey, we can just automate everything. What's your prediction? >> Yeah I'm an AI aficionados so apologies in advance for that. But, you know, I think that we've been seeing automation play within AI for some time now. And it's helped us do a lot of things especially for practitioners that are building AI outcomes in the enterprise. It's helped them to fill skills gaps, it's helped them to speed development and it's helped them to actually make AI better. 'Cause it, you know, in some ways provide some swim lanes and for example, with technologies like AutoML can auto document and create that sort of transparency that we talked about a little bit earlier. But I think there's an interesting kind of conversion happening with this idea of automation. And that is that we've had the automation that started happening for practitioners, it's trying to move out side of the traditional bounds of things like I'm just trying to get my features, I'm just trying to pick the right algorithm, I'm just trying to build the right model and it's expanding across that full life cycle, building an AI outcome, to start at the very beginning of data and to then continue on to the end, which is this continuous delivery and continuous automation of that outcome to make sure it's right and it hasn't drifted and stuff like that. And because of that, because it's become kind of powerful, we're starting to actually see this weird thing happen where the practitioners are starting to converge with the users. And that is to say that, okay, if I'm in Tableau right now, I can stand up Salesforce Einstein Discovery, and it will automatically create a nice predictive algorithm for me given the data that I pull in. But what's starting to happen and we're seeing this from the companies that create business software, so Salesforce, Oracle, SAP, and others is that they're starting to actually use these same ideals and a lot of deep learning (chuckles) to basically stand up these out of the box flip-a-switch, and you've got an AI outcome at the ready for business users. And I am very much, you know, I think that's the way that it's going to go and what it means is that AI is slowly disappearing. And I don't think that's a bad thing. I think if anything, what we're going to see in 2022 and maybe into 2023 is this sort of rush to put this idea of disappearing AI into practice and have as many of these solutions in the enterprise as possible. You can see, like for example, SAP is going to roll out this quarter, this thing called adaptive recommendation services, which basically is a cold start AI outcome that can work across a whole bunch of different vertical markets and use cases. It's just a recommendation engine for whatever you needed to do in the line of business. So basically, you're an SAP user, you look up to turn on your software one day, you're a sales professional let's say, and suddenly you have a recommendation for customer churn. Boom! It's going, that's great. Well, I don't know, I think that's terrifying. In some ways I think it is the future that AI is going to disappear like that, but I'm absolutely terrified of it because I think that what it really does is it calls attention to a lot of the issues that we already see around AI, specific to this idea of what we like to call at Omdia, responsible AI. Which is, you know, how do you build an AI outcome that is free of bias, that is inclusive, that is fair, that is safe, that is secure, that its audible, et cetera, et cetera, et cetera, et cetera. I'd take a lot of work to do. And so if you imagine a customer that's just a Salesforce customer let's say, and they're turning on Einstein Discovery within their sales software, you need some guidance to make sure that when you flip that switch, that the outcome you're going to get is correct. And that's going to take some work. And so, I think we're going to see this move, let's roll this out and suddenly there's going to be a lot of problems, a lot of pushback that we're going to see. And some of that's going to come from GDPR and others that Sanjeev was mentioning earlier. A lot of it is going to come from internal CSR requirements within companies that are saying, "Hey, hey, whoa, hold up, we can't do this all at once. "Let's take the slow route, "let's make AI automated in a smart way." And that's going to take time. >> Yeah, so a couple of predictions there that I heard. AI simply disappear, it becomes invisible. Maybe if I can restate that. And then if I understand it correctly, Brad you're saying there's a backlash in the near term. You'd be able to say, oh, slow down. Let's automate what we can. Those attributes that you talked about are non trivial to achieve, is that why you're a bit of a skeptic? >> Yeah. I think that we don't have any sort of standards that companies can look to and understand. And we certainly, within these companies, especially those that haven't already stood up an internal data science team, they don't have the knowledge to understand when they flip that switch for an automated AI outcome that it's going to do what they think it's going to do. And so we need some sort of standard methodology and practice, best practices that every company that's going to consume this invisible AI can make use of them. And one of the things that you know, is sort of started that Google kicked off a few years back that's picking up some momentum and the companies I just mentioned are starting to use it is this idea of model cards where at least you have some transparency about what these things are doing. You know, so like for the SAP example, we know, for example, if it's convolutional neural network with a long, short term memory model that it's using, we know that it only works on Roman English and therefore me as a consumer can say, "Oh, well I know that I need to do this internationally. "So I should not just turn this on today." >> Thank you. Carl could you add anything, any context here? >> Yeah, we've talked about some of the things Brad mentioned here at IDC and our future of intelligence group regarding in particular, the moral and legal implications of having a fully automated, you know, AI driven system. Because we already know, and we've seen that AI systems are biased by the data that they get, right? So if they get data that pushes them in a certain direction, I think there was a story last week about an HR system that was recommending promotions for White people over Black people, because in the past, you know, White people were promoted and more productive than Black people, but it had no context as to why which is, you know, because they were being historically discriminated, Black people were being historically discriminated against, but the system doesn't know that. So, you know, you have to be aware of that. And I think that at the very least, there should be controls when a decision has either a moral or legal implication. When you really need a human judgment, it could lay out the options for you. But a person actually needs to authorize that action. And I also think that we always will have to be vigilant regarding the kind of data we use to train our systems to make sure that it doesn't introduce unintended biases. In some extent, they always will. So we'll always be chasing after them. But that's (indistinct). >> Absolutely Carl, yeah. I think that what you have to bear in mind as a consumer of AI is that it is a reflection of us and we are a very flawed species. And so if you look at all of the really fantastic, magical looking supermodels we see like GPT-3 and four, that's coming out, they're xenophobic and hateful because the people that the data that's built upon them and the algorithms and the people that build them are us. So AI is a reflection of us. We need to keep that in mind. >> Yeah, where the AI is biased 'cause humans are biased. All right, great. All right let's move on. Doug you mentioned mentioned, you know, lot of people that said that data lake, that term is not going to live on but here's to be, have some lakes here. You want to talk about lake house, bring it on. >> Yes, I do. My prediction is that lake house and this idea of a combined data warehouse and data lake platform is going to emerge as the dominant data management offering. I say offering that doesn't mean it's going to be the dominant thing that organizations have out there, but it's going to be the pro dominant vendor offering in 2022. Now heading into 2021, we already had Cloudera, Databricks, Microsoft, Snowflake as proponents, in 2021, SAP, Oracle, and several of all of these fabric virtualization/mesh vendors joined the bandwagon. The promise is that you have one platform that manages your structured, unstructured and semi-structured information. And it addresses both the BI analytics needs and the data science needs. The real promise there is simplicity and lower cost. But I think end users have to answer a few questions. The first is, does your organization really have a center of data gravity or is the data highly distributed? Multiple data warehouses, multiple data lakes, on premises, cloud. If it's very distributed and you'd have difficulty consolidating and that's not really a goal for you, then maybe that single platform is unrealistic and not likely to add value to you. You know, also the fabric and virtualization vendors, the mesh idea, that's where if you have this highly distributed situation, that might be a better path forward. The second question, if you are looking at one of these lake house offerings, you are looking at consolidating, simplifying, bringing together to a single platform. You have to make sure that it meets both the warehouse need and the data lake need. So you have vendors like Databricks, Microsoft with Azure Synapse. New really to the data warehouse space and they're having to prove that these data warehouse capabilities on their platforms can meet the scaling requirements, can meet the user and query concurrency requirements. Meet those tight SLS. And then on the other hand, you have the Oracle, SAP, Snowflake, the data warehouse folks coming into the data science world, and they have to prove that they can manage the unstructured information and meet the needs of the data scientists. I'm seeing a lot of the lake house offerings from the warehouse crowd, managing that unstructured information in columns and rows. And some of these vendors, Snowflake a particular is really relying on partners for the data science needs. So you really got to look at a lake house offering and make sure that it meets both the warehouse and the data lake requirement. >> Thank you Doug. Well Tony, if those two worlds are going to come together, as Doug was saying, the analytics and the data science world, does it need to be some kind of semantic layer in between? I don't know. Where are you in on this topic? >> (chuckles) Oh, didn't we talk about data fabrics before? Common metadata layer (chuckles). Actually, I'm almost tempted to say let's declare victory and go home. And that this has actually been going on for a while. I actually agree with, you know, much of what Doug is saying there. Which is that, I mean I remember as far back as I think it was like 2014, I was doing a study. I was still at Ovum, (indistinct) Omdia, looking at all these specialized databases that were coming up and seeing that, you know, there's overlap at the edges. But yet, there was still going to be a reason at the time that you would have, let's say a document database for JSON, you'd have a relational database for transactions and for data warehouse and you had basically something at that time that resembles a dupe for what we consider your data life. Fast forward and the thing is what I was seeing at the time is that you were saying they sort of blending at the edges. That was saying like about five to six years ago. And the lake house is essentially on the current manifestation of that idea. There is a dichotomy in terms of, you know, it's the old argument, do we centralize this all you know in a single place or do we virtualize? And I think it's always going to be a union yeah and there's never going to be a single silver bullet. I do see that there are also going to be questions and these are points that Doug raised. That you know, what do you need for your performance there, or for your free performance characteristics? Do you need for instance high concurrency? You need the ability to do some very sophisticated joins, or is your requirement more to be able to distribute and distribute our processing is, you know, as far as possible to get, you know, to essentially do a kind of a brute force approach. All these approaches are valid based on the use case. I just see that essentially that the lake house is the culmination of it's nothing. It's a relatively new term introduced by Databricks a couple of years ago. This is the culmination of basically what's been a long time trend. And what we see in the cloud is that as we start seeing data warehouses as a check box items say, "Hey, we can basically source data in cloud storage, in S3, "Azure Blob Store, you know, whatever, "as long as it's in certain formats, "like, you know parquet or CSP or something like that." I see that as becoming kind of a checkbox item. So to that extent, I think that the lake house, depending on how you define is already reality. And in some cases, maybe new terminology, but not a whole heck of a lot new under the sun. >> Yeah. And Dave Menninger, I mean a lot of these, thank you Tony, but a lot of this is going to come down to, you know, vendor marketing, right? Some people just kind of co-op the term, we talked about you know, data mesh washing, what are your thoughts on this? (laughing) >> Yeah, so I used the term data platform earlier. And part of the reason I use that term is that it's more vendor neutral. We've tried to sort of stay out of the vendor terminology patenting world, right? Whether the term lake houses, what sticks or not, the concept is certainly going to stick. And we have some data to back it up. About a quarter of organizations that are using data lakes today, already incorporate data warehouse functionality into it. So they consider their data lake house and data warehouse one in the same, about a quarter of organizations, a little less, but about a quarter of organizations feed the data lake from the data warehouse and about a quarter of organizations feed the data warehouse from the data lake. So it's pretty obvious that three quarters of organizations need to bring this stuff together, right? The need is there, the need is apparent. The technology is going to continue to converge. I like to talk about it, you know, you've got data lakes over here at one end, and I'm not going to talk about why people thought data lakes were a bad idea because they thought you just throw stuff in a server and you ignore it, right? That's not what a data lake is. So you've got data lake people over here and you've got database people over here, data warehouse people over here, database vendors are adding data lake capabilities and data lake vendors are adding data warehouse capabilities. So it's obvious that they're going to meet in the middle. I mean, I think it's like Tony says, I think we should declare victory and go home. >> As hell. So just a follow-up on that, so are you saying the specialized lake and the specialized warehouse, do they go away? I mean, Tony data mesh practitioners would say or advocates would say, well, they could all live. It's just a node on the mesh. But based on what Dave just said, are we gona see those all morphed together? >> Well, number one, as I was saying before, there's always going to be this sort of, you know, centrifugal force or this tug of war between do we centralize the data, do we virtualize? And the fact is I don't think that there's ever going to be any single answer. I think in terms of data mesh, data mesh has nothing to do with how you're physically implement the data. You could have a data mesh basically on a data warehouse. It's just that, you know, the difference being is that if we use the same physical data store, but everybody's logically you know, basically governing it differently, you know? Data mesh in space, it's not a technology, it's processes, it's governance process. So essentially, you know, I basically see that, you know, as I was saying before that this is basically the culmination of a long time trend we're essentially seeing a lot of blurring, but there are going to be cases where, for instance, if I need, let's say like, Upserve, I need like high concurrency or something like that. There are certain things that I'm not going to be able to get efficiently get out of a data lake. And, you know, I'm doing a system where I'm just doing really brute forcing very fast file scanning and that type of thing. So I think there always will be some delineations, but I would agree with Dave and with Doug, that we are seeing basically a confluence of requirements that we need to essentially have basically either the element, you know, the ability of a data lake and the data warehouse, these need to come together, so I think. >> I think what we're likely to see is organizations look for a converge platform that can handle both sides for their center of data gravity, the mesh and the fabric virtualization vendors, they're all on board with the idea of this converged platform and they're saying, "Hey, we'll handle all the edge cases "of the stuff that isn't in that center of data gravity "but that is off distributed in a cloud "or at a remote location." So you can have that single platform for the center of your data and then bring in virtualization, mesh, what have you, for reaching out to the distributed data. >> As Dave basically said, people are happy when they virtualized data. >> I think we have at this point, but to Dave Menninger's point, they are converging, Snowflake has introduced support for unstructured data. So obviously literally splitting here. Now what Databricks is saying is that "aha, but it's easy to go from data lake to data warehouse "than it is from databases to data lake." So I think we're getting into semantics, but we're already seeing these two converge. >> So take somebody like AWS has got what? 15 data stores. Are they're going to 15 converge data stores? This is going to be interesting to watch. All right, guys, I'm going to go down and list do like a one, I'm going to one word each and you guys, each of the analyst, if you would just add a very brief sort of course correction for me. So Sanjeev, I mean, governance is going to to be... Maybe it's the dog that wags the tail now. I mean, it's coming to the fore, all this ransomware stuff, which you really didn't talk much about security, but what's the one word in your prediction that you would leave us with on governance? >> It's going to be mainstream. >> Mainstream. Okay. Tony Baer, mesh washing is what I wrote down. That's what we're going to see in 2022, a little reality check, you want to add to that? >> Reality check, 'cause I hope that no vendor jumps the shark and close they're offering a data niche product. >> Yeah, let's hope that doesn't happen. If they do, we're going to call them out. Carl, I mean, graph databases, thank you for sharing some high growth metrics. I know it's early days, but magic is what I took away from that, so magic database. >> Yeah, I would actually, I've said this to people too. I kind of look at it as a Swiss Army knife of data because you can pretty much do anything you want with it. That doesn't mean you should. I mean, there's definitely the case that if you're managing things that are in fixed schematic relationship, probably a relation database is a better choice. There are times when the document database is a better choice. It can handle those things, but maybe not. It may not be the best choice for that use case. But for a great many, especially with the new emerging use cases I listed, it's the best choice. >> Thank you. And Dave Menninger, thank you by the way, for bringing the data in, I like how you supported all your comments with some data points. But streaming data becomes the sort of default paradigm, if you will, what would you add? >> Yeah, I would say think fast, right? That's the world we live in, you got to think fast. >> Think fast, love it. And Brad Shimmin, love it. I mean, on the one hand I was saying, okay, great. I'm afraid I might get disrupted by one of these internet giants who are AI experts. I'm going to be able to buy instead of build AI. But then again, you know, I've got some real issues. There's a potential backlash there. So give us your bumper sticker. >> I'm would say, going with Dave, think fast and also think slow to talk about the book that everyone talks about. I would say really that this is all about trust, trust in the idea of automation and a transparent and visible AI across the enterprise. And verify, verify before you do anything. >> And then Doug Henschen, I mean, I think the trend is your friend here on this prediction with lake house is really becoming dominant. I liked the way you set up that notion of, you know, the data warehouse folks coming at it from the analytics perspective and then you get the data science worlds coming together. I still feel as though there's this piece in the middle that we're missing, but your, your final thoughts will give you the (indistinct). >> I think the idea of consolidation and simplification always prevails. That's why the appeal of a single platform is going to be there. We've already seen that with, you know, DoOP platforms and moving toward cloud, moving toward object storage and object storage, becoming really the common storage point for whether it's a lake or a warehouse. And that second point, I think ESG mandates are going to come in alongside GDPR and things like that to up the ante for good governance. >> Yeah, thank you for calling that out. Okay folks, hey that's all the time that we have here, your experience and depth of understanding on these key issues on data and data management really on point and they were on display today. I want to thank you for your contributions. Really appreciate your time. >> Enjoyed it. >> Thank you. >> Thanks for having me. >> In addition to this video, we're going to be making available transcripts of the discussion. We're going to do clips of this as well we're going to put them out on social media. I'll write this up and publish the discussion on wikibon.com and siliconangle.com. No doubt, several of the analysts on the panel will take the opportunity to publish written content, social commentary or both. I want to thank the power panelists and thanks for watching this special CUBE presentation. This is Dave Vellante, be well and we'll see you next time. (bright music)

>>Welcome back everybody to AWS reinvent. You're watching the cube, the leader in high tech coverage. My name is Dave Volante with my co-host David Nicholson. We're here all week. We got two sets, 20 plus thousand people here live at AWS reinvent. 21 of course last year was virtual. We got a hybrid event running. We had two studios running before the show running. A lot of pre-records really excited to have ninja Bardelli Walla, who is the chief product officer at data robot. Really interesting AI company. We're going to talk about insights with machine intelligence and then shout. It's great to see you again. It's been awhile. >>Great to see you as well. And I'm so happy to be on the cube. I think eight years since I first came on. >>When you launched the company that you founded back then Peck Sada on the cube, that was part >>Of the inner robot >>Family part of data, robot family. And of course, friend of the cube. Chris Lynch is the executive chairman of data robot. So a lot of connections, I always joke a hundred people in our industry, 99 seats, but tell us about data robot. What's the, what's the scoop these days. >>Thanks. Thanks very much for the opportunity to speak with both of you. Uh, I think we're seeing some very interesting trends. Uh, we've all been in the industry long enough to recognize, uh, that hype cycles they're cycles. They go in waves and, uh, the level of interest in AI has never been higher. Uh, every company in the world is looking for the opportunity to take advantage of AI, to improve their business processes, whether it's to improve their revenue it's to lower their cost profile or it's to lower their risk. What we're seeing that's most interesting is that, uh, we spend a lot of time working with companies on what we consider applied AI. That is how do we solve real business problems, uh, with the technology and not just run a bunch of experiments. You know, it's very tempting for a lot of us, Dave and David, uh, to, to do, uh, you know, spin up a spark cluster with 10,000 nodes and slosh a bunch of data through it. >>But the question we always ask at data robot is what is the business value of doing this? Why are we using these AI techniques and in order to solve what problem? So the biggest trend we see a data robot and one that we feel we're very well positioned to solve is that companies are coming out of that experimental phase. There's still a lot of experimentation going on and they're saying, okay, we, we stood up a cluster. Uh, we got a bunch of Python notebooks running around here, but we haven't really seen a return on our investment yet data robot, can you help us actually make AI real and concrete in terms of achieving a specific business outcome for us? >>Well, and I want to test something on your niche. That's something we've talked about a lot on the cube is a change in the way in which companies are architecting their data. When we first, it was like, okay, create a Hadoop cluster. And that spark came along to make that easier, but it was still this highly technical, highly centralized, hyper specialized roles where the business, people who have a really good understanding of the outcome had to kind of beg to get what they wanted because it was so technical and the success was defined as, Hey, it worked or we ran the experiment and it looks like it has promise. So now it seems like with companies like data robot, you're democratizing AI, allowing organizations to inject AI into their business processes, their applications. And it seems to be more business led. One of you could comment on that. >>I think that is a various dude observation. Uh, we launched this concept a little bit earlier this year of AI cloud. And the idea behind AI cloud is if you want to democratize AI, which is in fact has been DataRobot's vision since 2012, we were the first company on the cloud. The first AI cloud that ever existed was data robots in 2014. And the entire idea was that we knew that data scientists would always play a very important role in an organization, but yet the demand for AI would vastly outstrip the supply. And so in order to solve that challenge, we built AI cloud. We've actually spent over a million engineering hours in building this technology over the, over the last decade and put this together in a way where all of the different personas and the organizations, you have people who create AI applications. >>Those are the folks we usually think about, but those are the data scientists. Those are the analysts, those are the data engineers, but then you actually have to put it into production. You've got to run the system. So you also have to democratize this capability for the folks who are going to operate the system for the folks in risk and compliance. We're actually going to, uh, ensure that the system is operating in accordance with your policies and compliance regimes. And then the third wave of democratization, which we've just embarked on is then how do you bring AI into the hands of the actual business people? How do you put on a mobile device or a web browser, or in context, in an application with the decision, the ability for AI to drive a decision in your organization, which leads to an action, which helps drive you towards the outcome you're trying to optimize for. >>So AI cloud is about this pervasive tapestry, bringing together the creators, the consumers, the individuals who operate these systems into a single system that can lower the barrier to entry for people who don't have the skills, but allow you to plug in and go deep underneath the covers and modify whatever you need to, if you have that level of technical skill and that ability for us to kind of slide, slide the slider in one direction or the other, I could slide it to the right and say, I want all automation, something data robot has pioneered and is absolutely the leader in, but we can also, especially in these last couple of years, say, I want to be able to use as much code as I want to bring in. And the beauty of the model is that customers can choose how much they want to let the machine drive or how much they want to let the human being drive. David. I love that, >>That idea of a slider, because now you're talking about generalists getting access to really powerful tools. >>Yeah, no, exactly. And I, I'm curious, what's your view on where we are culturally with AI at this point? And what I mean by culturally is the idea that, okay, that's great. You put powerful tools in the hands of business users. Um, do most of us still need to have a lot of visibility under the covers to understand the inner workings so that we trust what we're being told? You know, I'm fine pulling a lever and having a little biscuit come out of SWOT as long as I've gotten a tour of the kitchen at some point in time. Yes. I mean, where are we with that? Where where's the level of >>Absolutely fantastic question and it's one that's, it's actually pervasive to the way data robot operates. So trust gets, uh, engendered by multiple different capabilities that you build throughout the platform. The first one is around, uh, explainability. So when you get a prediction from a system, just like you mentioned, you know, if, if the stakes are not very high, you know, you, uh, we're here in Las Vegas, of course I'm thinking of slot machines. If you get a biscuit at the end of it and it tastes pretty good. Hey, great. Right? When you're making a mission critical business decision, you don't want to be in the position where you don't understand why the system is making the decision. It does. So we have historically invested an enormous amount of effort in explainability tools, having the system actually at a prediction level, explain to you, why is it making the recommendation it's making? >>For example, the system says this customer has a high likelihood of churn. Why? Because their account balance has been declining over the last five months. Uh, number two, because their credit score has been going down. And what gives you the trust is actually the machine and the human able to communicate in the same language and same vernacular about the business value. So that's one part of it. The second part is about transparency, right? So one of the things that the automated machine learning movement, that data robot pioneered, uh, has been, I'd say rightfully criticized for frankly, is that it's too much of a black box. It's too much magic. I load my dataset. I press the start button and data robot does everything else for me. Well, that's not very satisfying when you have a 10 or a hundred million dollar decision coming on the other side, even if the technology is actually doing the job correctly, which data robot usually does. >>So where we've morphed and evolved our position in the market and where I have driven our technology portfolio at data robot is to say, you know what? There is a very important aspect of trust that needs to be brought to bear here, which is that if somebody wants to see code, let them see code. And in fact, the beauty of AI cloud is that on the same platform, the people who don't like code, but are, are very good at understanding the business domain con uh, the business domain knowledge and the context. They now have the ability to do that. But when they're at the stage before they're going to deploy anything to production. Now you can raise your hand at data robot and actually use our workflow and say, I need a coder to review this. I want the professional data scientist who has all this knowledge who understands and has read up on the latest advances in hyper parameter tuning to look at the model and tell me that this is going to be okay. And so we allow both the less technical folks and the very deep technical data scientists, the ability to collaborate on the same environment, which allows you to build trust in terms of the human side of, Hey, I don't want to just let anybody throw a model into production. I like, >>I mean, I see those, the transparency and the explainability is almost two sides of the same coin, right? Because you know, if you're gonna be accused of gender bias, you can say, no, here's how the system may, it's not like, you know, you think about the internet. It tells you it's a cat, but you don't really know how the machine determined that you're breaking apart, blowing away that black box. And the other thing I like what you said was you have data producers and data consumers, and you also talked about context because a lot of times the data producers, they don't necessarily care about the context or the PI data pipeline. People necessarily care about the context. So, okay. So now we're at the point where you're democratizing data, you're doing some great work. What are some of the blockers that you see today that you're obliterating with data robot? Maybe you could talk about that a little bit. Sure. >>So, so I think, uh, you know, one very important concept is that, uh, in a democracy, we talked about democratization. You still have rules, you still have governance. It's not a free for all the free for all version of that is called NRG. That's not what any company wants, right? So we have to blend the freedom and flexibility that we want businesses to have with the compliance and regulatory observability that we need in order to be successful. So what we're seeing in, in our, in our customer base and what companies are coming to data robot to discuss is, okay, we've tried these experiments. Now we want to actually get to real business value. And one of the things that's really unique about data robot is that we have put, uh, we have, we've worked in our system on over 1 million projects, training models, inside data robot. >>We have seen every type of use case across different industries, whether it's healthcare or manufacturing, uh, or, or retail, uh, we have the ability to understand those different data sets and actually to come up with models. So we have that breadth of information there if you aggregate that over time, right? So again, we did not come to AI. This is not a fad for us. We didn't start as one kind of company than slap the AI label on and say, Hey, we're an AI company now, right? We have been AI native since day one. And in that process, what we have found is working on these, this million plus projects on these data sets across these industries, we have a very good sense of which projects will actually deliver value and which don't. And that gets to a previous point that you were making, which is that you have to know and partner with an organization who it's not just about the technology. So we have fantastic people who we call our customer facing data scientists who will tell the customer, look, I know you think this is a really high value use case, but we've tried it at other customers. And unfortunately it didn't work very well. Let's steer you, cause you need with a, with a technology that is largely at the early stage and the maturity that organizations have with it, you need to help them in order to deliver success. And no vendor has delivered more successful production deployment of AI than data road. >>No, don't go down that path. It's a dead end as a cul-de-sac. So just avoid it. So we talked about transparency, explainability governance. Can you get that to the point where it's self-serve as you, as you put data in the hands of business, people where the context lives, the domain experts, can you get to self-serve and federate that governance? Yes. >>So you can, uh, that's one of the key principles of what we, what we do at data robot. And it comes back to a concept that I learned, uh, you, you both will remember. We were in the Sarbanes-Oxley crazy world of, I dunno, was that 15 years of saved data warehousing. >>Everybody wanted to talk about socks. You know, my wife would hear me on the phone. She'd be like, what is your sudden obsession with socks? I'm like, no, no, it's not what you fit. And so, um, but what came from Sarbanes Oxley are, are these, uh, longstanding principles around the segregation of duties and segregation of responsibilities. You can have democracy democratization with governance, if you have the right segregation of duties. So for example, I have somebody who can generate lots of different models, right? But I don't allow them to, to, uh, in a self-service way, just deploy into production. I actually have a workflow system which will go through multiple rigorous approvals and say, these three people have signed off, they've done an audit, uh, an, an audit assessment of this model. It's good to go, let's go and drop it into production. So the way that you get to self-service with governance is to have the right controls and policies and frameworks that surround the self-service model with the right checks and balances that implement the segregation of duties I'm talking >>And you get that right. And then you can automate it and then you can really scale, right? You gotta have your back because it's such a great topic. We, we barely scratched the surface. It was great to see you again, congratulations on all the success. And, uh, as I say any time, let's do this again. Fantastic. Thank >>You so much. All right, you're welcome. And thank you for watching you watching the cubes coverage of AWS reinvent 2021, Dave Volante for David Nicholson. Keep it right there. You're watching the cube, the leader in high-tech coverage.

>> From theCUBE Studios in Palo Alto in Boston, bringing you data-driven insights from theCUBE in ETR. This is Breaking Analysis with Dave Vellante. >> Snowflake is not going to grow into its valuation by stealing the croissant from the breakfast table of the on-prem data warehouse vendors. Look, even if snowflake got 100% of the data warehouse business, it wouldn't come close to justifying its market cap. Rather Snowflake has to create an entirely new market based on completely changing the way organizations think about monetizing data. Every organization I talk to says it wants to be, or many say they already are data-driven. why wouldn't you aspire to that goal? There's probably nothing more strategic than leveraging data to power your digital business and creating competitive advantage. But many businesses are failing, or I predict, will fail to create a true data-driven culture because they're relying on a flawed architectural model formed by decades of building centralized data platforms. Welcome everyone to this week's Wikibon Cube Insights powered by ETR. In this Breaking Analysis, I want to share some new thoughts and fresh ETR data on how organizations can transform their businesses through data by reinventing their data architectures. And I want to share our thoughts on why we think Snowflake is currently in a very strong position to lead this effort. Now, on November 17th, theCUBE is hosting the Snowflake Data Cloud Summit. Snowflake's ascendancy and its blockbuster IPO has been widely covered by us and many others. Now, since Snowflake went public, we've been inundated with outreach from investors, customers, and competitors that wanted to either better understand the opportunities or explain why their approach is better or different. And in this segment, ahead of Snowflake's big event, we want to share some of what we learned and how we see it. Now, theCUBE is getting paid to host this event, so I need you to know that, and you draw your own conclusions from my remarks. But neither Snowflake nor any other sponsor of theCUBE or client of SiliconANGLE Media has editorial influence over Breaking Analysis. The opinions here are mine, and I would encourage you to read my ethics statement in this regard. I want to talk about the failed data model. The problem is complex, I'm not debating that. Organizations have to integrate data and platforms with existing operational systems, many of which were developed decades ago. And as a culture and a set of processes that have been built around these systems, and they've been hardened over the years. This chart here tries to depict the progression of the monolithic data source, which, for me, began in the 1980s when Decision Support Systems or DSS promised to solve our data problems. The data warehouse became very popular and data marts sprung up all over the place. This created more proprietary stovepipes with data locked inside. The Enron collapse led to Sarbanes-Oxley. Now, this tightened up reporting. The requirements associated with that, it breathed new life into the data warehouse model. But it remained expensive and cumbersome, I've talked about that a lot, like a snake swallowing a basketball. The 2010s ushered in the big data movement, and Data Lakes emerged. With a dupe, we saw the idea of no schema online, where you put structured and unstructured data into a repository, and figure it all out on the read. What emerged was a fairly complex data pipeline that involved ingesting, cleaning, processing, analyzing, preparing, and ultimately serving data to the lines of business. And this is where we are today with very hyper specialized roles around data engineering, data quality, data science. There's lots of batch of processing going on, and Spark has emerged to improve the complexity associated with MapReduce, and it definitely helped improve the situation. We're also seeing attempts to blend in real time stream processing with the emergence of tools like Kafka and others. But I'll argue that in a strange way, these innovations actually compound the problem. And I want to discuss that because what they do is they heighten the need for more specialization, more fragmentation, and more stovepipes within the data life cycle. Now, in reality, and it pains me to say this, it's the outcome of the big data movement, as we sit here in 2020, that we've created thousands of complicated science projects that have once again failed to live up to the promise of rapid cost-effective time to insights. So, what will the 2020s bring? What's the next silver bullet? You hear terms like the lakehouse, which Databricks is trying to popularize. And I'm going to talk today about data mesh. These are other efforts they look to modernize datalakes and sometimes merge the best of data warehouse and second-generation systems into a new paradigm, that might unify batch and stream frameworks. And this definitely addresses some of the gaps, but in our view, still suffers from some of the underlying problems of previous generation data architectures. In other words, if the next gen data architecture is incremental, centralized, rigid, and primarily focuses on making the technology to get data in and out of the pipeline work, we predict it's going to fail to live up to expectations again. Rather, what we're envisioning is an architecture based on the principles of distributed data, where domain knowledge is the primary target citizen, and data is not seen as a by-product, i.e, the exhaust of an operational system, but rather as a service that can be delivered in multiple forms and use cases across an ecosystem. This is why we often say the data is not the new oil. We don't like that phrase. A specific gallon of oil can either fuel my home or can lubricate my car engine, but it can't do both. Data does not follow the same laws of scarcity like natural resources. Again, what we're envisioning is a rethinking of the data pipeline and the associated cultures to put data needs of the domain owner at the core and provide automated, governed, and secure access to data as a service at scale. Now, how is this different? Let's take a look and unpack the data pipeline today and look deeper into the situation. You all know this picture that I'm showing. There's nothing really new here. The data comes from inside and outside the enterprise. It gets processed, cleanse or augmented so that it can be trusted and made useful. Nobody wants to use data that they can't trust. And then we can add machine intelligence and do more analysis, and finally deliver the data so that domain specific consumers can essentially build data products and services or reports and dashboards or content services, for instance, an insurance policy, a financial product, a loan, that these are packaged and made available for someone to make decisions on or to make a purchase. And all the metadata associated with this data is packaged along with the dataset. Now, we've broken down these steps into atomic components over time so we can optimize on each and make them as efficient as possible. And down below, you have these happy stick figures. Sometimes they're happy. But they're highly specialized individuals and they each do their job and they do it well to make sure that the data gets in, it gets processed and delivered in a timely manner. Now, while these individual pieces seemingly are autonomous and can be optimized and scaled, they're all encompassed within the centralized big data platform. And it's generally accepted that this platform is domain agnostic. Meaning the platform is the data owner, not the domain specific experts. Now there are a number of problems with this model. The first, while it's fine for organizations with smaller number of domains, organizations with a large number of data sources and complex domain structures, they struggle to create a common data parlance, for example, in a data culture. Another problem is that, as the number of data sources grows, organizing and harmonizing them in a centralized platform becomes increasingly difficult, because the context of the domain and the line of business gets lost. Moreover, as ecosystems grow and you add more data, the processes associated with the centralized platform tend to get further genericized. They again lose that domain specific context. Wait (chuckling), there are more problems. Now, while in theory organizations are optimizing on the piece parts of the pipeline, the reality is, as the domain requires a change, for example, a new data source or an ecosystem partnership requires a change in access or processes that can benefit a domain consumer, the reality is the change is subservient to the dependencies and the need to synchronize across these discrete parts of the pipeline or actually, orthogonal to each of those parts. In other words, in actuality, the monolithic data platform itself remains the most granular part of the system. Now, when I complain about this faulty structure, some folks tell me this problem has been solved. That there are services that allow new data sources to really easily be added. A good example of this is Databricks Ingest, which is, it's an auto loader. And what it does is it simplifies the ingestion into the company's Delta Lake offering. And rather than centralizing in a data warehouse, which struggles to efficiently allow things like Machine Learning frameworks to be incorporated, this feature allows you to put all the data into a centralized datalake. More so the argument goes, that the problem that I see with this, is while the approach does definitely minimizes the complexities of adding new data sources, it still relies on this linear end-to-end process that slows down the introduction of data sources from the domain consumer beside of the pipeline. In other words, the domain experts still has to elbow her way into the front of the line or the pipeline, in this case, to get stuff done. And finally, the way we are organizing teams is a point of contention, and I believe is going to continue to cause problems down the road. Specifically, we've again, we've optimized on technology expertise, where for example, data engineers, well, really good at what they do, they're often removed from the operations of the business. Essentially, we created more silos and organized around technical expertise versus domain knowledge. As an example, a data team has to work with data that is delivered with very little domain specificity, and serves a variety of highly specialized consumption use cases. All right. I want to step back for a minute and talk about some of the problems that people bring up with Snowflake and then I'll relate it back to the basic premise here. As I said earlier, we've been hammered by dozens and dozens of data points, opinions, criticisms of Snowflake. And I'll share a few here. But I'll post a deeper technical analysis from a software engineer that I found to be fairly balanced. There's five Snowflake criticisms that I'll highlight. And there are many more, but here are some that I want to call out. Price transparency. I've had more than a few customers telling me they chose an alternative database because of the unpredictable nature of Snowflake's pricing model. Snowflake, as you probably know, prices based on consumption, just like AWS and other cloud providers. So just like AWS, for example, the bill at the end of the month is sometimes unpredictable. Is this a problem? Yes. But like AWS, I would say, "Kill me with that problem." Look, if users are creating value by using Snowflake, then that's good for the business. But clearly this is a sore point for some users, especially for procurement and finance, which don't like unpredictability. And Snowflake needs to do a better job communicating and managing this issue with tooling that can predict and help better manage costs. Next, workload manage or lack thereof. Look, if you want to isolate higher performance workloads with Snowflake, you just spin up a separate virtual warehouse. It's kind of a brute force approach. It works generally, but it will add expense. I'm kind of reminded of Pure Storage and its approach to storage management. The engineers at Pure, they always design for simplicity, and this is the approach that Snowflake is taking. Usually, Pure and Snowflake, as I have discussed in a moment, is Pure's ascendancy was really based largely on stealing share from Legacy EMC systems. Snowflake, in my view, has a much, much larger incremental market opportunity. Next is caching architecture. You hear this a lot. At the end of the day, Snowflake is based on a caching architecture. And a caching architecture has to be working for some time to optimize performance. Caches work well when the size of the working set is small. Caches generally don't work well when the working set is very, very large. In general, transactional databases have pretty small datasets. And in general, analytics datasets are potentially much larger. Is it Snowflake in the analytics business? Yes. But the good thing that Snowflake has done is they've enabled data sharing, and it's caching architecture serves its customers well because it allows domain experts, you're going to hear this a lot from me today, to isolate and analyze problems or go after opportunities based on tactical needs. That said, very big queries across whole datasets or badly written queries that scan the entire database are not the sweet spot for Snowflake. Another good example would be if you're doing a large audit and you need to analyze a huge, huge dataset. Snowflake's probably not the best solution. Complex joins, you hear this a lot. The working set of complex joins, by definition, are larger. So, see my previous explanation. Read only. Snowflake is pretty much optimized for read only data. Maybe stateless data is a better way of thinking about this. Heavily right intensive workloads are not the wheelhouse of Snowflake. So where this is maybe an issue is real-time decision-making and AI influencing. A number of times, Snowflake, I've talked about this, they might be able to develop products or acquire technology to address this opportunity. Now, I want to explain. These issues would be problematic if Snowflake were just a data warehouse vendor. If that were the case, this company, in my opinion, would hit a wall just like the NPP vendors that proceeded them by building a better mouse trap for certain use cases hit a wall. Rather, my promise in this episode is that the future of data architectures will be really to move away from large centralized warehouses or datalake models to a highly distributed data sharing system that puts power in the hands of domain experts at the line of business. Snowflake is less computationally efficient and less optimized for classic data warehouse work. But it's designed to serve the domain user much more effectively in our view. We believe that Snowflake is optimizing for business effectiveness, essentially. And as I said before, the company can probably do a better job at keeping passionate end users from breaking the bank. But as long as these end users are making money for their companies, I don't think this is going to be a problem. Let's look at the attributes of what we're proposing around this new architecture. We believe we'll see the emergence of a total flip of the centralized and monolithic big data systems that we've known for decades. In this architecture, data is owned by domain-specific business leaders, not technologists. Today, it's not much different in most organizations than it was 20 years ago. If I want to create something of value that requires data, I need to cajole, beg or bribe the technology and the data team to accommodate. The data consumers are subservient to the data pipeline. Whereas in the future, we see the pipeline as a second class citizen, with a domain expert is elevated. In other words, getting the technology and the components of the pipeline to be more efficient is not the key outcome. Rather, the time it takes to envision, create, and monetize a data service is the primary measure. The data teams are cross-functional and live inside the domain versus today's structure where the data team is largely disconnected from the domain consumer. Data in this model, as I said, is not the exhaust coming out of an operational system or an external source that is treated as generic and stuffed into a big data platform. Rather, it's a key ingredient of a service that is domain-driven and monetizable. And the target system is not a warehouse or a lake. It's a collection of connected domain-specific datasets that live in a global mesh. What is a distributed global data mesh? A data mesh is a decentralized architecture that is domain aware. The datasets in the system are purposely designed to support a data service or data product, if you prefer. The ownership of the data resides with the domain experts because they have the most detailed knowledge of the data requirement and its end use. Data in this global mesh is governed and secured, and every user in the mesh can have access to any dataset as long as it's governed according to the edicts of the organization. Now, in this model, the domain expert has access to a self-service and obstructed infrastructure layer that is supported by a cross-functional technology team. Again, the primary measure of success is the time it takes to conceive and deliver a data service that could be monetized. Now, by monetize, we mean a data product or data service that it either cuts cost, it drives revenue, it saves lives, whatever the mission is of the organization. The power of this model is it accelerates the creation of value by putting authority in the hands of those individuals who are closest to the customer and have the most intimate knowledge of how to monetize data. It reduces the diseconomies at scale of having a centralized or a monolithic data architecture. And it scales much better than legacy approaches because the atomic unit is a data domain, not a monolithic warehouse or a lake. Zhamak Dehghani is a software engineer who is attempting to popularize the concept of a global mesh. Her work is outstanding, and it's strengthened our belief that practitioners see this the same way that we do. And to paraphrase her view, "A domain centric system must be secure and governed with standard policies across domains." It has to be trusted. As I said, nobody's going to use data they don't trust. It's got to be discoverable via a data catalog with rich metadata. The data sets have to be self-describing and designed for self-service. Accessibility for all users is crucial as is interoperability, without which distributed systems, as we know, fail. So what does this all have to do with Snowflake? As I said, Snowflake is not just a data warehouse. In our view, it's always had the potential to be more. Our assessment is that attacking the data warehouse use cases, it gave Snowflake a straightforward easy-to-understand narrative that allowed it to get a foothold in the market. Data warehouses are notoriously expensive, cumbersome, and resource intensive, but they're a critical aspect to reporting and analytics. So it was logical for Snowflake to target on-premise legacy data warehouses and their smaller cousins, the datalakes, as early use cases. By putting forth and demonstrating a simple data warehouse alternative that can be spun up quickly, Snowflake was able to gain traction, demonstrate repeatability, and attract the capital necessary to scale to its vision. This chart shows the three layers of Snowflake's architecture that have been well-documented. The separation of compute and storage, and the outer layer of cloud services. But I want to call your attention to the bottom part of the chart, the so-called Cloud Agnostic Layer that Snowflake introduced in 2018. This layer is somewhat misunderstood. Not only did Snowflake make its Cloud-native database compatible to run on AWS than Azure in the 2020 GCP, what Snowflake has done is to obstruct cloud infrastructure complexity and create what it calls the data cloud. What's the data cloud? We don't believe the data cloud is just a marketing term that doesn't have any substance. Just as SAS is Simplified Application Software and iOS made it possible to eliminate the value drain associated with provisioning infrastructure, a data cloud, in concept, can simplify data access, and break down fragmentation and enable shared data across the globe. Snowflake, they have a first mover advantage in this space, and we see a number of fundamental aspects that comprise a data cloud. First, massive scale with virtually unlimited compute and storage resource that are enabled by the public cloud. We talk about this a lot. Second is a data or database architecture that's built to take advantage of native public cloud services. This is why Frank Slootman says, "We've burned the boats. We're not ever doing on-prem. We're all in on cloud and cloud native." Third is an obstruction layer that hides the complexity of infrastructure. and fourth is a governed and secured shared access system where any user in the system, if allowed, can get access to any data in the cloud. So a key enabler of the data cloud is this thing called the global data mesh. Now, earlier this year, Snowflake introduced its global data mesh. Over the course of its recent history, Snowflake has been building out its data cloud by creating data regions, strategically tapping key locations of AWS regions and then adding Azure and GCP. The complexity of the underlying cloud infrastructure has been stripped away to enable self-service, and any Snowflake user becomes part of this global mesh, independent of the cloud that they're on. Okay. So now, let's go back to what we were talking about earlier. Users in this mesh will be our domain owners. They're building monetizable services and products around data. They're most likely dealing with relatively small read only datasets. They can adjust data from any source very easily and quickly set up security and governance to enable data sharing across different parts of an organization, or, very importantly, an ecosystem. Access control and governance is automated. The data sets are addressable. The data owners have clearly defined missions and they own the data through the life cycle. Data that is specific and purposely shaped for their missions. Now, you're probably asking, "What happens to the technical team and the underlying infrastructure and the cluster it's in? How do I get the compute close to the data? And what about data sovereignty and the physical storage later, and the costs?" All these are good questions, and I'm not saying these are trivial. But the answer is these are implementation details that are pushed to a self-service layer managed by a group of engineers that serves the data owners. And as long as the domain expert/data owner is driving monetization, this piece of the puzzle becomes self-funding. As I said before, Snowflake has to help these users to optimize their spend with predictive tooling that aligns spend with value and shows ROI. While there may not be a strong motivation for Snowflake to do this, my belief is that they'd better get good at it or someone else will do it for them and steal their ideas. All right. Let me end with some ETR data to show you just how Snowflake is getting a foothold on the market. Followers of this program know that ETR uses a consistent methodology to go to its practitioner base, its buyer base each quarter and ask them a series of questions. They focus on the areas that the technology buyer is most familiar with, and they ask a series of questions to determine the spending momentum around a company within a specific domain. This chart shows one of my favorite examples. It shows data from the October ETR survey of 1,438 respondents. And it isolates on the data warehouse and database sector. I know I just got through telling you that the world is going to change and Snowflake's not a data warehouse vendor, but there's no construct today in the ETR dataset to cut a data cloud or globally distributed data mesh. So you're going to have to deal with this. What this chart shows is net score in the y-axis. That's a measure of spending velocity, and it's calculated by asking customers, "Are you spending more or less on a particular platform?" And then subtracting the lesses from the mores. It's more granular than that, but that's the basic concept. Now, on the x-axis is market share, which is ETR's measure of pervasiveness in the survey. You can see superimposed in the upper right-hand corner, a table that shows the net score and the shared N for each company. Now, shared N is the number of mentions in the dataset within, in this case, the data warehousing sector. Snowflake, once again, leads all players with a 75% net score. This is a very elevated number and is higher than that of all other players, including the big cloud companies. Now, we've been tracking this for a while, and Snowflake is holding firm on both dimensions. When Snowflake first hit the dataset, it was in the single digits along the horizontal axis and continues to creep to the right as it adds more customers. Now, here's another chart. I call it the wheel chart that breaks down the components of Snowflake's net score or spending momentum. The lime green is new adoption, the forest green is customers spending more than 5%, the gray is flat spend, the pink is declining by more than 5%, and the bright red is retiring the platform. So you can see the trend. It's all momentum for this company. Now, what Snowflake has done is they grabbed a hold of the market by simplifying data warehouse. But the strategic aspect of that is that it enables the data cloud leveraging the global mesh concept. And the company has introduced a data marketplace to facilitate data sharing across ecosystems. This is all about network effects. In the mid to late 1990s, as the internet was being built out, I worked at IDG with Bob Metcalfe, who was the publisher of InfoWorld. During that time, we'd go on speaking tours all over the world, and I would listen very carefully as he applied Metcalfe's law to the internet. Metcalfe's law states that the value of the network is proportional to the square of the number of connected nodes or users on that system. Said another way, while the cost of adding new nodes to a network scales linearly, the consequent value scores scales exponentially. Now, apply that to the data cloud. The marginal cost of adding a user is negligible, practically zero, but the value of being able to access any dataset in the cloud... Well, let me just say this. There's no limitation to the magnitude of the market. My prediction is that this idea of a global mesh will completely change the way leading companies structure their businesses and, particularly, their data architectures. It will be the technologists that serve domain specialists as it should be. Okay. Well, what do you think? DM me @dvellante or email me at david.vellante@siliconangle.com or comment on my LinkedIn? Remember, these episodes are all available as podcasts, so please subscribe wherever you listen. I publish weekly on wikibon.com and siliconangle.com, and don't forget to check out etr.plus for all the survey analysis. This is Dave Vellante for theCUBE Insights powered by ETR. Thanks for watching. Be well, and we'll see you next time. (upbeat music)

(upbeat music) >> From the CUBE studios in Palo Alto, in Boston, connecting with thought leaders all around the world. This is a CUBE conversation. >> Hi everyone, this is Dave Vellante from the CUBE. And we're getting ready for the Snowflake Data cloud summit four geographies, eight tracks more than 40 sessions for this global event. Starts on November 17th, where we're tracking the rise of the Data cloud. You're going to hear a lot about that, now, by now, you know, the story of Snowflake or you know, what maybe you don't but a new type of cloud native database was introduced in the middle part of last decade. And a new set of analytics workloads has emerged that is powering a transformation within organizations. And it's doing this by putting data at the core of businesses and organizations. You know, for years we marched to the cadence of Moore's law. That was the innovation engine of our industry, but now that's changed it's data plus machine intelligence plus cloud. That's the new innovation cocktail for the technology industry and industries overall. And at the Data cloud summit we'll hear from Snowflake executives, founders, technologists, customers, and ecosystems partners. And of course, you going to hear from interviews on the CUBE. So, let's dig in a little bit more and help me are two Snowflake experts. Felipe Hoffa is a data cloud advocate and Kent Graziano is a chief technical evangelist post at Snowflake. Gents, great to see you. Thanks for coming on. >> Yeah, thanks for having us on, this is great. >> Thank you. >> So guys first, I got to congratulate you on getting to this point. You've achieved beyond escape velocity and obviously one of the most important IPOs of the year, but you got a lot of work to do. I know that what, what are the substantive aspects behind the Data cloud? >> I mean, it's a new concept right? We've been talking about infrastructure clouds and SaaS applications living in application clouds and Data cloud is the ability to really share all that data that we've been collected. You know, we've spent what how many a decade or more with big data now but have we been able to use it effectively? And that's really where the Data cloud is coming in and Snowflake and making that a more seamless, friendly, easy experience to get access to the data. I've been in data warehousing for nearly 30 years now. And our dream has always been to be able to augment an organization's analytics with data from outside their organization. And that's just been a massive pain in the neck with having to move files around and replicate the data and maybe losing track of where it came from or where it went. And the Data cloud is really giving our customers the ability to do that in a much more governed way, a much more seamless way and really make it push button to give anyone access to the data they need and have the performance to do the analytics in near real time. It's total game changer is as you already know and just it's crazy what we're able to do today compared it to what we could do when I started out in my career. >> Well, I'm going to come back to that 'cause I want to tap your historical perspective, but Felipe let me ask you, So, why did you join Snowflake? You're you're the newbie here? What attracted you? >> Exactly? I'm the newbie, I used to work at Google until August. I was there for 10 years. I was a developer advocate there also for data you might have heard about the BigQuery. I was doing a lot of that. And then as time went by Snowflake started showing up more and more in my feeds within my customers in my community. And it came the time, well, I felt that like, you know, when wherever you're working, once in a while you think I should leave this place I should try something new, I should move my career forward. While at Google, I thought that so many times, as anyone would do, and it was only when Snowflake showed up, like where Snowflake is going now, why Snowflake is being received by all the customers that I saw this opportunity. And I decided that moving to Snowflake would be a step forward for me. And so far I'm pretty happy, like the timing has been incredible, but more than the timing and everything, it's really, really a great place for data. What I love first is data, sharing data, analyzing data and how Snowflake is doing it's for me to mean phenomenal. >> So, Kent, I want to come back to you and I say tap maybe your historical perspective here. And you said it's always been a dream that you could do these other things bringing in external data. I would say this, that I don't want to push a little bit on this because I have often said that the EDW marketplace really never lived up to its promises of 360 degree views of the customer real time or near real time analytics. And, and it really has been as you kind of described are a real challenge for a lot of organizations. When Hadoop came in we got excited that it was going to actually finally live up to that vision and, and duped it a lot and don't get me wrong, I mean, the whole concept of bring that compute to data and lowering the cost and so forth. But it certainly didn't minimize complexity. And, and it seems like, feels like Snowflake is on the cusp of actually delivering on that promise that we've been talking about for 30 years. I wonder, if you could share your perspective is it, are we going to get there this time? >> Yeah. And as far as I can tell working with all of our customers some of them are there. I mean, they thought through those struggles that you were talking about that I saw throughout my career and now with getting on Snowflake they're delivering customer 360 they're integrating weblogs and IOT data with structured data from their ERP systems or CRM systems, their supply chain systems. And it really is coming to fruition. I mean, the industry leaders, you know, Bill Inman and Claudia Imhoff, they've had this vision the whole time but the technology just wasn't able to support it. And the cloud, as we said about the internet, changed everything. And then Ben wine teary, and they're in their vision and building the system, taking the best concepts from the Hadoop world and the data Lake world and the enterprise data warehouse world and putting it all together into this, this architecture that's now Snowflake and the Data cloud solve it. I mean, it's the classic benefit of hindsight is 2020 after years in the industry, they'd seen these problems and said like, how can we solve them? Does the Cloud let us solve these problems? And the answer was yes, but it did require writing everything from scratch and starting over with, because the architecture of the Cloud just allows you to do things that you just couldn't do before. >> Yeah. I'm glad you brought up you know, some of the originators of the data warehouse because it really wasn't their fault. They were trying to solve a problem. It was the marketers that took it and really kind of made promises that they couldn't keep. But, the reality is when you talk to customers in the so old EDW days and this is the other thing I want to tap you guys' brains on. It was very challenging. I mean, one customer one time referred to it as a snake, swallowing a basketball. And what he meant by that is every time there's a change Sarbanes Oxley comes and we have to ingest all this new data. It's like, Oh, it's to say everything slows down to a grinding halt. Every time Intel came out with a new microprocessor, they would go out and grab a new server as fast as they possibly could. He called it chasing the chips and it was this endless cycle of pain. And so, you know, the originators of the data whereas they didn't have the compute power they didn't have the Cloud. And so, and of course they didn't have the 30, 40 years of pain to draw upon. But I wonder if you could, could maybe talk a little bit about the kinds of things that can be done now that we haven't been able to do here to form. >> Well, yeah. I remember early on having a conversation with Bill about this idea of near real time data warehousing and saying, is this real, is this something really people need? And at the time he was a couple of decades ago, he said now to them they just want to load their data sooner than once a month. That was the goal. And that was going to be near real time for them. And, but now I'm seeing it with our customers. It's like, now we can do it, you know, with things like the Kafka technology and snow pipe in Snowflake that people are able to get that refresh way faster and have near real time analytics access to that data in a much more timely manner. And so it really is coming true. And the, the compute power that's there, as you said, we've now got this compute power in the Cloud that we never dreamed of. I mean, you would think of only certain, very large, massive global companies or governments could afford super computers. And that's what it would have taken. And now we've got nearly the power of a super computer in our mobile device that we all carry around with us. So being able to harness all that now in the Cloud is really opening up opportunities to do things with data and access data in a way that, again really, we just kind of dreamed of before as like we can democratize data when we get to this point. And I think that's where we are. We're at that inflection point where now it's possible to do it. So the challenge on organizations is going to be how do we do it effectively? How do we do it with agility? And how do we do it in a governed manner? You mentioned Sarbanes Oxley, GDPR, CCPA, all of those are out there. And so we have all of that as well. And so that's where we're going to get into it, right into the governance and being able to do that in a very quick, flexible, extensible manner and Snowflakes really letting people do it now. >> Well, yeah. And you know, again, we've been talking about Hadoop and I, again, for all my fond thoughts of that era, and it's not like Hadoop is gone but it was a lot of excitement around it, but governance was a huge problem. And it was kind of a bolt on. Now, Felipe I going to ask you, like, when you think about a company like Google, your former employer, you know, data is at the core of their business. And so many companies the data is not at the core of their business. Something else is, it's a process or a manufacturing facility or whatever it is. And the data is sort of on the outskirts. You know, we often talk about in, in stove pipes. And so we're now seeing organizations really put data at the core of their, it becomes central to their DNA. I'm curious as to your thoughts on that. And also, if you've got a lot of experience with developers, is there a developer angle here in this new data world? >> For sure, I mean, I love seeing everything like throughout my career at Google and my two months here and talking to so many companies, you never thought before like these are database companies but they are the ones that keep rowing. The ones that keep moving to the next stage of their development is because they are focusing on data. They are adapting the processes, they are learning from it. Me, I focus a lot on developers. So, I met when I started this career as an advocate of first, I was a software engineer and my work so far, has we worked, I really loved talking to the engineers on the other companies. Like, maybe I'm not the one solving the business problem, but at the end of the day, when these companies have a business problem that they want to grow, they want to have data. There are other engineers that are scientists like me that want to work for the company and bring the best technology to solve the problems. And Yeah, there's so much where data can help, yes, as we evolved the system for the company, and also for us, for understanding the systems things like of survivability, and recently there was a big company a big launch on survivability (indistinct) whether they are running all of their data warehousing needs. And all of that needs on snowflake, just because running these massive systems and being able to see how they're working generates a lot of data. And then how do you manage it? How do you analyze it? Or Snowflake is really there to help cover the two areas. >> It's interesting my business partner, John farrier cohost of the CUBE, he said, gosh I would say middle of the last decade, maybe even around the time 2013, when Snowflake was just coming out, he said, he predicted the data would be the new development kit. And it's really at the center of a lot of the data life cycle the what I call the data pipelines. I know people use that term differently but I'm very excited about the Data cloud summit and what we're going to learn there. And I get to interview a lot of really cool people. So, I appreciate you guys coming up, but, Kent who should attend the Data cloud summit, I mean, what should they expect to learn? >> Well, as you said earlier, Dave, there's so many tracks and there's really kind of something for everyone. So, we've got a track on unlocking the value of the Data cloud, which is really going to speak to the business leaders, you know, as to what that vision is, what can we do from an organizational perspective with the Data cloud to get that value from the data to move our businesses forward. But we've also done for the technicians migrating to snowflake. Sessions on how to do the migration, modernizing your data Lake, data science, how to do analytics with the, and data science in Snowflake and in the Data cloud, and even down to building apps. So the developers and building data products. So, you know, we've got stuff for developers, we've got stuff for data scientists. We've got stuff for the data architects like myself and the data engineers on how to build all of this out. And then there's going to be some industry solution spotlights as well. So we can talk about different verticals folks in FinTech and healthcare, there's going to be stuff for them. And then for our data superheroes we have a hallway track where we're going to get talks from the folks that are in our data superheroes which is really our community advocacy program. So these are folks who are out there in the trenches using Snowflake delivering value at their organizations. And they're going to talk down and dirty. How did they make this stuff happen? So it's going to be to some hope, really something for everyone, fireside chats with our executives. Of course something I'm really looking forward to myself. So was fun to hear from Frank and Christian and Benoit about what's the next big thing, what are we doing now? Where are we going with all of this? And then there is going to be a some awards we'll be giving out our data driver awards for our most innovative customers. So this is going to be a lot, a lot for everybody to consume and enjoy and learn about this, this new space of, of the Data cloud. >> Well, thank you for that Kent. And I'll second that, at least there's going to be a lot for everybody. If you're an existing Snowflake customer there's going to be plenty of two or one content, we can get in to the how to use and the best practice, if you're really not that familiar with Snowflake, or you're not a customer, there's a lot of one-on-one content going on. So, Felipe, I'd love to hear from you what people can expect at the Data cloud summit. >> Totally, so I would like to plus one to everyone that can say we have a phenomenal schedule that they, the executive will be there. I really wanted to especially highlight the session I'm preparing with Trevor Noah. I'm sure you might have heard of him. And we are having him at the Data cloud summit and we are going to have a session. We are going to talk about data. We are preparing a session. That's all about how people that love data that people that want to make that actionable. How can they bring storytelling and make it more, have more impact as he has well learn to do through his life? >> That's awesome, So, we have Trevor Noah, we're not just going to totally geek out here. we're going to have some great entertainment as well. So, I want you to go to snowflake.com and click on Data cloud summit 2020 there's four geos. It starts on November 17th and then runs through the week and in the following week in Japan. So, so check that out. We'll see you there. This is Dave Vellante for the CUBE. Thanks for watching. (upbeat music)

>> (Instructor)From the cube studios in Palo Alto, in Boston, connecting with thought leaders all around the world. This is a cube conversation. >> Hi everyone. This is Dave Volante, the cube, and we're getting ready for the snowflake data cloud summit four geographies eight tracks, more than 40 sessions for this global event starts on November 17th, where we're tracking the rise of the data cloud. You're going to hear a lot about that now by now, you know the story of Snowflake or you know, what maybe you don't, but a new type of cloud native database was introduced in the middle part of last decade. And a new set of analytics workloads has emerged that is powering a transformation within organizations. And it's doing this by putting data at the core of businesses and organizations. You know for years, we marched to the cadence of Moore's law. That was the innovation engine of our industry, but now that's changed it's data plus machine intelligence plus cloud. That's the new innovation cocktail for the technology industry and industries overall. And at the data cloud summit, we'll hear from snowflake executives, founders, technologists, customers, and ecosystems partners. And of course, you're going to hear from interviews on the cube. So let's dig in a little bit more and to help me, are two snowflake experts, Filipe Hoffa is a data cloud advocate and Kent Graziano is a chief technical evangelists post at Snowflake. Gents great to see you. Thanks for coming on. >> Yeah thanks for having us on this is great. >> Thank you. >> So guys, first, I got to congratulate you on getting to this point. You've achieved beyond escape velocity, and obviously one of the most important IPOs of the year, but you got a lot of work to do I know that Filipe, let me start with you data cloud. What's a data cloud and what are we going to learn about it at the data cloud summit? >> Oh, that's an excellent question. And let me tell you a little bit about our story here. And I really, really, really admire what Kent has done. I joined the snowflake like less than two months ago, and for me it's been a huge learning experience. And I look up to Kent a lot on how we deliver the message and how do we deliver all of that. So I would love to hear his answer first. >> Okay, that's cool. Okay Kent later on. So talk of data cloud, that's a catchy phrase, right? But it vectors into at least two of the components of my innovation, innovation cocktail. What, what are the substantive substantive aspects behind the data cloud? >> I mean, it's a, it's a new concept, right? We've been talking about infrastructure clouds and SAS applications living in an application clouds so data cloud is the ability to really share all that data that we've been collecting. You know, we've, we've spent what, how many days a decade or more with big data now, but have we been able to use it effectively? And that's, that's really where the data cloud is coming in and snowflake in making that a more seamless, friendly, easy experience to get access to the data. I've been in data warehousing for nearly 30 years now. And our dream has always been to be able to augment an organization's analytics with data from outside their organization. And that's just been a massive pain in the neck with having to move files around and replicate the data and maybe losing track of where it came from or where it went. And the data cloud is really giving our customers the ability to do that in a much more governed way, a much more seamless way, and really make it push button to give anyone access to the data they need and have the performance to do the analytics in near real time. It's it's total game changer as, as you already know, and just it's crazy what we're able to do today, compared to what we could do when I started out in my career. >> Well, I'm going to come back to that cause I want to tap your historical perspective, but Filipe, let me ask you. So why did you join snowflake? You're you're the newbie here. What attracted you? >> Exactly, I'm the newbie. I used to work at Google until August. I was there for 10 years. I was a developer advocate there also for data, you might have heard about a big query. I was doing a lot of that and then as time went by, Snowflake started showing up more and more in my feeds, within my customers, in my community. And it came the time. When, I felt that like, you know, when wherever you're working, once in a while you think I should leave this place, I should try something new. I should move my career forward. While at Google, I thought that so many times as anyone would do, and it was only when snowflake showed up, like where snowflake is going now, how snowflake is, is being received by all the customers that I saw this opportunity. And I decided that moving to Snowflake would be a step forward for me. And so far I'm pretty happy. Like the timing has been incredible, but more than the timing and everything, it's really, really a great place for data. What I love first is data sharing data, analyzing data and how Snowflake is doing it it promotes me in phenomena. >> So Ken, I want to come back to you and I say, tap, maybe your historical perspective here. And you said, you know, it's always been a dream that you could do these other things bring in external data. I would say this, that I don't want to push a little bit on this because I have often said that the EDW marketplace really never lived up to its promises of 360 degree views of the customer in real time or near real time analytics. And, and it really has been, as you kind of described are a real challenge for a lot of organizations when Hadoop came in you know, we had, we we we got excited that it was kind of going to actually finally live up to that vision and and and we duped it a lot. And it don't get me wrong. I mean, the whole concept of, you know, bring the compute to data and the lowering the cost and so forth, but it certainly didn't minimize complexity. And, and it seems like, feels like Snowflake is on the cusp of actually delivering that promise that we've been talking about for 30 years. I wonder if you could share your perspective, is it, are we going to get there this time? >> Yeah. And as far as I can tell working with all of our customers, some of them are there. I mean, they're, they Fought through those struggles that you were talking about that I saw throughout my career and now with getting on Snowflake they're, they're delivering customer 360, they're integrating weblogs and IOT data with structured data from their ERP systems or CRM systems, their supply chain systems. And it really is coming to fruition. I mean, the, you know, the industry leaders, you know, Bill Inman and Claudia M Hoff, they've had this vision the whole time, but the technology just wasn't able to support it. And the cloud, as we said about the internet, changed everything and then Ben Y and Terry, in their vision and building the system, taking the best concepts from the Hadoop world and the data Lake world and the enterprise data warehouse world, and putting it all together into this, this architecture, that's now, you know Snowflake and the data cloud solved it. I mean, it's the, you know, the, the classic benefit of her insight is 2020 after years in the industry, they had seen these problems and said like, how can we solve them? Does the cloud let us solve these problems? And the answer was yes, but it did require writing everything from scratch and starting over with because the architecture the cloud just allows you to do things that you just couldn't do before. Yeah I'm glad you brought up, you know, some of the originators of the data warehouse, because it really wasn't their fault. They were trying to solve a problem. That was the marketers that took it and really kind of made promises that they couldn't keep. But the reality is when you talk to customers in the, in the, so the old EDW days, and this is the other thing I want to, I want to tap your guys' brains on. It was very challenging. I mean, one, one customer, one time referred to it as a snake, swallowing a basketball. And what he meant by that is you know, every time there's a change, you know, Sarbanes Oxley comes and we have to ingest all this new data. It's like, Oh, it's just everything slows down to a grinding halt. Every time Intel came out with a new microprocessor, they would go out and grab a new server as fast as they possibly could. He called it chasing the chips, and it was this endless cycle of pain. And so, you know, the originators of the data whereas they didn't, they didn't have you know the compute power, they didn't have the cloud. >> Yeah. >> And so, and of course they didn't have the 30- 40 years of pain to draw upon. But, but I wonder if you could, could maybe talk a little bit about the kinds of things that can be done now that we haven't been able to do here before. >> Well, yeah I remember early on having a conversation with, with Bill about this idea of near real time data warehousing and saying, is this real? Is this something really need people need? And at the time it was, was a couple of decades ago, he said no to them they just want to load their data sooner than once a month. >> Yeah. >> That was the goal. And that was going to be near real time for them. And, but now I'm seeing it with our customers. It's like, now we can do it, you know, with things like the Kafka technology and snow pipe in, in Snowflake, that people are able to get that refresh way faster and have near real time analytics access to that data in a much more timely manner. And so it really is coming true. And the, the compute power that's there, as you said, you know we, we've now got this compute power in the cloud that we never dreamed of. I mean, you would think of only certain very large, massive global companies or governments could afford supercomputers. And that's what it would have taken. And now we've got nearly the power of a supercomputer in our mobile device that we all carry around with us. So being able to harness all that now in the cloud is really opening up opportunities to do things with data and access data in a way that again really we just kind of dreamed of before. It's like, we can, we can democratize data when we get to this point. And I think that's the, that's where we are, we're at that inflection point where now it's, it's possible to do it. So the challenge on organizations is going to be, how do we do it effectively? How do we do it with agility? And how do we do it in a governed manner? You mentioned Sarbanes Oxley, GDPR, CCPA, all of those are out there. And so we have all of that as well. And so that's where, that's where we're going to get into it, right. Is into the governance and being able to do that in a very quick, flexible, extensible manner and you know, Snowflakes really letting people do it now. >> Well, yeah and you know, again, we've been talking about Hadoop and again, for all my, my fond thoughts of that era, and it's not like hadoop is gone, but, but it was a lot of excitement around it but but governance was a huge problem and it was kind of a ball tough enough. Felipe I got to ask you, like when you think about a company like Google your former employer, you know, data is at the core of their business. And so many companies, the data is not at the core of their business. Something else is it's a process or a manufacturing facility or you know whatever it is. And the data is sort of on the outskirts. You know, we often talk about in, in stove pipes. And so we're now seeing organizations really put data at the core of their it becomes, you know, central to their, to their DNA. I'm curious as to your thoughts on that. And also if you've got a lot of experience with developers, is there, is there a developer angle here in this new data world? >> Oh, for sure. I mean, I love seeing every, like throughout my career at Google and my two months here and talking to so many companies, you never thought before, like these are database companies, but the the ones that keep rowing. The ones that keep moving to the next stage of their development is because they are focusing on data. They are adapting the processes they learning from it. And me, I focus a lot on developers. So I mean when I started This career as an advocate. First I was a software engineer and my work so far, has been work, I really loved talking to the engineers on the other companies. Like maybe I'm not the one solving the business problem, but at the end of the day, when these companies have a business problem that they want to row, they want to have data. There are other engineers that are scientists likes me that are, that, that want to work for work for the company and bring the best technology to solve the problems. Yeah, there's so much where data can help as we evolve the system for the company. And also for us for understanding the systems, things like observability and recently, there was a big company, a big launch on observability the company name is observable, where they are running all of their data warehousing needs. And all of their data needs on Snowflake, just because running these massive systems and being able to see how they're working generates a lot of data. And then how do you manage it? How do you analyze it? Or snowflake is already there to help. >> Well you know >> I covered the two areas. >> It's interesting my, my business partner, John farrier, cohost of the cube, he said, gosh, I would say middle of the last decade, maybe even around the time, you know, 2013, when Snowflake was just coming out, he said, he predicted the data would be the new development kit. And you know, it's really at the center of a lot of, you know, the data life cycle, the, the, what I call the data pipelines. I know people use that term differently, but, but I'm, I'm very excited about the data cloud summit and what we're going to learn there. And I get to interview a lot of really cool people. And so I appreciate you guys coming on, but Kent, who, who should attend the data cloud summit, I mean, what, what are the, what should they expect to learn? >> Well, as you said earlier, Dave, there's, there's so many tracks and there's really kind of something for everyone. So we've got a track on unlocking the value of the data cloud, which is really going to speak to, you know, the business leaders, you know, as to what that vision is, what can we do from an organizational perspective, with the data cloud to get that value from the data to, to move our businesses forward. But we've also got, you know, for the technicians migrating to Snowflake training sessions on how to do the migration, modernizing your data like data science, you know how to do analytics with the, and data science in Snowflake and in the data cloud and even down to building apps. So the developers and building data products. So, you know, we've got stuff for developers, we've got stuff for data scientists. We've got stuff for the, the data architects like myself and the data engineers on how to, how to build all of this out. And then there's going to be some industry solutions spotlights as well. So we can talk about different verticals of folks in FinTech and, and in healthcare. There's going to be stuff for them. And then for our, our data superheroes, we have a hallway track where we're going to get talks from the folks that are in our data superheroes, which is really our community advocacy program. So these are folks who are out there in the trenches using Snowflake, delivering value at, at their organizations. And they're going to talk you know down and dirty. How did they make this stuff happen? So there's going to be just really something for everyone, fireside chats with our executives, of course, something I'm really looking forward to in myself. It's always fun to, to hear from Frank and Christian. And Benwah about, you know, what's the next big thing, you know, what are we doing now? Where are we going with all of this? And then there is going to be some awards. We'll be giving out our data driver awards for our most innovative customers. So this is going to be a lot, a lot for everybody to consume and enjoy and learn about this, this new space of, of the data cloud. >> Well, thank you for that Kent. And I'll second that, I mean, there's going to be a lot for everybody. If you're an existing Snowflake customer, there's going to be plenty of two on one content we can get in to the how to's and the best practice. If you're really not that familiar with Snowflake, or you're not a customer, there's a lot of one-on-one content going on. If you're an investor and you want to figure out, okay, what is this vision? And can, you know, will this company grow into its massive valuation and how are they going to do that? I think you're going to, you're going to hear about the data cloud and really try get a perspective. And you can make your own judgment as to, to, you know, whether or not you think that it's going to be as large a market as many people think. So Felipe, I'd love to hear from you what people can expect at the data cloud summit. >> Totally, so I would love to plus one to everyone that Kent said. We have a phenomenal schedule that the the executive will be there. And I really wanted to specially highlight the session I'm preparing with Trevor Noah. I'm sure you might have heard of him. And we are having him at the data cloud summit, and we are going to have a session. We're going to talk about data. We are preparing a session, That's all about how people that love data, that people that want to make data actionable. How can they bring storytelling and make it more, have more impact as he has well learned to do through his life. >> That's awesome, So yeah, Trevor Noah, we're not just going to totally geek out here. We're going to, we're going to have some great entertainment as well. So I want you to go to snowflake.com and click on data cloud summit, 2020 there's four geos. It starts on November 17th and then runs through the week and then the following week in Japan. So, so check that out. We'll see you there. This is Dave Volante for the cube. Thanks for watching. (soft music)

>>from the Cube Studios in Palo Alto in Boston, connecting with thought leaders all around >>the world. This is a cute conversation high, but this is Day Volonte. And as you know, we've been tracking the next generation of clouds. Sometimes we call it Cloud to two point. Frank's Lukman is here to really unpack this with me. Frank. Great to see you. Thanks for coming on. >>Yeah, you as well. They could see it >>s o obviously hot off your AIPO A lot of buzz around that. Uh, that's fine. We could we could talk about that, but I really want to talk about the future. What? Before we get off the I p o. That was something you told me when you're CEO service. Now you said, hey, we're priced to perfection, so it looks like snowflakes gonna be priced to perfection. It's a marathon, though. You You made that clear. I presume it's not any different here for you. Yeah, >>well, I think you know the service now. Journey was different in the sense that we were kind of under the underdogs, and people sort of discovered over the years the full potential of the company and I think there's stuff like they pretty much discovered a day. One. It's a little bit more, More sometimes it's nice to be an underdog. Were a bit of an over dog in this, uh, this particular scenario, but, you know, it is what it is, Andre. You know, it's all about execution delivering the results, delivering on our vision, Uh, you know, being great with our customers. And, uh, hopefully the chips will fall where they where they may. At that point, >>yeah, you're you're You're a poorly kept secret at this point, Frank. After a while, I wanted, you know, I've got some excerpts of your book that that I've been reading. And, of course, I've been following your career since the two thousands. You're off sailing. You mentioned in your book that you were kind of retired. You were done, and then you get sucked back in now. Why? I mean, are you in this for the sport? What's the story here? >>Uh, actually, that that's not a bad way of characterizing it. I think I am in that, uh, you know, for the sport, uh, you know the only way to become the best version of yourself is to be to be under the gun and, uh, you know, every single day. And that's that's certainly what we are. It sort of has its own rewards building great products, building great companies, regardless off you know what the spoils. Maybe it has its own rewards. And I It's hard for people like us to get off the field and, you know, hang it up. So here we are. >>You know, you're putting forth this vision now the data cloud, which obviously it's good marketing, but I'm really happy because I don't like the term Enterprise Data Warehouse. I don't think it reflects what you're trying to accomplish. E D. W. It's slow on Lee. A few people really know how to use it. The time value of data is gone by the time you know, your business is moving faster than the data in the D. W. And it really became a savior because of Sarbanes Oxley. That's really what it came a reporting mechanism. So I've never seen What you guys are doing is is e d w. So I want you to talk about the data cloud. I want to get into the to the vision a little bit and maybe challenge you on a couple things so our audience can better understand it. Yes. So >>the notion of a data cloud is is actually, uh, you know, type of cloud that we haven't had. I mean, data has been been fragmented and locked up in a million different places in different clouds. Different cloud regions, obviously on premise, um, And for data science teams, you know, they're trying thio drive analysis across datasets, which is incredibly hard, Which is why you know, a lot of this resorts to, you know, programming on bond things of that sort of. ITT's hardly scalable because the data is not optimized. The economics are not optimized. There's no governance model and so on. But a data cloud is actually the ability thio loosely couple and lightly Federated uh, data, regardless of where it is. So it doesn't have scale limitations or performance limitations. Uh, the way traditional data warehouses have had it. So we really have a fighting chance off really killing the silos and unlocking the bunkers and allowing the full promise of data sciences and ml On day I thio really happen. I mean, a lot of lot of the analysis that happens on data is on the single data set because it's just too damn hard, you know, to drive analysis across multiple data sets. And, you know, when we talk to our customers, they have very precise designs on what they're trying to do. They say, Look, we are trying to discover, you know, through through through deep learning You know what the patterns are that lead to transactions. You know, whether it's if you're streaming company. Maybe it's that you're signing up for a channel or you're buying a movie or whatever it is. What is the pattern you know, of data points that leads us to that desired outcome. Once you have a very accurate description of the data relationships, you know that results in that outcome, you can then search for it and scale it, you know, tens of million times over. That's what digital enterprises do, right? So in order to discover these patterns enriched the data to the point where the patterns become incredibly predictive. Uh, that's that's what snowflake is formed, right? But it requires a completely Federated Data mo because you're not gonna find a data pattern in the in the single data set per se right? So that's that's what it's all about. I mean, the outcomes of a data cloud are very, very closely related to the business outcomes that the user is seeking, right? It's not some infrastructure process. It has a very remote relationship with business outcome. This is very, very closely related. >>So it doesn't take a brain surgeon to look at the Trillion Years Club. And so I could see that I could see the big you know, trillion dollars apple $2 trillion market cap companies. They got data at the core, whereas most companies most incumbents. Yeah, it might be a bottling plant that the core, some manufacturing or some other processes they put, they put data around it in these silos. It seems like you're trying toe really? Bring that innovation and put data at the core. And you've got an architecture to do that. You talk about your multi cluster shared storage architecture. You mentioned you mentioned data sharing it. Will this, in your opinion, enable, for instance, incumbents to do what a lot of the startups were able to do with the cloud days? I mean they got access to data centers, which they they couldn't have before the cloud you're trying to do with something similar with data. >>Yeah, so So, you know, obviously there's no doubt that the cloud is a critical enabler. This wouldn't be happening. Uh, you know what? I was at the same time, the trails that have been blessed by the likes of Facebook and Google. Uh, e the reason those enterprises are so extraordinary valuable is is because of what they know. Uh, you know, through data and how they can monetize what they know through data. But that is now because that power is now becoming available, you know, to every single enterprise out there. Right, Because the data platform, the underlying cloud capabilities, we are now delivering that to anybody who wants it. Now, you still need to have strong date engineering data science capabilities. It's not like falling off a log, but fundamentally, those capabilities are now, you know, broadly accessible in the marketplace. >>So we're talking upfront about some of the differences between what you've done earlier in your career. Like I said, you're the worst kept secret, you know, Data domain. I would say it was sort of somewhat of a niche market. You you blew it up until it was very disruptive, but it was somewhat limited in what could be done. Uh, and and maybe some of that limitation, you know, wouldn't have occurred if you stay the price, uh, independent company service. Now you mop the table up because you really had no competition there, Not the case here. You you've got some of the biggest competitors in the world, so talk about that. And what gives you confidence that you can continue to dominate, >>But, you know, it's actually interesting that you bring up these companies. I mean, data. The man was a scenario where we were constrained on market and literally we were a data backup company. As you recall, we needed to move into backup software. Need to move the primary storage. While we knew it, we couldn't execute on it because it took tremendous resource is which, back in the day, it was much harder than one of this right now. So we ended up selling the company to E M. C and and now part of Dell. But way short, uh, we're left with some trauma from that experience, Uh, that, you know, why couldn't we, you know, execute on that transformation? So coming to service now, we were extremely. I'm certainly need personally, extremely attuned to the challenges that we have endured in our prior company. One of the reasons why you saw service now break out at scale at tremendous growth rights is because of what we have learned from the prior journey. We're not gonna ever get caught again in a situation where we could not sustain our markets and sustain our growth. So if service I was very much the execution model was very much a reaction to what we had encountered in the prior company. Now coming into snowflake totally different deal. Because not only is there's a large market, this is a developing market. I think you've pointed out in some of your broadcasting that this market is very much in flux on the reason is that you know, technology is now capable of doing things for for people and enterprises that they could never do before. So people are spending way mawr resource is than they ever thought possible on these new capability. So you can't think in terms of static markets and static data definitions, it means nothing. Okay, These things are so in transition right now, it's very difficult for people you know to to scope that the scale of this opportunity. >>Yeah. I wanna understand you're thinking around and, you know, I've written about the TAM, and can Snowflake grow into its valuation and the way I drew it, I said, Okay, you got data Lakes and you got Enterprise Data Warehouse. That's pretty well understood. But I called it data as a service to cover the closest analogy to your data cloud. And then even beyond that, when you start bringing in the edge and real time data, uh, talk about how you're thinking about that, Tam. And what what you have to do to participate. You have toe, you know, bring adjacent capabilities, ISAT this read data sharing that will get you there. In other words, you're not like a transaction system. You hear people talking about converge databases, you hear? Talk about real time inference at the edge that today anyway, isn't what snowflake is about. Does that vision of data sharing and the data cloud does that allow you to participate in that massive, multi $100 billion tam that that I laid out and probably others as well. >>Yeah, well, it is always difficult. Thio defined markets based on historical concept that probably not gonna apply whole lot for much longer. I mean, the way we think of it is that data is the beating heart of the digital enterprise on, uh, you know, digital enterprises today. What do you look at? People against the car door dash or so on. Um, they were built from the ground up to be digital on the prices and data Is the beating heart off their operation Data operations is their manufacturing, if you will, um, every other enterprise out there is is working very hard to become digital or part digital and is going to learn to develop data platforms like what we're talking about here to data Cloud Azaz. Well, as the expertise in terms of data engineering and data scientist to really fully become a digital enterprise, right. So, you know, we view data as driving operations off the digital enterprise. That's really what it iss right data, and it's completely data driven. And there's no people involved. People are developing and supporting the process. But in the execution, it is end to end. Data driven. Being that data is the is the signal that initiates the process is technol assess. Their there being a detective, and then they fully execute the entire machinery probe Problematic machinery, if you will, um, you know, of the processes that have been designed, for example, you know, I may fit a certain pattern. You know, that that leads to some transactional context. But I've not fully completed that pattern until I click on some Lincoln. And all of a sudden proof I have become, you know, a prime prospect system, the text that in the real time and then unleashes Oh, it's outreach and capabilities to get me to transact me. You and I are experiencing this every day. You know, when we're when we're online, you just may not fully re election. That's what's happening behind the scenes. That's really what this is all about. So and so to me, this is sort of the new online transaction processing is enter and, uh, you know, data digital. Uh, no process that is continually acquiring, analyzing and acting on data. >>Well, you've talked about the time time value of of data. It loses value over time. And to the extent that you can actually affect decisions, maybe before you lose the customer before you lose the patient even even more importantly or before you lose the battle. Uh, there's all kinds of, you know, mental models that you can apply this. So automation is a key part of that. And then again, I think a lot of people like you said, if you just try to look at historical markets, you can't really squint through those and apply them. You really have toe open up your mind and think about the new possibilities. And so I could see your your component of automation. I I see what's happening in the r P. A space and and I could see See these this massive opportunities Thio really change society, change business, your last thoughts. >>There's just there's just no scenario that I can envision where data is not completely core in central to a digital enterprise, period. >>Yeah, I think I really do think, Frank, your your your Your vision is misunderstood somewhat. I think people say Okay. Hey, we'll bet on salute men Scarpelli the team. That's great to do that. But I think this is gonna unfold in a way that people may be having predicted that maybe you guys, yourselves and your founders, you know, haven't have aren't able to predict as well. But you've got that good, strong architectural philosophy that you're pursuing and it just kind of feels right, doesn't it? >>You know, I mean, one of the 100 conversations and, uh, you know, things is the one of the reasons why we also wrote our book. You know, the rights of the data cloud is to convey to the marketplace that this is not an incremental evolution, that this is not sort of building on the past. There is a real step function here on the way to think about it is that typically enterprises and institutions will look at a platform like snowflakes from a workload context. In other words, I have this business. I have this workload. This is very much historically defined, by the way. And then they benchmark us, you know, against what they're what they're already doing on some legacy platform. And they decided, like, Yeah, this is a good fit. We're gonna put Snowflake here. Maybe there, but it's still very workload centric, which means that we are essentially perpetuating the mentality off the past. Right? We were doing it. Wanna work, load of the time We're creating the new silos and the new bunkers of data in the process. And we're really not approaching this with level of vision that the data science is really required to drive maximum benefit from data. So our arguments and this is this is not an easy arguments is to say, toc IOS on any other sea level person that wants to listen to that look, you know, just thinking about, you know, operational context and operational. Excellent. It's like we have toe have a platform that allows us unfettered access to the data that, you know, we may need to, you know, bring the analytical power to right. If you have to bring in political power to a diversity of data sets, how are we going to do that right? The data lives in, like, 500 different places. It's just not possible, right, other than with insane amounts of programming and complexity, and then we don't have the performance, and we don't have to economics, and we don't have the governance and so on. So you really want to set yourself up with a data cloud so that you can unleash your data science, uh, capabilities, your machine learning your deep learning capabilities, aan den, you really get the full throttle advantage. You know of what the technology can do if you're going to perpetuate the silo and bunkering of data by doing it won't work. Load of the time. You know, 5, 10 years from now, we're having the same conversation we've been having over the last 40 years, you know? >>Yeah. Operationalize ing your data is gonna require busting down those those silos, and it's gonna require something like the data cloud to really power that to the next decade and beyond. Frank's movement Thanks so much for coming in. The Cuban helping us do a preview here of what's to come. >>You bet, Dave. Thanks. >>All right. Thank you for watching. Everybody says Dave Volonte for the Cube will see you next time

>> Narrator: From theCUBE Studios in Palo Alto in Boston, connecting with thought leaders all around the world, this is theCUBE Conversation. >> Hi everyone, this is Dave Vellante at theCUBE, and we're getting ready for the Snowflake Data Cloud Summit. four geographies, eight tracks, more than 40 sessions for this global event. starts on November 17th, where we're tracking the rise of the data cloud. You're going to hear a lot about that. Now, by now, you know the story of Snowflake or, you know what? Maybe you don't. But a new type of cloud-native database was introduced in the middle part of the last decade. And a new set of analytics workloads has emerged, that is powering a transformation within the organizations. And it's doing this by putting data at the core of businesses and organizations. For years, we marched to the cadence of Moore's law. That was the innovation engine of our industry, but now that's changed. It's data, plus machine intelligence, plus cloud. That's the new innovation cocktail for the technology industry and industries overall. And in the Data Cloud Summit, we'll hear from Snowflake executives, founders, technologists, customers, and ecosystems partners. And of course, you're going to hear from interviews on theCUBE. So let's dig in a little bit more. And to help me are two Snowflake experts. Felipe Hoffa is a data cloud advocate and Kent Graziano is a chief technical evangelist, both at Snowflake. Gents, great to see you, thanks for coming on. >> Thanks for having us on, this is great. >> Thank you. >> So guys, first, I got to congratulate you on getting to this point. You've achieved beyond escape velocity and obviously one of the most important IPOs of the year, but you got a lot of work to do and I know that. Felipe, let me start with you. Data cloud, what's a data cloud and what are we going to learn about it at the Data Cloud Summit? >> Oh, that's an excellent question. And, let me tell you a little bit about our story here. And I really, really, really admire what Kent has done. I joined Snowflake like less than two months ago and for me, it's been a huge learning experience. And I look up to Kent a lot on how we deliver the method here, how do we deliver all of that? So, I would love to hear his answer first. >> Dave: Okay, that's cool. Okay Kent, leader on. (Kent laughing) So we took it. Data cloud, that's a catchy phrase, right? But it vectors into at least two of the components of my innovation cocktail. What are the substantive aspects behind the data cloud? >> I mean, it's a new concept, right? We've been talking about infrastructure clouds and SaaS applications living in the application cloud, so data cloud is the ability to really share all that data that we've been collecting. We've spent what? How many da-- A decade or more with big data now, but have we been able to use it effectively? And that's really where the data cloud is coming in and Snowflake, in making that a more seamless, friendly, easy experience to get access to the data. I've been in data warehousing for nearly 30 years now. And our dream has always been to be able to augment an organization's analytics with data from outside their organization. And that's just been a massive pain in the neck with having to move files around and replicate the data and maybe losing track of where it came from or where it went. And the data cloud is really giving our customers the ability to do that in a much more governed way, a much more seamless way, and really make it push button to give anyone access to the data they need and have the performance to do the analytics in near real-time. It's a total game changer as you already know. And just, it's crazy what we're able to do today compared to what we could do when I started out in my career. >> Well, I'm going to come back to that 'cause I want to tap your historical perspective. But Felipe, let me ask you, so why did you join Snowflake? You're the newbie here, what attracted you? >> And finally, I'm the newbie. I used to work at Google until August. I was there for 10 years, I was a developer advocate there also for data, you might have heard about the BigQuery, I was doing a lot of that. And though as time went by, Snowflake started showing up more and more in my feeds, within my customers, in my community. And it came the time when I felt like-- Wherever you're working, once in a while you think, "I should leave this place, "I should try something new, "I should move my career forward." While at Google, I thought that so many times as anyone will do. And it was only when Snowflake showed up, like where Snowflake is going now, how Snowflake is being received by all the customers, that I saw this opportunity. And I decided that moving to Snowflake would be a step forward for me. And so far I'm pretty happy, like the timing has been incredible, but more than the timing and everything, it's really, really a great place for data. What I love first is data, sharing data, analyzing data and how Snowflake is doing it, its promising phenomena. >> So, Kent, I want to come back to you and I said, tap maybe your historical perspective here. And you said, it's always been a dream that you could do these other things, bring in external data. I would say this, that I would want to push a little bit on this because I have often said that the EDW marketplace really never lived up to its promises of 360 degree views of the customer, in real-time or near real-time analytics. And it really has been, as you kind of described it, a real challenge for a lot of organizations. When Hadoop came in, we had-- We got excited that it was going to actually finally live up to that vision and Hadoop did a lot. And don't get me wrong, I mean, the whole concept of, bring the computer data and lowering the cost and so forth. But it certainly didn't minimize complexity. And it seems like, feels like Snowflake is on the cusp of actually delivering on that promise that we've been talking about for 30 years. I wonder if you could share your perspective as an o-- Are we going to get there this time? >> Yeah. And as far as I can tell working with all of our customers, some of them are there. I mean, they thought through those struggles that you were talking about, that I saw throughout my career. And now with getting on Snowflake they're delivering customer 360, they're integrating weblogs and IOT data with structured data from their ERP systems or CRM systems, their supply chain systems and it really is coming to fruition. I mean, the industry leaders, Bill Inmon and Claudia Imhoff, they've had this vision the whole time, but the technology just wasn't able to support it and the cloud, as we said about the internet, changed everything. And then Benoit and Thierry in their vision in building the system, taking the best concepts from the Hadoop world and the data lake world and the enterprise data warehouse world, and putting it all together into this architecture, that's now Snowflake and the data cloud, solved it. I mean, it's-- The classic benefit of hindsight is 20/20, after years in the industry, they had seen these problems and said like, "How can we solve them? "Does the cloud let us solve these problems?" And the answer was, yes, but it did require writing everything from scratch and starting over with, because the architecture of the cloud just allows you to do things that you just couldn't do before. >> Yeah, I'm glad you brought up some of the originators of the data warehouse, because it really wasn't their fault, they were trying to solve a problem. It was the marketers that took it and really kind of made promises that they couldn't keep. But, the reality is when you talk to customers in the sort of the old EDW days, and this is the other thing I want to tap you guys' brains on, it was very challenging. I mean, and one customer one time referred to it as a snake swallowing a basketball. And what he meant by that is, every time there's a change, or Sarbanes-Oxley comes and we have to ingest all this new data. It's like aargh! It's just everything slows down to a grinding halt. Every time Intel came out with a new microprocessor they would go out and grab a new server as fast as they possibly could, he called it chasing the chips. And it was this endless cycle of pain. And so, the originators of the data warehouse, they didn't have the compute power, they didn't have the cloud. And so-- And of course they didn't have like 30, 40 years of pain to draw upon. But I wonder if you could maybe talk a little bit about the kinds of things that can be done now that we haven't been able to do here tofore. >> Well, yeah. I remember early on having a conversation with Bill about this idea of near real-time data warehousing and saying, "Is this real? "Is this something really people need?" And at the time, it was a couple of decades ago, he said, "No, to them, they just want to load their data "sooner than once a month." That was the goal. And they-- That was going to be near real-time for them. And, but now I'm seeing it with our customers. It's like, now we can do it. With things like the Kafka technology and Snowpipe in Snowflake, that people are able to get that refresh way faster and have near real-time analytics access to that data in a much more timely manner. And so it really is coming true. And the compute power that's there, as you said, we've now got this compute power in the cloud that we never dreamed of. I mean, you would think of only certain, very large, massive global companies or governments could afford supercomputers. And that's what it would have taken. And now we've got nearly the power of a super computer in our mobile device that we all carry around with us. So being able to harness all of that now in the cloud, is really opening up opportunities to do things with data and access data in a way that, again, really, we just kind of dreamed of before. Its like, we can democratize data when we get to this point. And I think that's where we are, we're at that inflection point, where now it's possible to do it. So the challenge on organizations is going to be how do we do it effectively? How do we do it with agility? And how do we do it in a governed manner? You mentioned Sarbanes-Oxley, GDPR, CCPA, all of those are out there. And so we have all of that as well. And so that's where we're going to get into it, ride us into the governance and being able to do that in a very quick, flexible, extensible manner. And Snowflakes really letting people do it now. >> Well, yeah. And again, we've been talking about Hadoop, and again, for all my fond thoughts of that era, and it's not like Hadoop is gone, but there was a lot of excitement around it, but governance was a huge problem. And it was kind of a bolt on. And now, Felipe I got to ask you, when you think about a company like Google, your former employer, data is at the core of their business. And so many companies, the data is not at the core of their business, something else is, it's a process or a manufacturing facility or whatever it is. And the data is sort of on the outskirts. We often talk about in stovepipes. And so we're now seeing organizations really, put data at the core of their... And it becomes central to their DNA. I'm curious as to your thoughts on that. And also, if you've got a lot of experience with developers, is there a developer angle here in this new data world? >> Oh, for sure. I mean, I love seeing every-- Like throughout my career at Google and my two months here, I'm talking to so many companies, that you never thought before, like these are database companies. But the ones that keep growing, the ones that keep moving to the next stage of their development is because they are focusing on data, they are adopting the processes, They are learning from it. And, me per-- I focus a lot on developers, so I mean, when I started this career as an advocate, first, I was a software engineer. And my work so far, has been... (mumbles) I really love talking to the engineers on the other companies, like... Maybe I'm not the one solving the business problem, but at the end of the day, when these companies have a business problem through out the world, they want to have data. There are other engineers that are scientists like me that are... That want to work for the company and bring the best technology to solve the problems. Yeah, for example, there's so much where data can help. If, as we evolve the systems for the company and also for us for understanding these systems, things like observability. And recently, there was a big company, a big launch on observability, on the company names of Cyberroam, where they are running all of their data warehousing needs and all of their data needs on Snowflake. Just because running these massive systems and being able to see how they're working, generates a lot of data. And then how do you manage it? How do you analyze it? Snowflake is ready there to help and support the two areas. >> It's interesting, my business partner, John Furrier, co-host of theCUBE, he said, gosh, I would say the middle of the last decade, maybe even around the time, 2013, when Snowflake was just coming out. He said... He predicted that data would be the new development kit. And, it's really at the center of a lot of the data life cycle, the-- What I call the data pipelines, I know people use that term differently. But, I'm very excited about the Data Cloud Summit and what we're going to learn there. And I get to interview a lot of really cool people. And so I appreciate you guys coming on. But Kent, who should attend the Data Cloud Summit? I mean, what are the-- What should they expect to learn? >> Well, as you said earlier Dave, there's so many tracks and there's really kind of something for everyone. So we've got a track on unlocking the value of the data cloud, which is really going to speak to the business leaders, as to what that vision is, what can we do from an organizational perspective with the data cloud to get them value from the data to move our businesses forward? But we've also got for the technicians, migrating to Snowflake. Training sessions on how to do the migration and modernizing your data lake, data science. How to do analytics with, and data science in Snowflake and in the data cloud. And even down to building apps, for the developers and building data products. So, we've got stuff for developers, we've got stuff for data scientists, we've got stuff for the data architects like myself and the data engineers, on how to build all of this out. And then there's going to be some industry solutions spotlights as well. So we can talk about different verticals, folks in FinTech and in healthcare, there's going to be stuff for them. And then for our data superheroes, we have a hallway track where we're going to get talks from the folks that are in our data superheroes, which is really our community advocacy program. So these are folks that are out there in the trenches using Snowflake, delivering value at their organizations. And they're going to talk down and dirty of how did they make this stuff happen? So there's going to be just really, something for everyone. Fireside chats with our executives, of course, something I'm really looking forward to myself. It's always fun to hear from Frank and Christian and Benoit, about what's the next big thing, what are we doing now? Where are we going with all of this? And then there is going to be some awards. We'll be giving out our Data Driver Awards for our most innovative customers. So there's going to be a lot for everybody to consume and enjoy and learn about this new space of the data cloud. >> Well, thank you for that Kent and I'll second that, and there's going to be a lot for everybody. If you're an existing Snowflake customer, there's going to be plenty of two of one content, where we can get in to the how tos and the best practice. If you're really not that familiar with Snowflake or you're not a customer, there's a lot of one-on-one content going on. If you're an investor and you want to figure out, "Okay, what is this vision? "And can, will this company grow into its massive valuation? "And how are they going to do that?" I think you're going to hear about the data cloud and really try to get a perspective and you can make your own judgment as to whether or not you think that it's going to be as large a market as many people think. So Felipe, I'd love to hear from you what people can expect at the Data Cloud Summit. >> Totally. So I would love to plus one to every one that Kent said, we have a phenomenal schedule that day, the executives will be there. But I really wanted to especially highlight the session I'm preparing with Trevor Noah. I'm sure you must have heard of him. And we are having him at the Data Cloud Summit, and we are going to have a session. We are going to talk about data. We are preparing a session that's all about how people that love data, that people that want to make that actionable, how can they bring storytelling and make it have more impact as he has well learned to do through his life. >> That's awesome. So, yeah, Trevor Noah, we're not just going to totally geek out here. We're going to have some great entertainment as well. So I want you to go to snowflake.com and click on Data Cloud Summit 2020. There's four geos. It starts on November 17th and then runs through the week and then the following week in Japan. So, check that out, we'll see you there. This is Dave Vellante for theCUBE. Thanks for watching. (upbeat music)

>> Narrator: From "theCUBE" studios in Palo Alto, in Boston, connecting with thought leaders all around the world, this is "theCUBE" conversation. >> Hi everybody, this is Dave Vellante. and as you know, we've been tracking the next generation of cloud. Sometimes we call it cloud 2.0, Frank Slootman is here to really unpack this with me. Frank, great to see you. Thanks for coming on. >> Yeah, you as well Dave, good to see you. >> Yeah so, obviously hot off your IPO, a lot of buzz around that, that's fine. We could talk about that, but I really want to talk about the future. What, before we get off the IPO though, was something you told me when you were CEO of ServiceNow. You said, "Hey, we're priced to perfection." So it looks like Snowflake is going to be priced to perfection, it's a marathon though. You made that clear. I presume it's not any different here for you. >> Well, I think, you know, the ServiceNow journey was different in the sense that we were kind of underdogs and people sort of discovered over the years, the full potential of the company. And, I think with Snowflake, they pretty much just discovered it day one (laughs). It's a little bit more, sometimes it's nice to be an underdog or a bit of an overdog in this particular scenario, but yeah, it is what it is. And, it's all about execution, delivering the results, being great with our customers and, hopefully the (indistinct) where they may at that point. >> Yeah, you're a poorly kept secret at this point, Frank, after a while. I've got some excerpts of your book that I've been reading and of course I've been following your career since the 2000's. You're off sailing. You mentioned in your book that you were kind of retired, you were done, and then you got sucked back in. Now, why? Are you in this for the sport? What's the story here? >> Actually, that's not a bad way of characterizing it. I think I am in it, for the sport, the only way to become the best version of yourself is to be under the gun, every single day. And that's certainly what we are. It sort of has its own rewards, building great products, building great companies, regardless of what the spoils may be, it has its own reward. It's hard for people like us to get off the field and hang it up, so here we are. >> You're putting forth this vision now, the data cloud, which obviously it's good marketing, but I'm really happy because I don't like the term enterprise data warehouse. I don't think it reflects, what you're trying to accomplish. EDW, it's slow, only a few people really know how to use it, the time value of data is gone by the time, your business is moving faster than the data in EDW. And it really became, as the savior because of Sarbanes-Oxley, that's really, what became of a reporting mechanism. So I have never seen what you guys are doing as EDW. So I want you to talk about the data cloud, I want to get into the vision a little bit and maybe challenge you on a couple things so our audience can better understand it. >> Yeah, so the notion of a data cloud is actually a type of cloud that we haven't had. Data has been fragmented and locked up in a million different places, in different clouds, different cloud regions, obviously, on premise. And for data science teams, they're trying to drive analysis across datasets, which is incredibly hard, which is why a lot of this resorts to programming and things of that sort. It's hardly scalable because the data is not optimized, the economics are not optimized, there's no governance model and so on. But the data cloud is actually the ability to loosely couple and lightly federate data, regardless of where it is, so it doesn't have scale limitations or performance limitations the way traditional data warehouses have had it. So we really have a fighting chance of really killing the silos and unlocking the bunkers and allowing the full promise of data sciences and ML and AI to really happen. A` lot of the analysis that happens on data is on a single dataset because it's just too damn hard to drive analysis across multiple datasets. When we talk to our customers, they have very precise designs on what they're trying to do. They say, "Look, we are trying to discover through deep learning what the patterns are that lead to transactions, whether it's... If you're a streaming company, maybe it's that you're signing up for a channel or you're buying a movie or whatever it is. What is the pattern of datapoints that leads us to that desired outcome?" Once you have a very accurate description of the data relationships that result in that outcome, you can then search for it and scale it tens of million times over. That's what digital enterprises do, right? So in order to discover these patterns, enrich the data to the point where the patterns become incredibly predictive, that's what Snowflake is for, right? But it requires a completely federated data model because you're not going to find a data pattern in a single dataset, per se, right? So that's what it's all about. The outcomes of a data cloud are very, very closely related to the business outcomes that the user is seeking, right? It's not some infrastructure process. that has a very remote relationship with business outcome. This is very, very closely related. >> So it doesn't take a brain surgeon to look at the trillionaires' club. (chuckles) So I can see that. I can see the big trillion dollars, Apple, $2 trillion market cap companies, they get data at the core. Whereas most companies, most incumbents, it might be a bottling plant at the core or some manufacturing or some of the process, and they put data rounded in these silos. It seems like you're trying to really bring that innovation and put data at the core and you've got an architecture to do that. You're talking about your multi cluster shared storage architecture. You mentioned data sharing. Will this, in your opinion enable, for instance, incumbents to do what a lot of the startups were able to do with the cloud days. hey got access to data centers which they couldn't have before the cloud. Are you trying to do something similar with data? >> Yeah, so obviously there's no doubt that the cloud is a critical enabler. This wouldn't be happening without it. At the same time, the trails that have been blazed by Alexa, Facebook and Google. The reason that those enterprises are so extraordinarily valuable is because of what they know through data and how they can monetize what they know through data. But that power is now becoming available to every single enterprise out there, right. Because the data platforms and the underlying cloud capabilities, we are now delivering that to anybody who wants it. Now you still need to have strong data engineering, data science capabilities. It's not like falling off of a log, but fundamentally those capabilities are now broadly accessible in the marketplace. >> So we talking up front about some of the differences between what you've done early in your career, like I said, you're the worst kept secret, Data Domain I would say it was somewhat of a niche market. You blew it up until it was very disruptive, but it was somewhat limited in what could be done. And maybe some of that limitation, you know, wouldn't have occurred if you stayed an independent company. ServiceNow, you mopped the table up 'cause you really had no competition there. Not the case here. You've got some of the biggest competitors in the world. So talk about that and what gives you confidence that you can continue to dominate? >> It's actually interesting that you bring up these companies. Data Domain, it was a scenario where we were constrained on market and we were a data backup company as you recall, we needed to move into backup software, needed to move into primary storage. While we knew it, we couldn't execute on it because it took tremendous resources which, back in the day, it was much harder than what it is right now. So we ended up selling the company to EMC and now part of Dell, but we're left with some trauma from that experience in the sense that, why couldn't we execute on that transformation? So coming to ServiceNow, we were extremely, and certainly me personally, extremely attuned to the challenges that we had endured in our prior company, and one of the reasons why you saw ServiceNow break out at scale, at tremendous growth rates is because of what we learned from the prior journey. We were not going to ever get caught again in the situation where we could not sustain our markets and sustain our growth. So ServiceNow is very much, the execution model, very much a reaction to what we had encountered in the prior company. Now coming into Snowflake a totally different deal because not only is this a large market this is a developing market. I think you've pointed out in some of your broadcasting, that this market is very much influx. And the reason is that technology is now capable of doing things for people and enterprises that they could never do before. So people are spending way more resources than they ever thought possible on these new capabilities. So you can't think in terms of static markets and static data definitions, it means nothing. Okay, these things are so in transition right now. It's very difficult for people to scope the scale of this opportunity. >> Yeah, I want to understand your thinking around and, you know, I've written about the TAM and can Snowflake grow into it's valuation and the way I drew it, I said, okay, I've got data lakes and you've got an enterprise data warehouse, that's pretty well understood but I called it data as a service company the closest analogy to your data cloud. And then even beyond that when you start bringing in the Edge and real time data. Talk about how you're thinking about that TAM what you have to do to participate. Do you have to bring adjacent capabilities? Or is it this read data sharing that will get you there? In other words, you're not like a transaction system. You hear people talking about converged databases. You're going to talk about real time inference at the Edge that today anyway, isn't what Snowflake is about. Does that vision of data sharing in the data cloud, does that allow you to participate in that massive multi hundred billion dollar TAM that I laid out and probably others as well? >> Yeah, well, it's always difficult to define markets based on historical concept that probably not going to apply a whole lot or for much longer. I mean the way we think of it is that data is the beating heart of the digital enterprise and digital enterprises today, what are you looking at people like the car door dash or so on. They were built from the ground up to be digital enterprises. And data is the beating heart of their operation, data operations is their manufacturing if you will. Every other enterprise out there is working very hard to become digital or part digital and is going to learn to develop a data platform like what we're talking about here to data cloud as well as the expertise in terms of data engineering and data sciences to really fully become a digital enterprise, right? So we view data as driving operations, all the all the digital enterprise, that's really what it is, right? And it's completely data driven end-to-end. There's no people involved and the people are developing and supporting the process but in the execution, it is end-to-end data driven. Meaning that data is the signal that initiates the process he's taking, but as they're, as they're being detected, and then they fully execute the entire machinery, programmatic machinery if you will, all of the processes have been designed. Now for example, I may fit a certain pattern, that leads to some transactional law context, but that's not fully completed, that pattern until I click on some link and all of a sudden, poof, I have become a prime prospect. System detects that in the real time and then unleashes all its outreach and capabilities to get me to transact. You and I are experiencing this every day. When we're, when we're online, you just may not fully realize (laughs) that that's what's happening behind the scenes. That's really what this is all about. So to me, this is sort of the new online transaction processing is an end to end data digital process that is continually acquiring, analyzing and acting on data. >> Well, you've talked about the time, time value of, of data. It loses value over time. And to the extent that you can actually affect decisions, maybe prior, before you lose the customer, before you lose the patient, even even more importantly, or before you lose the battle. There's all kinds of mental models that you can apply this. So automation is a key part of that and then again, I think a lot of people, like you said, if you just try to look at historical markets, you can't really squint through those and apply them. You really have to open up your mind and think about the new possibilities. And so I could see >> Exactly. >> Your component of automation. I see what's happening in the RPA space, and I could see these just massive opportunities to really change society, change business. Your last thoughts. >> While there's just no scenario that I can envision where data is not completely core and central to a digital enterprise period. >> Yeah, I think I really do think Frank, your vision is misunderstood somewhat. I think people say, "Okay hey, we'll bet on Slootman, "Scott Pelley, the team." That's great to do that, but I think this is going to unfold in a way that people maybe haven't predicted and maybe you guys yourselves and your founders you know haven't, aren't able to predict as well, but you've got that good, strong architectural philosophy that you're pursuing and it just kind of feels right, doesn't it? >> One of the harder conversations and the this is one of the reasons why we also wrote our book "The Rise of the Data Cloud" is to convey to the marketplace that this is not an incremental evolution. It is just not sort of building on the past. There is a real step function here. And the way to think about it is that typically enterprises and institutions will look at a platform like Snowflake from a workload context. In other words, I have this business, I have this workload, which is very much historically defined by the way, and then they benchmark us against what they're already doing on some legacy platform and they decide, "Yeah, this is a good fit, we're going to put Snowflake here, maybe there." But, it's still very workload centric which means that we are, essentially, perpetuating the mentality of the past. We were doing it one workload at a time, we're creating the new silos and the new bunkers of data in the process. And we're really not approaching this with the level of vision that the data scientists really require to drive maximum benefit from data. So our argument, and this is not an easy argument, is to say to CIOs and any other C-level person that wants to listen and say, "Look, just thinking about operational context and operational excellence, it's like we have to have a platform that allows use unfettered access to the data that we may need to bring the analytical power to." If you have to bring analytical power to a diversity of datasets, how are we going to do that? The data lives in 500 different places, it's just not possible, other than with insane amounts of programming and complexity and then we don't have the performance and we don't have the economics and we don't have the governance and so on. So you really want to set yourself up with a data cloud so that you can unleash your data science capabilities, your machine learning, your deep learning capabilities and then really get the full throttle advantage of what the technology can do. If you going to perpetuate the silo-ing and bunkering of data by doing it one workload at a time, five, 10 years from now, we're having the same conversations we've been having over the last 40 years. >> Yeah, operationalizing your data is going to require busting down those silos and it's going to require something like the data cloud to really power that to the next decade and beyond. Frank Slootman, thanks so much for coming to "theCUBE" and helping us do a preview here of what's to come. >> You bet Dave, thanks. >> All right, thank you for watching everybody. This is Dave Vellante from the "theCUBE". We'll see you next time.

(enlightening music) >> From theCUBE's studios in Palo Alto and Boston, connecting with thought leaders all around the world, this is a CUBE conversation. >> As we've been reporting, there's a new class of workloads emerging in the cloud. Early cloud was all about IaaS, spinning up storage, compute, and networking infrastructure to support startups, SaaS, easy experimentation, dev test, and increasingly moving business workloads into the cloud. Modern cloud workloads are combining data. They're infusing machine intelligence into application's AI. They're simplifying analytics and scaling with the cloud to deliver business insights in near real time. And at the center of this mega trend is a new class of data stores and analytic databases, what some called data warehouses, a term that I think is outdated really for today's speed of doing business. Welcome to this week's Wikibon CUBE Insights, powered by ETR. In this breaking analysis, we update our view of the emerging cloud native analytic database market. Today, we want to do three things. First, we'll update you on the basics of this market, what you really need to know in the space. The next thing we're going to do, take a look into the competitive environment, and as always, we'll dig into the ETR spending data to see which companies have the momentum in the market, and maybe, ahead of some of the others. Finally, we're going to close with some thoughts on how the competitive landscape is likely to evolve. And we want to answer the question will the cloud giants overwhelm the upstarts, or will the specialists continue to thrive? Let's take a look at some of the basics of this market. We're seeing the evolution of the enterprise data warehouse market space. It's an area that has been critical to supporting reporting and governance requirements for companies, especially post Sarbanes-Oxley, right? However, historically, as I've said many times, EDW has failed to deliver on its promises of a 360-degree view of the business and real-time customer insights. Classic enterprise data warehouses are too cumbersome, they're too complicated, they're too slow, and don't keep pace with the speed of the business. Now, EDW is about a $20 billion market, but the analytic database opportunity in the cloud, we think is much larger, why is that? It's because cloud computing unlocks the ability to rapidly combine multiple data sources, bring data science tooling into the mix, very quickly analyze data, and deliver insights to the business. More importantly, even more importantly, allow a line of business pros to access data in a self service mode. It's a new paradigm that uses the notion of DevOps as applied to the data pipeline, agile data or what we sometimes called DataOps. This is a highly competitive marketplace. In the early part of last decade, you saw Google bring BigQuery to market, Snowflake was founded, AWS did a one-time license deal to acquire the IP to ParAccel, an MPP database, on which it built Redshift. In the latter part of the decade, Microsoft threw his hat in the ring with SQL DW, which Microsoft has now evolved into Azure Synapse. They did so at the Build conference, a few weeks ago. There are other players as well like IBM. So you can see, there's a lot at stake here. The cloud vendors want your data, because they understand this is one of the key ingredients of the next decade of innovation. No longer is Moore's Law, the mainspring of growth. We've said this many times. Rather today, it's data driven, and AI to push insights and scale with the cloud. Here's the interesting dynamic that is emerging in the space. Snowflake is a cloud specialist in this field, having raised more than a billion dollars in venture, a billion four, a billion five. And it's up against the big cloud players, who are moving fast and often stealing moves from Snowflake and driving customers to their respective platforms. Here's an example that we reported on at last year's re:Invent. It's an article by Tony Baer. He wrote this on ZDNet talking about how AWS RA3 separates compute from storage, and of course, this was a founding architectural principle for Snowflake. Here's another example from the information. They were reporting on Microsoft here turning up the heat on Snowflake. And you can see the highlighted text, where the author talks about Microsoft trying to divert customers to its database. So you got this weird dynamic going on. Snowflake doesn't run on-prem, it only runs in the cloud. Runs on AWS, runs on Azure, runs on GCP. The cloud players again, they all want your data to go into their database. So they want you to put their data into their respective platforms. At the same time, they need SaaS ISVs to run in the cloud because it sells infrastructure services. So, is Snowflake, are they going to pivot to run on-prem to try to differentiate from the cloud giants? I asked Frank Slootman, Snowflake's CEO, about the on-prem opportunity, and his perspective earlier this year. Let's listen to what he said. >> Okay, we're not doing this endless hedging that people have done for 20 years, sort of keeping a leg in both worlds. Forget it, this will only work in the public cloud because this is how the utility model works, right? I think everybody is coming to this realization, right? I mean the excuses are running out at this point. We think that it'll, people will come to the public cloud a lot sooner than we will ever come to the private cloud. It's not that we can't run a private cloud, it just diminishes the potential and the value that we that we bring. >> Okay, so pretty definitive statements by Slootman. Now, the question I want to pose today is can Snowflake compete, given the conventional wisdom that we saw in the media articles that the cloud players are going to hurt Snowflake in this market. And if so, how will they compete? Well, let's see what the customers are saying and bring in some ETR survey data. This chart shows two of our favorite metrics from the ETR data set. That is Net Score, which is on the y-axis. Net Score, remember is a measure of spending momentum and market share, which is on the x-axis. Market share is a measure of pervasiveness in the data set. And what we show here are some of the key players in the EDW and cloud native analytic database market. I'll make a couple of points, and we'll dig into this a little bit further. First thing I want to share is you can see from this data, this is the April ETR survey, which was taken at the height of the US lockdown for the pandemic. The survey captured respondents from more than 1,200 CIOs and IT buyers, asking about their spending intentions for analytic databases for the companies that we show here on this kind of x-y chart. So the higher the company is on the vertical axis, the stronger the spending momentum relative to last year, and you could see Snowflake has a 77% Net Score. It leaves all players with AWS Redshift showing very strong, as well. Now in the box in the lower right, you see a chart. Those are the exact Net Scores for all the vendors in the Shared N. A Shared N is a number of citations for that vendor within the N of the 1,269. So you can see the N's are quite large, certainly large enough to feel comfortable with some of the conclusions that we're going to make today. Microsoft, they have a huge footprint. And they somewhat skew the data with its very high market share due to its volume. And you could see where Google sits, it's at good momentum, not as much presence in the marketplace. We've also added a couple of on-prem vendors, Teradata and Oracle primarily on-prem, just for context. They're two companies that compete, they obviously have some cloud offerings, but again, most of their base is on-prem. So what I want to do now is drill into this a little bit more by looking at Snowflake within the individual clouds. So let's look at Snowflake inside of AWS. That's what this next chart shows. So it's customer spending momentum Net Score inside of AWS accounts. And we cut the data to isolate those ETR survey respondents running AWS, so there's an N there of 672 that you can see. The bars show the Net Score granularity for Snowflake and Amazon Redshift. Now, note that we show 96 Shared N responses for Snowflake and 213 for Redshift within the overall N of 672 AWS accounts. The colors show 2020 spending intentions relative to 2019. So let's read left to right here. The replacements are red. And then, the bright red, then, you see spending less by 6% or more, that's the pinkish, and then, flat spending, the gray, increasing spending by more than 6%, that's the forest green, and then, adding to the platform new, that's the lime green. Now, remember Net Score is derived by subtracting the reds from the greens. And you can see that Snowflake has more spending momentum in the AWS cloud than Amazon Redshift, by a small margin, but look at, 80% of the AWS accounts plan to spend more on Snowflake with 35%, they're adding new. Very strong, 76% of AWS customers plan to spend more in 2020 relative to 2019 on Redshift with only 12% adding the platform new. But nonetheless, both are very, very strong, and you can see here, the key point is minimal red and pink, but not a lot of people leaving, not a lot of people spending less. It's going to be critical to see in the June ETR survey, which is in the field this month, if Snowflake is able to hold on to these new accounts that it's gained in the last couple of months. Now, let's look at how Snowflake is doing inside of Azure and compare it to Microsoft. So here's the data from the ETR survey, same view of the data here except we isolate on Azure accounts. The N there is 677 Azure accounts. And we show Snowflake and Microsoft cuts for analytic databases with 83 and 393 Shared N responses respectively. So again, enough I feel to draw some conclusions from this data. Now, note the Net Scores. Snowflake again, winning with 78% versus 51% from Microsoft. 51% is strong but 78% is there's a meaningful lead for Snowflake within the Microsoft base, very interesting. And once again, you see massive new ads, 41% for Snowflake, whereas Microsoft's Net Score is being powered really by growth from existing customers, that forest green. And again, very little red for both companies. So super positive there. Okay, let's take a look now at how Snowflake's doing inside of Google accounts, GCP, Google Cloud Platform. So here's the ETR data, same view of that data, but now, we isolate on GCP accounts. There are fewer, 298 running, then, you got those running Snowflake and Google Analytic databases, largely BigQuery, but could be some others in there but the Snowflake Shared N is 49, it's smaller than on the other clouds, because the company just announced support for GCP, just about a year ago. I think it was last June, but still large enough to draw conclusions from the data. I feel pretty comfortable with that. We're not slicing and dicing it too finely. And you could see Google Shared N at 147. Look at the story. I sound like a broken record. Snowflake is again winning by a meaningful margin if you measure this Net Score or spending momentum. So 77.6% Net Score versus Google at 54%, with Snowflake at 80% in the green. Both companies, very little red. So this is pretty impressive. Snowflake has greater spending momentum than the captive cloud providers in all three of the big US-based clouds. So the big question is can Snowflake hold serve, and continue to grow, and how are they going to to be able to do that? Look, as I said before, this is a very competitive market. We reported that how Snowflake is taking share from some of the legacy on-prem data warehouse players like Teradata and IBM, and from what our data suggests, Lumen and Oracle too. I've reported how IBM is stretched thin on its research and development budget, spends about $6 billion a year, but it's got to spend it across a lot of different lines. Oracle's got more targeted spending R&D. They can target more toward database and direct more of its free cash flow to database than IBM can. But Amazon, and Microsoft, and Google, they don't have that problem. They spend a ton of dough on R&D. And here's an example of the challenge that Snowflake faces. Take a look at this partial list that I drew together of recent innovations. And we show here a set of features that Snowflake has launched in 2020, and AWS since re:Invent last year. I don't have time to go into these, but we do know this that AWS is no slouch at adding features. Amazon, as a company, spends two x more on research and development than Snowflake is worth as a company. So why do I like Snowflake's chances. Well, there are several reasons. First, every dime that Snowflake spends on R&D, go-to market, and ecosystem, goes into making its databases better for its customers. Now, I asked Frank Slootman in the middle of the lockdown how he was allocating precious capital during the pandemic. Let's listen to his response. I've said, there's no layoffs on our radar, number one. Number two, we are hiring. And number three is, we have a higher level of scrutiny on the hires that we're making. And I am very transparent. In other words, I tell people, "Look, I prioritize the roles that are closest "to the drivetrain of the business." Right, it's kind of common sense. But I wanted to make sure that this is how we're thinking about this. There are some roles that are more postponable than others. I'm hiring in engineering, without any reservation because that is the long term, strategic interest of the company. >> But you know, that's only part of the story. And so I want to spend a moment here on some other differentiation, which is multi-cloud. Now, as many of you know, I've been sort of cynical of multi-cloud up until recently. I've said that multi-cloud is a symptom, more of a symptom of multi-vendor and largely, a bunch of vendor marketing hooey today. But that's beginning to change. I see multi-cloud as increasingly viable and important to organizations, not only because CIOs are being asked to clean up the crime scene, as I've often joked, but also because it's increasingly becoming a strategy, right cloud for the right workload. So first, let me reiterate what I said at the top. New workloads are emerging in the cloud, real-time AI, insights extraction, and real-time inferencing is going to be a competitive differentiator. It's all about the data. The new innovation cocktail stems from machine intelligence applied to that data with data science tooling and simplified interfaces that enable scaling with the cloud. You got to have simplicity if you're going to scale and cloud is the best way to scale. It's really the only way to scale globally. So as such, we see cross-cloud exploitation is a real differentiator for Snowflake and others that build high quality cloud native capabilities from multiple clouds, and I want to spend a minute on this topic generally and talk about what it means for Snowflake specifically. Now, we've been pounding the table lately saying that building capabilities natively for the cloud versus putting a wrapper around your stack and making it run in the cloud is key. It's a big difference, why is this? Because cloud native means taking advantage of the primitive capabilities within respective clouds to create the highest performance, the lowest latency, the most efficient services, for that cloud, and the most secure, really exploiting that cloud. And this is enabled only by natively building in the cloud, and that's why Slootman is so dogmatic on this issue. Multi-cloud can be a differentiator for Snowflake. We can think about data lives everywhere. And you want to keep data, where it lives ideally, you don't want to have to move it, whether it's on AWS, Azure, whatever cloud is holding that data. If the answer to your query requires tapping data that lives in multiple clouds across a data network, and the app needs fast answers, then, you need low latency access to that data. So here's what I think. I think Snowflake's game is to automate by extracting, abstracting, sorry, the complexity around the data location, of course, latency is a part of that, metadata, bandwidth concerns, the time to get to query and answers. All those factors that build complexity into the data pipeline and then optimizing that to get insights, irrespective of data location. So a differentiating formula is really to not only be the best analytic database but be cloud agnostic. AWS, for example, they got a cloud agenda, as do Azure and GCP. Their number one answer to multi-cloud is put everything on our cloud. Yeah, Microsoft and Google Anthos, they would argue against that but we know that behind the scenes, that's what they want. They got offerings across clouds but Snowflake is going to make this a top priority. They can lead with that, and they must be best at it. And if Snowflake can do this, it's going to have a very successful future, in our opinion. And by all accounts, and the data that we shared, Snowflake is executing well. All right, so that's a wrap for this week's CUBE Insights, powered by ETR. Don't forget, all these breaking analysis segments are available as podcasts, just Google breaking analysis with Dave Vellante. I publish every week on wikibon.com and siliconangle.com. Check out etr.plus. That's where all the survey data is and reach out to me, I'm @dvellante on Twitter, or you can hit me up on my LinkedIn posts, or email me at david.vellante@siliconangle.com. Thanks for watching, everyone. We'll see you next time. (enlightening music)

>>Bye from San Francisco. It's the cube covering RSA conference, 2020 San Francisco brought to you by Silicon angle media. >>Hey, welcome back. Everyone's keeps coverage here in San Francisco at the Moscone center for RSA conference 2020 I'm John, your host, as cybersecurity goes to the next generation as the new cloud scale, cyber threats are out there, the real impact a company's business and society will be determined by the industry. This technology and the people that a cube alumni here, caramel Jaffer, SVP, senior vice president of strategy and corporate development for iron net. Welcome back. Thanks to Shawn. Good to be here. Thanks for having so iron net FC general Keith Alexander and you got to know new CEO of there. Phil Welsh scaler and duo knows how to scale up a company. He's right. Iron is doing really well. The iron dome, the vision of collaboration and signaling. Congratulations on your success. What's a quick update? >> Well look, I mean, you know, we have now built the capability to share information across multiple companies, multiple industries with the government in real time at machine speed. >>Really bringing people together, not just creating collected security or clip to defense, but also collaborating real time to defend one another. So you're able to divide and conquer Goliath, the enemy the same way they come after you and beat them at their own game. >> So this is the classic case of offense defense. Most corporations are playing defense, whack-a-mole, redundant, not a lot of efficiencies, a lot of burnout. Exactly. Not a lot of collaboration, but everyone's talking about the who the attackers are and collaborating like a team. Right? And you guys talk about this mission. Exactly. This is really the new way to do it. It has, the only way it works, >> it is. And you know, you see kids doing it out there when they're playing Fortnite, right? They're collaborating in real time across networks, uh, to, you know, to play a game, right? You can imagine that same construct when it comes to cyber defense, right? >>There's no reason why one big company, a second big company in a small company can't work together to identify all the threats, see that common threat landscape, and then take action on it. Trusting one another to take down the pieces they have folk to focus on and ultimately winning the battle. There's no other way a single company is gonna be able defend itself against a huge decency that has virtually unlimited resources and virtually unlimited human capital. And you've got to come together, defend across multiple industries, uh, collectively and collaboratively. >> Do you mean, we talked about this last time and I want to revisit this and I think it's super important. I think it's the most important story that's not really being talked about in the industry. And that is that we were talking last time about the government protects businesses. If someone dropped troops on the ground in your neighborhood, the government would protect you digitally. >>That's not happening. So there's really no protection for businesses. Do they build their own militia? Do they build their own army? Who was going to, who's going to be their heat shield? So this is a big conversation and a big, it brings a question. The role of the government. We're going to need a digital air force. We're going to need a digital army, Navy, Navy seals. We need to have that force, and this has to be a policy issue, but in the short term, businesses and individuals are sitting out there being attacked by sophisticated mission-based teams of hackers and nation States, right? Either camouflaging or hiding, but attacking still. This is a huge issue. What's going on? Are people talking about this in D C well, >> John, look not enough. People are talking about it, right? And forget DC. We need to be talking about here, out here in the Silicon Valley with all these companies here at the RSA floor and bring up the things you're bringing up because this is a real problem we're facing as a nation. >>The Russians aren't coming after one company, one state. They're coming after our entire election infrastructure. They're coming after us as a nation. The Chinese maybe come after one company at a time, but their goal is to take our electoral properties, a nation, repurpose it back home. And when the economic game, right, the Iranians, the North Koreans, they're not focused on individual actors, but they are coming after individual actors. We can't defend against those things. One man, one woman, one company on an Island, one, one agency, one state. We've got to come together collectively, right? Work state with other States, right? If we can defend against the Russians, California might be really good at it. Rhode Island, small States can be real hard, defends against the Russians, but if California, Rhode Island come together, here's the threats. I see. Here's what it's. You see share information, that's great. Then we collaborate on the defense and work together. >>You take these threats, I'll take those threats and now we're working as a team, like you said earlier, like those kids do when they're playing fortnight and now we're changing the game. Now we're really fighting the real fight. >> You know, when I hear general Keith Alexander talking about his vision with iron net and what you guys are doing, I'm inspired because it's simply put, we have a mission to protect our nation, our people, and a good businesses, and he puts it into kind of military, military terms, but in reality, it's a simple concept. Yeah, we're being attacked, defend and attack back. Just basic stuff. But to make it work as the sharing. So I got to ask you, I'm first of all, I love the, I love what he has, his vision. I love what you guys are doing. How real are we? What's the progression? >>Where are we on the progress bar of that vision? Well, you know, a lot's changed to the last year and a half alone, right? The threats gotten a lot, a lot more real to everybody, right? Used to be the industry would say to us, yeah, we want to share with the government, but we want something back for, right. We want them to show us some signal to today. Industry is like, look, the Chinese are crushing us out there, right? We can beat them at a, at some level, but we really need the governor to go do its job too. So we'll give you the information we have on, on an anonymized basis. You do your thing. We're going to keep defending ourselves and if you can give us something back, that's great. So we've now stood up in real time of DHS. We're sharing with them huge amounts of data about what we're seeing across six of the top 10 energy companies, some of the biggest banks, some of the biggest healthcare companies in the country. >>Right? In real time with DHS and more to come on that more to come with other government agencies and more to come with some our partners across the globe, right? Partners like those in Japan, Singapore, Eastern Europe, right? Our allies in the middle East, they're all the four lenses threat. We can bring their better capability. They can help us see what's coming at us in the future because as those enemies out there testing the weapons in those local areas. I want to get your thoughts on the capital markets because obviously financing is critical and you're seeing successful venture capital formulas like forge point really specialized funds on cyber but not classic industry formation sectors. Like it's not just security industry are taking a much more broader view because there's a policy implication is that organizational behavior, this technology up and down the stack. So it's a much broad investment thesis. >>What's your view of that? Because as you do, you see that as a formula and if so, what is this new aperture or this new lens of investing to be successful in funding? Companies will look, it's really important what companies like forge point are doing. Venture capital funds, right? Don Dixon, Alberta Pez will land. They're really innovating here. They've created a largest cybersecurity focused fund. They just closed the recently in the world, right? And so they really focus on this industry. Partners like, Kleiner Perkins, Ted Schlein, Andrea are doing really great work in this area. Also really important capital formation, right? And let's not forget other funds. Ron Gula, right? The founder of tenable started his own fund out there in DC, in the DMV area. There's a lot of innovation happening this country and the funding on it's critical. Now look, the reality is the easy money's not going to be here forever, right? >>It's the question is what comes when that inevitable step back. We don't. Nobody likes to talk about it. I said the guy who who bets on the other side of the craps game in Vegas, right? You don't wanna be that guy, but let's be real. I mean that day will eventually come. And the question is how do you bring some of these things together, right? Bring these various pieces together to really create long term strategies, right? And that's I think what's really innovative about what Don and Alberto are doing is they're building portfolio companies across a range of areas to create sort of an end to end capability, right? Andrea is doing things like that. Ted's doing stuff like that. It's a, that's really innovation. The VC market, right? And we're seeing increased collaboration VC to PE. It's looking a lot more similar, right? And now we're seeing innovative vehicles like stacks that are taking some of these public sort of the reverse manner, right? >>There's a lot of interests. I've had to be there with Hank Thomas, the guys chief cyber wrenches. So a lot of really cool stuff going on in the financing world. Opportunities for young, smart entrepreneurs to really move out in this field and to do it now. And money's still silver. All that hasn't come as innovation on the capital market side, which is awesome. Let's talk about the ecosystem in every single market sector that I've been over, my 30 year career has been about a successful entrepreneurship check, capital two formation of partnerships. Okay. You're on the iron net, front lines here. As part of that ecosystem, how do you see the ecosystem formula developing? Is it the same kind of model? Is it a little bit different? What's your vision of the ecosystem? Look, I mean partnerships channel, it's critical to every cyber security company. You can't scale on your own. >>You've got to do it through others, right? I was at a CrowdStrike event the other day. 91% of the revenue comes from the channel. That's an amazing number. You think about that, right? It's you look at who we're trying to talk about partnering with. We're talking about some of the big cloud players. Amazon, Microsoft, right? Google, right on the, on the vendor side. Pardon me? Splunk crashes, so these big players, right? We want to build with them, right? We want to work with them because there's a story to tell here, right? When we were together, the AECOS through self is defendant stronger. There's no, there's no anonymity here, right? It's all we bring a specialty, you bring specialty, you work together, you run out and go get the go get the business and make companies safer. At the end of the day, it's all about protecting the ecosystem. What about the big cloud player? >>Cause he goes two big mega trends. Obviously cloud computing and scale, right? Multi-cloud on the horizon, hybrids, kind of the bridge between single public cloud and multi-cloud and then AI you've got the biggies are generally will be multiple generations of innovation and value creation. What's your vision on the impact of the big waves that are coming? Well, look, I mean cloud computing is a rate change the world right? Today you can deploy capability and have a supercomputer in your fingertips in in minutes, right? You can also secure that in minutes because you can update it in real time. As the machine is functioning, you have a problem, take it down, throw up a new virtual machine. These are amazing innovations that are creating more and more capability out there in industry. It's game changing. We're happy, we're glad to be part of that and we ought to be helping defend that new amazing ecosystem. >>Partnering with companies like Microsoft. They didn't AWS did, you know, you know, I'm really impressed with your technical acumen. You've got a good grasp of the industry, but also, uh, you have really strong on the societal impact policy formulation side of government and business. So I want to get your thoughts for the young kids out there that are going to school, trying to make sense of the chaos that's going on in the world, whether it's DC political theater or the tech theater, big tech and in general, all of the things with coronavirus, all this stuff going on. It's a, it's a pretty crazy time, but a lot of work has to start getting done that are new problems. Yeah. What is your advice as someone who's been through the multiple waves to the young kids who have to figure out what half fatigue, what problems are out there, what things can people get their arms around to work on, to specialize in? >>What's your, what's your thoughts and expertise on that? Well, John, thanks for the question. What I really like about that question is is we're talking about what the future looks like and here's what I think the future looks like. It's all about taking risks. Tell a lot of these young kids out there today, they're worried about how the world looks right? Will America still be strong? Can we, can we get through this hard time we're going through in DC with the world challenges and what I can say is this country has never been stronger. We may have our own troubles internally, but we are risk takers and we always win. No matter how hard it gets them out of how bad it gets, right? Risk taking a study that's building the American blood. It's our founders came here taking a risk, leaving Eagle to come here and we've succeeded the last 200 years. >>There is no question in my mind that trend will continue. So the young people out there, I don't know what the future has to hold. I don't know if the new tape I was going to be, but you're going to invent it. And if you don't take the risks, we're not succeed as a nation. And that's what I think is key. You know, most people worry that if they take too many risks, they might not succeed. Right? But the reality is most people you see around at this convention, they all took risks to be here. And even when they had trouble, they got up, they dust themselves off and they won. And I believe that everybody in this country, that's what's amazing about the station is we have this opportunity to, to try, if we fail to get up again and succeed. So fail fast, fail often, and crush it. >>You know, some of the best innovations have come from times where you had the cold war, you had, um, you had times where, you know, the hippie revolution spawn the computer. So you, so you have the culture of America, which is not about regulation and stunting growth. You had risk-taking, you had entrepreneurship, but yet enough freedom for business to operate, to solve new challenges, accurate. And to me the biggest imperative in my mind is this next generation has to solve a lot of those new questions. What side of the street is the self driving cars go on? I see bike lanes in San Francisco, more congestion, more more cry. All this stuff's going on. AI could be a great enabler for that. Cyber security, a direct threat to our country and global geopolitical landscape. These are big problems. State and local governments, they're not really tech savvy. They don't really have a lot ID. >>So what do they do? How do they serve their, their constituents? You know, look John, these are really important and hard questions, but we know what has made technology so successful in America? What's made it large, successful is the governor state out of the way, right? Industry and innovators have had a chance to work together and do stuff and change the world, right? You look at California, you know, one of the reasons California is so successful and Silicon Valley is so dynamic. You can move between jobs and we don't enforce non-compete agreements, right? Because you can switch jobs and you can go to that next higher value target, right? That shows the value of, you know, innovation, creating innovation. Now there's a real tendency to say, when we're faced with challenges, well, the government has to step in and solve that problem, right? The Silicon Valley and what California's done, what technology's done is a story about the government stayed out and let innovators innovate, and that's a real opportunity for this nation. >>We've got to keep on down that path, even when it seemed like the easier answer is, come on in DC, come on in Sacramento, fix this problem for us. We have demonstrated as a country that Americans and individual are good at solve these problems. We should allow them to do that and innovate. Yeah. One of my passions is to kind of use technology and media to end communities to get to the truth faster. A lot of, um, access to smart minds out there, but young minds, young minds, uh, old minds, young minds though. It's all there. You gotta get the data out and that's going to be a big thing. That's the, one of the things that's changing is the dark arts of smear campaigns. The story of Bloomberg today, Oracle reveals funding for dark money, group biting, big tech internet accountability projects. Um, and so the classic astroturfing get the Jedi contract, Google WASU with Java. >>So articles in the middle of all this, but using them as an illustrative point. The lawyers seem to be running the kingdom right now. I know you're an attorney, so I'm recovering, recovering. I don't want to be offensive, but entrepreneurship cannot be stifled by regulation. Sarbanes Oxley slowed down a lot of the IPO shifts to the latest stage capital. So regulation, nest and every good thing. But also there's some of these little tactics out in the shadows are going to be revealed. What's the new way to get this straightened out in your mind? We'll look, in my view, the best solution for problematic speech or pragmatic people is more speech, right? Let's shine a light on it, right? If there are people doing shady stuff, let's talk about it's an outfit. Let's have it out in the open. Let's fight it out. At the end of the day, what America's really about is smart ideas. >>Winning. It's a, let's get the ideas out there. You know, we spent a lot of time, right now we're under attack by the Russians when it comes to our elections, right? We spent a lot of time harping at one another, one party versus another party. The president versus that person. This person who tells committee for zap person who tells committee. It's crazy when the real threat is from the outside. We need to get past all that noise, right? And really get to the next thing which is we're fighting a foreign entity on this front. We need to face that enemy down and stop killing each other with this nonsense and turn the lights on. I'm a big believer of if something can be exposed, you can talk about it. Why is it happening exactly right. This consequences with that reputation, et cetera. You got it. >>Thanks for coming on the queue. Really appreciate your insight. Um, I want to just ask you one final question cause you look at, look at the industry right now. What is the most important story that people are talking about and what is the most important story that people should be talking about? Yeah. Well look, I think the one story that's out there a lot, right, is what's going on in our politics, what's going on in our elections. Um, you know, Chris Krebs at DHS has been out here this week talking a lot about the threat that our elections face and the importance about States working with one another and States working with the federal government to defend the nation when it comes to these elections in November. Right? We need to get ahead of that. Right? The reality is it's been four years since 2016 we need to do more. That's a key issue going forward. What are the Iranians North Koreans think about next? They haven't hit us recently. We know what's coming. We got to get ahead of that. I'm going to come again at a nation, depending on staff threat to your meal. Great to have you on the QSO is great insight. Thanks for coming on sharing your perspective. I'm John furrier here at RSA in San Francisco for the cube coverage. Thanks for watching.