>> Announcer: It's theCUBE covering the Virtual Vertica Big Data Conference 2020, brought to you by Vertica. >> Hi, everybody. Welcome to this digital coverage of the Vertica Big Data Conference. You're watching theCUBE and my name is Dave Volante. It's my pleasure to invite in Ben White, who's the Senior Database Engineer at Domo. Ben, great to see you, man. Thanks for coming on. >> Great to be here and here. >> You know, as I said, you know, earlier when we were off-camera, I really was hoping I could meet you face-to-face in Boston this year, but hey, I'll take it, and, you know, our community really wants to hear from experts like yourself. But let's start with Domo as the company. Share with us what Domo does and what your role is there. >> Well, if I can go straight to the official what Domo does is we provide, we process data at BI scale, we-we-we provide BI leverage at cloud scale in record time. And so what that means is, you know, we are a business-operating system where we provide a number of analytical abilities to companies of all sizes. But we do that at cloud scale and so I think that differentiates us quite a bit. >> So a lot of your work, if I understand it, and just in terms of understanding what Domo does, there's a lot of pressure in terms of being real-time. It's not, like, you sometimes don't know what's coming at you, so it's ad-hoc. I wonder if you could sort of talk about that, confirm that, maybe add a little color to it. >> Yeah, absolutely, absolutely. That's probably the biggest challenge it is to being, to operating Domo is that it is an ad hoc environment. And certainly what that means, is that you've got analysts and executives that are able to submit their own queries with out very... With very few limitations. So from an engineering standpoint, that challenge in that of course is that you don't have this predictable dashboard to plan for, when it comes to performance planning. So it definitely presents some challenges for us that we've done some pretty unique things, I think, to address those. >> So it sounds like your background fits well with that. I understand your people have called you a database whisperer and an envelope pusher. What does that mean to a DBA in this day and age? >> The whisperer part is probably a lost art, in the sense that it's not really sustainable, right? The idea that, you know, whatever it is I'm able to do with the database, it has to be repeatable. And so that's really where analytics comes in, right? That's where pushing the envelope comes in. And in a lot of ways that's where Vertica comes in with this open architecture. And so as a person who has a reputation for saying, "I understand this is what our limitations should be, but I think we can do more." Having a platform like Vertica, with such an open architecture, kind of lets you push those limits quite a bit. >> I mean I've always felt like, you know, Vertica, when I first saw the stone breaker architecture and talked to some of the early founders, I always felt like it was the Ferrari of databases, certainly at the time. And it sounds like you guys use it in that regard. But talk a little bit more about how you use Vertica, why, you know, why MPP, why Vertica? You know, why-why can't you do this with RDBMS? Educate us, a little bit, on, sort of, the basics. >> For us it was, part of what I mentioned when we started, when we talked about the very nature of the Domo platform, where there's an incredible amount of resiliency required. And so Vertica, the MPP platform, of course, allows us to build individual database clusters that can perform best for the workload that might be assigned to them. So the open, the expandable, the... The-the ability to grow Vertica, right, as your base grows, those are all important factors, when you're choosing early on, right? Without a real idea of how growth would be or what it will look like. If you were kind of, throwing up something to the dark, you look at the Vertica platform and you can see, well, as I grow, I can, kind of, build with this, right? I can do some unique things with the platform in terms of this open architecture that will allow me to not have to make all my decisions today, right? (mutters) >> So, you're using Vertica, I know, at least in part, you're working with AWS as well, can you describe sort of your environment? Do you give anything on-prem, is everything in cloud? What's your set up look like? >> Sure, we have a hybrid cloud environment where we have a significant presence in public files in our own private cloud. And so, yeah, having said that, we certainly have a really an extensive presence, I would say, in AWS. So, they're definitely the partner of our when it comes to providing the databases and the server power that we need to operate on. >> From a standpoint of engineering and architecting a database, what were some of the challenges that you faced when you had to create that hybrid architecture? What did you face and how did you overcome that? >> Well, you know, some of the... There were some things we faced in terms of, one, it made it easy that Vertica and AWS have their own... They play well together, we'll say that. And so, Vertica was designed to work on AWS. So that part of it took care of it's self. Now our own private cloud and being able to connect that to our public cloud has been a part of our own engineering abilities. And again, I don't want to make little, make light of it, it certainly not impossible. And so we... Some of the challenges that pertain to the database really were in the early days, that you mentioned, when we talked a little bit earlier about Vertica's most recent eon mode. And I'm sure you'll get to that. But when I think of early challenges, some of the early challenges were the architecture of enterprise mode. When I talk about all of these, this idea that we can have unique databases or database clusters of different sizes, or this elasticity, because really, if you know the enterprise architecture, that's not necessarily the enterprise architecture. So we had to do some unique things, I think, to overcome that, right, early. To get around the rigidness of enterprise. >> Yeah, I mean, I hear you. Right? Enterprise is complex and you like when things are hardened and fossilized but, in your ad hoc environment, that's not what you needed. So talk more about eon mode. What is eon mode for you and how do you apply it? What are some of the challenges and opportunities there, that you've found? >> So, the opportunities were certainly in this elastic architecture and the ability to separate in the storage, immediately meant that for some of the unique data paths that we wanted to take, right? We could do that fairly quickly. Certainly we could expand databases, right, quickly. More importantly, now you can reduce. Because previously, in the past, right, when I mentioned the enterprise architecture, the idea of growing a database in itself has it's pain. As far as the time it takes to (mumbles) the data, and that. Then think about taking that database back down and (telephone interference). All of a sudden, with eon, right, we had this elasticity, where you could, kind of, start to think about auto scaling, where you can go up and down and maybe you could save some money or maybe you could improve performance or maybe you could meet demand, At a time where customers need it most, in a real way, right? So it's definitely a game changer in that regard. >> I always love to talk to the customers because I get to, you know, I hear from the vendor, what they say, and then I like to, sort of, validate it. So, you know, Vertica talks a lot about separating compute and storage, and they're not the only one, from an architectural standpoint who do that. But Vertica stresses it. They're the only one that does that with a hybrid architecture. They can do it on-prem, they can do it in the cloud. From your experience, well first of all, is that true? You may or may not know, but is that advantageous to you, and if so, why? >> Well, first of all, it's certainly true. Earlier in some of the original beta testing for the on-prem eon modes that we... I was able to participate in it and be aware of it. So it certainly a realty, they, it's actually supported on Pure storage with FlashBlade and it's quite impressive. You know, for who, who will that be for, tough one. It's probably Vertica's question that they're probably still answering, but I think, obviously, some enterprise users that probably have some hybrid cloud, right? They have some architecture, they have some hardware, that they themselves, want to make use of. We certainly would probably fit into one of their, you know, their market segments. That they would say that we might be the ones to look at on-prem eon mode. Again, the beauty of it is, the elasticity, right? The idea that you could have this... So a lot of times... So I want to go back real quick to separating compute. >> Sure. Great. >> You know, we start by separating it. And I like to think of it, maybe more of, like, the up link. Because in a true way, it's not necessarily separated because ultimately, you're bringing the compute and the storage back together. But to be able to decouple it quickly, replace nodes, bring in nodes, that certainly fits, I think, what we were trying to do in building this kind of ecosystem that could respond to unknown of a customer query or of a customer demand. >> I see, thank you for that clarification because you're right, it's really not separating, it's decoupling. And that's important because you can scale them independently, but you still need compute and you still need storage to run your work load. But from a cost standpoint, you don't have to buy it in chunks. You can buy in granular segments for whatever your workload requires. Is that, is that the correct understanding? >> Yeah, and to, the ability to able to reuse compute. So in the scenario of AWS or even in the scenario of your on-prem solution, you've got this data that's safe and secure in (mumbles) computer storage, but the compute that you have, you can reuse that, right? You could have a scenario that you have some query that needs more analytic, more-more fire power, more memory, more what have you that you have. And so you can kind of move between, and that's important, right? That's maybe more important than can I grow them separately. Can I, can I borrow it. Can I borrow that compute you're using for my (cuts out) and give it back? And you can do that, when you're so easily able to decouple the compute and put it where you want, right? And likewise, if you have a down period where customers aren't using it, you'd like to be able to not use that, if you no longer require it, you're not going to get it back. 'Cause it-it opened the door to a lot of those things that allowed performance and process department to meet up. >> I wonder if I can ask you a question, you mentioned Pure a couple of times, are you using Pure FlashBlade on-prem, is that correct? >> That is the solution that is supported, that is supported by Vertica for the on-prem. (cuts out) So at this point, we have been discussing with them about some our own POCs for that. Before, again, we're back to the idea of how do we see ourselves using it? And so we certainly discuss the feasibility of bringing it in and giving it the (mumbles). But that's not something we're... Heavily on right now. >> And what is Domo for Domo? Tell us about that. >> Well it really started as this idea, even in the company, where we say, we should be using Domo in our everyday business. From the sales folk to the marketing folk, right. Everybody is going to use Domo, it's a business platform. For us in engineering team, it was kind of like, well if we use Domo, say for instance, to be better at the database engineers, now we've pointed Domo at itself, right? Vertica's running Domo in the background to some degree and then we turn around and say, "Hey Domo, how can we better at running you?" So it became this kind of cool thing we'd play with. We're now able to put some, some methods together where we can actually do that, right. Where we can monitor using our platform, that's really good at processing large amounts of data and spitting out useful analytics, right. We take those analytics down, make recommendation changes at the-- For now, you've got Domo for Domo happening and it allows us to sit at home and work. Now, even when we have to, even before we had to. >> Well, you know, look. Look at us here. Right? We couldn't meet in Boston physically, we're now meeting remote. You're on a hot spot because you've got some weather in your satellite internet in Atlanta and we're having a great conversation. So-so, we're here with Ben White, who's a senior database engineer at Domo. I want to ask you about some of the envelope pushing that you've done around autonomous. You hear that word thrown around a lot. Means a lot of things to a lot of different people. How do you look at autonomous? And how does it fit with eon and some of the other things you're doing? >> You know, I... Autonomous and the idea idea of autonomy is something that I don't even know if that I have already, ready to define. And so, even in my discussion, I often mention it as a road to it. Because exactly where it is, it's hard to pin down, because there's always this idea of how much trust do you give, right, to the system or how much, how much is truly autonomous? How much already is being intervened by us, the engineers. So I do hedge on using that. But on this road towards autonomy, when we look at, what we're, how we're using Domo. And even what that really means for Vertica, because in a lot of my examples and a lot of the things that we've engineered at Domo, were designed to maybe overcome something that I thought was a limitation thing. And so many times as we've done that, Vertica has kind of met us. Like right after we've kind of engineered our architecture stuff, that we thought that could help on our side, Vertica has a release that kind of addresses it. So, the autonomy idea and the idea that we could analyze metadata, make recommendations, and then execute those recommendations without innervation, is that road to autonomy. Once the database is properly able to do that, you could see in our ad hoc environment how that would be pretty useful, where with literally millions of queries every hour, trying to figure out what's the best, you know, profile. >> You know for- >> (overlapping) probably do a better job in that, than we could. >> For years I felt like IT folks sometimes were really, did not want that automation, they wanted the knobs to turn. But I wonder if you can comment. I feel as though the level of complexity now, with cloud, with on-prem, with, you know, hybrid, multicloud, the scale, the speed, the real time, it just gets, the pace is just too much for humans. And so, it's almost like the industry is going to have to capitulate to the machine. And then, really trust the machine. But I'm still sensing, from you, a little bit of hesitation there, but light at the end of the tunnel. I wonder if you can comment? >> Sure. I think the light at the end of the tunnel is even in the recent months and recent... We've really begin to incorporate more machine learning and artificial intelligence into the model, right. And back to what we're saying. So I do feel that we're getting closer to finding conditions that we don't know about. Because right now our system is kind of a rule, rules based system, where we've said, "Well these are the things we should be looking for, these are the things that we think are a problem." To mature to the point where the database is recognizing anomalies and taking on pattern (mutters). These are problems you didn't know happen. And that's kind of the next step, right. Identifying the things you didn't know. And that's the path we're on now. And it's probably more exciting even than, kind of, nailing down all the things you think you know. We figure out what we don't know yet. >> So I want to close with, I know you're a prominent member of the, a respected member of the Vertica Customer Advisory Board, and you know, without divulging anything confidential, what are the kinds of things that you want Vertica to do going forward? >> Oh, I think, some of the in dated base for autonomy. The ability to take some of the recommendations that we know can derive from the metadata that already exists in the platform and start to execute some of the recommendations. And another thing we've talked about, and I've been pretty open about talking to it, talking about it, is the, a new version of the database designer, I think, is something that I'm sure they're working on. Lightweight, something that can give us that database design without the overhead. Those are two things, I think, as they nail or basically the database designer, as they respect that, they'll really have all the components in play to do in based autonomy. And I think that's, to some degree, where they're heading. >> Nice. Well Ben, listen, I really appreciate you coming on. You're a thought leader, you're very open, open minded, Vertica is, you know, a really open community. I mean, they've always been quite transparent in terms of where they're going. It's just awesome to have guys like you on theCUBE to-to share with our community. So thank you so much and hopefully we can meet face-to-face shortly. >> Absolutely. Well you stay safe in Boston, one of my favorite towns and so no doubt, when the doors get back open, I'll be coming down. Or coming up as it were. >> Take care. All right, and thank you for watching everybody. Dave Volante with theCUBE, we're here covering the Virtual Vertica Big Data Conference. (electronic music)

everybody welcome to this digital coverage of the verdict of big data conference you're watching the cube and my name is Dave Galante it's my pleasure to invite in Ben white who's the senior database engineer at Domo been great to see you man thanks for coming on great to be here and here you know as I said you know earlier when we were off camera I really was hoping I could meet you face to face and in Boston this year but hey I'll take it and you know our community really wants to hear from experts like yourself but let's start with with domo is the company share with us what Domo does and what your role is there well if Parker can go straight to the official what Domo does is we provide we process data at bi to scale with we provide VI leverage a cloud scale in record time and so what that means is that you know we are a business operating system where we provide a number of analytical abilities to companies of all sizes but we do that at cloud scale and so I think that difference is quite a bit so a lot of your work if I understand it and just in terms of understanding with Domo does--is there's a lot of pressure in terms of being real-time it's not like you sometimes don't know what's coming at you so it's AD Hoch I wonder if you could sort of talk about that confirm that and maybe add a little color to it yeah absolutely absolutely that's probably the biggest challenge it is to being the operating Domo is that it is an ad hoc environment and certainly what that means is that you've got analysts and executives that are able to submit their own queries without very with very few limitations so from an engineering standpoint the challenge in that of course is that you don't have this predictable dashboard to plan for when it comes to performance planning and so it definitely presents some challenges for us that we've done some pretty unique things I think to address those right sounds like your background fits well with that I understand here if people have called you a database whisperer and an envelope pusher what does that mean to do a DBA in this in this day and age well the whisperer part is probably a lost art in the sense that it's not really sustainable right the idea that you know whatever it is I'm able to do with the database it has to be repeatable and so that's really what analytics comes in right and that's where pushing the envelope comes in in a little right away that's what vertical comes in with this open architecture and so as a person who has a reputation for saying I understand this is what our limitations should be but I think we can do more having a platform like vertical is such an open architecture kinda lets you push those limits by the bit I mean I've always felt like you know vertical when I first saw the Stonebreaker architecture and doctors some of the early founders I always felt like it was the Ferrari of databases certainly at the time and it sounds like you guys use it in that in that regard but talk a little bit more about how you use Vertica why in a ym ppy Vertica you know why why can't you do this with our DBMS educate us a little bit on some of the basics but for us it was part of what I mentioned when we start and we talked about the very nature of the demo platform where there's a an incredible amount of resiliency required and so Vertica the NPP platform of course allows us to build individual database clusters that can perform best for the workload that may be assigned to them so the the open the expandable the the the ability to grow vertically as your base grow those are all important factors when you're losing early on right without a real idea of how growth would be or what it would look like if you were kind of doing that something to the dark you looked at the vertical platforming you can see well as I grow I can kind of feel with this right I can do some some unique things with the platform in terms of this poking architecture that will allow me to not have to make all my decisions today right about Harlem so you're using Vertica I know at least in part you you working with AWS as well can you describe sort of your environment that you give anything on Prem is everything in the cloud what's your setup sure we have a hybrid cloud environment where we have a significant presence in public files in our own private cloud and so yeah having said that we certainly have a really an extensive presence I will say an AWS and so they're definitely the partner of our when it comes to providing the databases the server power that we need to operator but from the standpoint of engineering and architecting a database what was some of the challenges that you faced when you had to create that hybrid architecture what did you face and how did you overcome that well you know some of the there are some things we need faced in terms of wine and made it easy that Vertica and AWS have their own they play well together we'll say that and so vertical is designed to reprise I'm gonna AWS and so that part of it the care of itself not our own private cloud and being able to connect that because our public clouds has been a part of our own engineering ability and again I don't want to make a little light of it it's certainly not impossible and so we've some of the challenges though this pertains to the database really were in their early days that you mentioned when we talked a little bit earlier about marathas most recent Eon mode and I'm sure you'll get to that but when I think of our early challenges some of the early challenges were the architecture of enterprise mode when I talk about all of these this idea that we could have unique databases or database clusters of different sizes so this elasticity that's really if you know that the enterprise architecture that's not necessarily dandified architecture so we added this Munich things I think to overcome that right early to get around the rigidness though enterprise yeah I mean I hear you right Enterprise is complex and and you like when things are hardened and fossilized but in your ad hoc environment that's not what you needed so talking more about Aeon mode what what is e on mode for you and how do you apply it what are some of the challenges and opportunities there that you found um so the opportunities were certainly in its elastic architecture the ability to separate the storage immediately meant that for some of the unique data paths that we wanted to take right we could do that fairly quickly certainly we could expand databases right quickly but more importantly now you could reduce because previously in the past right when I mention the Enterprise Architect with the idea of growing a database in itself has its pain right as far as the time it takes to speed the data in that but to read to then think about taking that database back down no Innova though all of us under the eon right you had this elasticity where you could kind of start to think about auto scaling where you go up and down and maybe used to save some money or maybe you could improve performance or maybe in needham and at a time when the customers needed most in a real way right so it was definitely a game in that regard I always have to talk to the customers because I get to you know I hear from the vendor what they say and I think they sort of validate it so you know Vertica talks a lot about separating compute and storage they're not the only one from an architectural standpoint to do that but Vertica stresses that they're the only one that does that with a hybrid architecture they can do it off ram they can do it in the cloud from your experience well first of all is it true you may or may not know it is that advantageous to you and if so why well first of all it's certainly true earlier in some of the original beta ethnic for the arm prim GI mode stuff we I was able to participate in it and be aware of it so it's certainly a reality day I'm it's actually supported on pure spirit with flash played and it's time quite impressive you know for who who that who that will be for tough one a Spartacus question that they're probably still answering but I think obviously some enterprise users that probably have some hybrid cloud right they have some architecture they have some hardware that their sales want to make you so we certainly would probably fit into one of their you know their market segments that they would say we might be the wants to look at on pram er mo begin the the beauty of it is the elasticity right that the idea that you could have this and so a lot of times so I want to go back real quick to separating them and you know we start by separating it and I like to think of it maybe more as like decoupling because a new in a true way it's not necessary separated there's ultimately you bring the compute and the doors back together but to be able to typically couple it quickly replace knows bring in those that's certainly fits I think what we were trying to do in building this Emma I'll me let the ecosystem that could respond to a unknown or of a customer demand I see thank you for that clarification because you're right it's really not separating its decoupling in it that's important because you can scale them independently but you still need compute and you still need storage to run you your workloads but from a cost standpoint you're not to buy it in in chunks you can you can't buy granular segments for whatever your workload requires is that is that the correct understanding yeah and to be able to the ability to be able to reuse compute throw it in a scenario of AWS or even in the scenario your on-prem solution you've got this data that's safest here and ask for your in your storage but then the compute that you have you can reuse that right you could have a scenario that you have some query that needs more analytic more firepower more memory more what have you that you haven't so you can kind of move to the next important right that's maybe more important then and I grow them separately can I can I borrow it can I borrow that computer use for my perfect give it back type of thing and you can do that when you're so easily a couple different ooh all right and likewise if you have a down period where customers aren't using it you'd like to be able to not use that if you no longer require if you'd like to give it back go in it open the door to a lot of those things that allow performance and cross the spark to meet up we're going to ask you a question winsome pure a couple times are you using pure flash blade on-prem is that correct that is the solution that is supported that is supported by Vertica for the on print so at this point we were we have been discuss with them about some our own PLC's for that time before again we back to the idea of how do we see ourselves using it and so we've certainly discussed the feasibility of bringing it in and give it a job but that's not something we're Oh happily all right now then what is Domo for Domo tell us about that we really started this this idea even in the company where we say you know we should be using Domo in our everyday business the sales folks the marketing folks right everybody we're gonna use Domo it's a business platform for us in the engineering team it was kind of like well if we use Domo say for instance to be better at the database engineers now we've pointed Domo edits tell fried verdict is running Domo in the background for some degree and then we turn around and say hey Domo how can we better at running you and so it became this kind of cool thing we played with where we're now able to put some dumb methods together where we can actually do their eye we can monitor using our platform it's really good at processing large amounts of data and spitting out useful analytics right we take those analytics out make recommendation changes that the day so now you've got still more for Domo happening it allows us to sit at home and and work now even when we have to even before we had to well you know look look at us here right it couldn't mean in Boston physically we're now meeting remote you're you're on a hot spot because you got some weather and your satellite internet and in Atlanta and we're having a great conversation so so we're here with with Ben white who's the senior database engineer at Domo I want to ask you about some of the envelope-pushing that you've done around autonomous you hear that that word thrown around a lot means a lot of things to a lot of different people how do you look at autonomous and how does it fit with Eon and some of the other things that you're doing you know I'm a tall amidst the idea of economy is something that I don't even know that I'm I have already ready to define and so even in my discussion I often mention it as a road to it exactly where it is it's hard to pin down because there's always this idea how much trust do you give right to the system or how much how much is truly autonomous how much authority is being intervened by us the engineers so I do hate on using it but on this road towards autonomy when we look at what would how we're using Domo and even what that really means to vertical because in a lot of my examples and a lot of the things that we've engineered a demo work designs maybe over something I thought was a limitation day and so many times Oh as we've done that verdict is kind of met us like right after we've kind of engineered our architecture stuff than we thought it felt on our side Vertica has some released it kinda addresses it so the autonomy idea and the idea that we could analyzed metadata make recommendations and then execute those recommendations without intervention is that road to autonomy and once the databases start able to do that you can see in our ad-hoc environment how that would be pretty pretty useful where with literally millions of queries every hour trying to figure out what's the best you know probably for years I felt like I I T folks sometimes we really did not want that automation they wanted the knobs to turn but but I wonder if you comment I feel as though the level of complexity now with cloud with on-prem with you know hybrid multi clouds the scale the speed the real-time it just gets the pace is just too much for for humans and so it's almost like you know the industries is gonna have to capitulate to the Machine and then really trust the machine but I'm sitting I'm still sensing from you a little bit of hesitation there but light at the end of the tunnel I wonder if you could comment sure I think that in the light of the tunnel is even in recent months in recent we've really began incorporating more machine learning in artificial intelligence to the model right and back to where we're saying it so I do feel they were getting close for too finding conditions that we don't know about because right now our system is kind of a rule rules based system where we've said well these are the things that we should be looking for these are the things that we think are a problem to mature to the point where the database is recognized and anomalies and taken on at imagining saying these are problems you didn't know happen and that's kind of the next step right identifying the things you didn't know and that's where that's the path we're on now and that's probably more exciting even then kind of nailing down all the things you think you know and to figure out what we don't know yet so I want to close with I know you're a prominent member of the respected member of the Vertica a customer advisory board you know without divulging anything confidential to me what are the kinds of things that you want Vertica to do going forward I think some of the end a in database autonomy the ability to take some of the recommendations that we know we can derive from the metadata that already exists in the platform and start to execute some of the recommendation another thing we talk about and I'm gonna pretty open about talking to it is talking about it is the new version of the database designer I think it's something that I'm sure they're working on lightweight something that can give us that's database design without the overhead those are two things I think as they nail or particularly the database designer as they respect that they'll really have all the components in place to do in based economy and I think that's just some victory where they're headed yeah nice well Ben listen I really appreciate you coming on your a thought leader be very open open-minded verdict is you know really open community I mean they've always been quite transparent in terms of where they're going it's just awesome to have guys like you on the cube to share with our community so thank you so much and hopefully we can meet face to face currently absolutely will you stay safe in Boston I'm one of my favorite towns and so no doubt when this when the doors get back open I'll be from coming down or coming I'm gonna do work take care all right and thank you for watching everybody Villante with a cube we're here covering the virtual Vertica of big data conference you [Music]

>>Okay, We're now going into the technical deep dive. We're gonna geek out here a little bit. Ben Wa Dodgeville is here. He's co founder of Snowflake and president of products. And also joining us is Christian Kleinerman. Who's the senior vice president of products. Gentlemen, welcome. Good to see you. >>Yeah, you that >>get this year, they Thanks for having us. >>Very welcome. So it been well, we've heard a lot this morning about the data cloud, and it's becoming my view anyway, the linchpin of your strategy. I'm interested in what technical decisions you made early on. That that led you to this point and even enabled the data cloud. >>Yes. So? So I would say that that a crowd was built in tow in three phases. Really? The initial phase, as you call it, was it was really about 20 minutes. One regions Teoh, Data Cloud and and that region. What was important is to make that region infinity, infinity scalable, right. And that's our architectural, which we call the beauty cross to share the architectural er so that you can plug in as many were clues in that region as a Z without any limits. The limit is really the underlying prop Provide the, you know, resource is which you know, Cal provide the region as a really no limits. So So that z you know, region architecture, I think, was really the building block of the snowflake. That a cloud. But it really didn't stop there. The second aspect Waas Well, it was really data sharing. How you know munity internets within the region, how to share data between 10 and off that region between different customers on that was also enabled by architectures Because we discover, you know, compute and storage so compute You know clusters can access any storage within the region. Eso that's based off the data cloud and then really faced three Which is critical is the expansion the global expansion how we made you know, our cloud domestic layers so that we could talk You know the snowflake vision on different clouds on DNA Now we are running in three cloud on top of three cloud providers. We started with the ws and US West. We moved to assure and then uh, Google g c p On how this this crowd region way started with one crowd region as I said in the W S U S West, and then we create we created, you know, many you know, different regions. We have 22 regions today, all over the world and all over the different in the cloud providers. And what's more important is that these regions are not isolated. You know, Snowflake is one single, you know, system for the world where we created this global data mesh which connects every region such that not only there's no flex system as a whole can can be aware of for these regions, But customers can replicate data across regions on and, you know, share. There are, you know, across the planet if need be. So So this is one single, you know, really? I call it the World Wide Web. Off data that, that's, you know, is this vision of the data cloud. And it really started with this building block, which is a cloud region. >>Thank you for that. Ben White Christian. You and I have talked about this. I mean, that notion of a stripping away the complexity and that's kind of what the data cloud does. But if you think about data architectures, historically they really had no domain knowledge. They've really been focused on the technology toe ingest and analyze and prepare And then, you know, push data out to the business and you're really flipping that model, allowing the sort of domain leaders to be first class citizens if you will, uh, because they're the ones that creating data value, and they're worrying less about infrastructure. But I wonder, do you feel like customers air ready for that change? >>I I love the observation. They've that, uh, so much energy goes in in in enterprises, in organizations today, just dealing with infrastructure and dealing with pipes and plumbing and things like that and something that was insightful from from Ben Juan and and our founders from from Day one WAAS. This is a managed service. We want our customers to focus on the data, getting the insights, getting the decisions in time, not just managing pipes and plumbing and patches and upgrades, and and the the other piece that it's it's it's an interesting reality is that there is this belief that the cloud is simplifying this, and all of a sudden there's no problem but actually understanding each of the public cloud providers is a large undertaking, right? Each of them have 100 plus services, uh, sending upgrades and updates on a constant basis. And that just distracts from the time that it takes to go and say, Here's my data. Here's my data model. Here's how it make better decisions. So at the heart of everything we do is we wanna abstract the infrastructure. We don't wanna abstract the nuance of each of the cloud providers. And as you said, have companies focus on This is the domain expertise or the knowledge for my industry. Are all companies ready for it? I think it's a It's a mixed bag. We we talk to customers on a regular basis every way, every week, every day, and some of them are full on. They've sort of burned the bridges and, like I'm going to the cloud, I'm going to embrace a new model. Some others. You can see the complete like, uh, shock and all expressions like What do you mean? I don't have all these knobs. 2 to 3 can turn. Uh, but I think the future is very clear on how do we get companies to be more competitive through data? >>Well, Ben Ben. Well, it's interesting that Christian mentioned to manage service and that used to be in a hosting. Guys run around the lab lab coats and plugging things in. And of course, you're looking at this differently. It's high degrees of automation. But, you know, one of those areas is workload management. And I wonder how you think about workload management and how that changes with the data cloud. >>Yeah, this is a great question. Actually, Workload management used to be a nightmare. You know, traditional systems on it was a nightmare for the B s and they had to spend most a lot of their time, you know, just managing workloads. And why is that is because all these workloads are running on the single, you know, system and a single cluster The compete for resources. So managing workload that always explain it as explain Tetris, right? You had the first to know when to run. This work will make sure that too big workers are not overlapping. You know, maybe it really is pushed at night, you know, And And you have this 90 window which is not, you know, efficient. Of course, for you a TL because you have delays because of that. But but you have no choice, right? You have a speaks and more for resource is and you have to get the best out of this speaks resource is. And and for sure you don't want to eat here with her to impact your dash boarding workload or your reports, you know, impact and with data science and and And this became a true nine man because because everyone wants to be that a driven meaning that all the entire company wants to run new workers on on this system. And these systems are completely overwhelmed. So so, well below management was, and I may have before Snowflake and Snowflake made it really >>easy. The >>reason is it's no flag. We leverage the crowds who dedicates, you know, compute resources to each work. It's in the snowflake terminology. It's called a warehouse virtual warehouse, and each workload can run in its own virtual warehouse, and each virtual warehouse has its own dedicated competition resources. It's on, you know, I opened with and you can really control how much resources which workload gas by sizing this warehouses. You know, I just think the compute resources that they can use When the workload, you know, starts to execute automatically. The warehouse, the compute resources are turned off, but turned on by snowflake is for resuming a warehouse and you can dynamically resized this warehouse. It can be done by the system automatically. You know if if the conference see of the workload increases or it can be done manually by the administrator or, you know, just suggesting, you know, uh, compute power. You know, for each workload and and the best off that model is not only it gives you a very fine grain. Control on resource is that this work can get Not only workloads are not competing and not impacting it in any other workload. But because of that model, you can hand as many workload as you want. And that's really critical because, as I said, you know, everyone in the organization wants to use data to make decisions, So you have more and more work roads running. And then the Patriots game, you know, would have been impossible in in a in a centralized one single computer, cross the system On the flip side. Oh, is that you have to have a zone administrator off the system. You have to to justify that. The workload is worth running for your organization, right? It's so easy in literally in seconds, you can stand up a new warehouse and and start to run your your crazy on that new compute cluster. And of course, you have to justify if the cost of that because there is a cost, right, snowflake charges by seconds off compute So that cost, you know, is it's justified and you have toe. You know, it's so easy now to hire new workflow than you do new things with snowflake that that that you have to to see, you know, and and look at the trade off the cost off course and managing costs. >>So, Christian been while I use the term nightmare, I'm thinking about previous days of workload management. I mean, I talked to a lot of customers that are trying to reduce the elapsed time of going from data insights, and their nightmare is they've got this complicated data lifecycle. Andi, I'm wondering how you guys think about that. That notion of compressing elapsed time toe data value from raw data to insights. >>Yeah, so? So we we obsess or we we think a lot about this time to insight from the moment that an event happens toe the point that it shows up in a dashboard or a report or some decision or action happens based on it. There are three parts that we think on. How do we reduce that life cycle? The first one which ties to our previous conversation is related toe. Where is their muscle memory on processes or ways of doing things that don't actually make us much sense? My favorite example is you say you ask any any organization. Do you run pipelines and ingestion and transformation at two and three in the morning? And the answer is, Oh yeah, we do that. And if you go in and say, Why do you do that? The answer is typically, well, that's when the resource is are available Back to Ben Wallace. Tetris, right? That's that's when it was possible. But then you ask, Would you really want to run it two and three in the morning? If if you could do it sooner, we could do it. Mawr in time, riel time with when the event happened. So first part of it is back to removing the constraints of the infrastructures. How about running transformations and their ingestion when the business best needs it? When it's the lowest time to inside the lowest latency, not one of technology lets you do it. So that's the the the easy one out the door. The second one is instead of just fully optimizing a process, where can you remove steps of the process? This is where all of our data sharing and the snowflake data marketplace come into place. How about if you need to go in and just data from a SAS application vendor or maybe from a commercial data provider and imagine the dream off? You wouldn't have to be running constant iterations and FTP s and cracking C S V files and things like that. What if it's always available in your environment, always up to date, And that, in our mind, is a lot more revolutionary, which is not? Let's take away a process of ingesting and copying data and optimize it. How about not copying in the first place? So that's back to number two on, then back to number three is is what we do day in and day out on making sure our platform delivers the best performance. Make it faster. The combination of those three things has led many of our customers, and and And you'll see it through many of the customer testimonials today that they get insights and decisions and actions way faster, in part by removing steps, in part by doing away with all habits and in part because we deliver exceptional performance. >>Thank you, Christian. Now, Ben Wa is you know, we're big proponents of this idea of the main driven design and data architecture. Er, you know, for example, customers building entire applications and what I like all data products or data services on their data platform. I wonder if you could talk about the types of applications and services that you're seeing >>built >>on top of snowflake. >>Yeah, and And I have to say that this is a critical aspect of snowflake is to create this platform and and really help application to be built on top of this platform. And the more application we have, the better the platform will be. It is like, you know, the the analogies with your iPhone. If your iPhone that no applications, you know it would be useless. It's it's an empty platforms. So So we are really encouraging. You know, applications to be belong to the top of snowflake and from there one actually many applications and many off our customers are building applications on snowflake. We estimated that's about 30% are running already applications on top off our platform. And the reason is is off course because it's it's so easy to get compute resources. There is no limit in scale in our viability, their ability. So all these characteristics are critical for for an application on DWI deliver that you know from day One Now we have improved, you know, our increased the scope off the platform by adding, you know, Java in competition and Snow Park, which which was announced today. That's also you know, it is an enabler. Eso in terms off type of application. It's really, you know, all over and and what I like actually needs to be surprised, right? I don't know what well being on top of snowflake and how it will be the world, but with that are sharing. Also, we are opening the door to a new type of applications which are deliver of the other marketplace. Uh, where, You know, one can get this application died inside the platform, right? The platform is distributing this application, and today there was a presentation on a Christian T notes about, >>you >>know, 20 finds, which, you know, is this machine learning, you know, which is providing toe. You know, any users off snowflake off the application and and machine learning, you know, to find, you know, and apply model on on your data and enrich your data. So data enrichment, I think, will be a huge aspect of snowflake and data enrichment with machine learning would be a big, you know, use case for these applications. Also, how to get there are, you know, inside the platform. You know, a lot of applications led him to do that. Eso machine learning. Uh, that engineering enrichments away. These are application that we run on the platform. >>Great. Hey, we just got a minute or so left in. Earlier today, we ran a video. We saw that you guys announced the startup competition, >>which >>is awesome. Ben, while you're a judge in this competition, what can you tell us about this >>Yeah, >>e you know, for me, we are still a startup. I didn't you know yet, you know, realize that we're not anymore. Startup. I really, you know, you really feel about you know, l things, you know, a new startups, you know, on that. That's very important for Snowflake. We have. We were started yesterday, and we want to have new startups. So So the ends, the idea of this program, the other aspect off that program is also toe help, you know, started to build on top of snowflake and to enrich. You know, this this pain, you know, rich ecosystem that snowflake is or the data cloud off that a cloud is And we want to, you know, add and boost. You know that that excitement for the platform, so So the ants, you know, it's a win win. It's a win, you know, for for new startups. And it's a win, ofcourse for us. Because it will make the platform even better. >>Yeah, And startups, or where innovation happens. So registrations open. I've heard, uh, several, uh, startups have have signed up. You goto snowflake dot com slash startup challenge, and you can learn mawr. That's exciting program. An initiative. So thank you for doing that on behalf of of startups out there and thanks. Ben Wa and Christian. Yeah, I really appreciate you guys coming on Great conversation. >>Thanks for David. >>You're welcome. And when we talk, Thio go to market >>pros. They >>always tell us that one of the key tenets is to stay close to the customer. Well, we want to find out how data helps us. To do that in our next segment. Brings in to chief revenue officers to give us their perspective on how data is helping their customers transform. Business is digitally. Let's watch.

hello everybody and thank you for joining us today at the virtual Vertica BBC 2020 today's breakout session is entitled the road to autonomous database management how Domo is delivering SLA for less my name is su LeClair I'm the director of marketing at Vertica and I'll be your host for this webinar joining me is Ben white senior database engineer at Domo but before we begin I want to encourage you to submit questions or comments during the virtual session you don't have to wait just type your question or comment in the question box below the slides and click Submit there will be a Q&A session at the end of the presentation we'll answer as many questions as we're able to during that time any questions that we aren't able to address or drew our best to answer them offline alternatively you can visit vertical forums to post your questions there after the session our engineering team is planning to join the forum to keep the conversation going also as a reminder you can maximize your screen by clicking the double arrow button in the lower right corner of the slide and yes this virtual session is being recorded and will be available to view on demand this week we'll send you notification as soon as it's ready now let's get started then over to you greetings everyone and welcome to our virtual Vertica Big Data conference 2020 had we been in Boston the song you would have heard playing in the intro would have been Boogie Nights by heatwaves if you've never heard of it it's a great song to fully appreciate that song the way I do you have to believe that I am a genuine database whisperer then you have to picture me at 3 a.m. on my laptop tailing a vertical log getting myself all psyched up now as cool as they may sound 3 a.m. boogie nights are not sustainable they don't scale in fact today's discussion is really all about how Domo engineers the end of 3 a.m. boogie nights again well I am Ben white senior database engineer at Domo and as we heard the topic today the road to autonomous database management how Domo is delivering SLA for less the title is a mouthful in retrospect I probably could have come up with something snazzy er but it is I think honest for me the most honest word in that title is Road when I hear that word it evokes for me thoughts of the journey and how important it is to just enjoy it when you truly embrace the journey often you look up and wonder how did we get here where are we and of course what's next right now I don't intend to come across this too deep so I'll submit there's nothing particularly prescient and simply noticing the elephant in the room when it comes to database economy my opinion is then merely and perhaps more accurately my observation the office context imagine a place where thousands and thousands of users submit millions of ad-hoc queries every hour now imagine someone promised all these users that we could deliver bi leverage at cloud scale in record time I know what many of you should be thinking who in the world would do such a thing of course that news was well received and after the cheers from executives and business analysts everywhere and chance of Keep Calm and query on finally started to subside someone that turns an ass that's possible we can do that right except this is no imaginary place this is a very real challenge we face the demo through imaginative engineering demo continues to redefine what's possible the beautiful minds at Domo truly embrace the database engineering paradigm that one size does not fit all that little philosophical nugget is one I would pick up while reading the white papers and books of some guy named stone breaker so to understand how I and by extension Domo came to truly value analytic database administration look no further than that philosophy and what embracing it would mean it meant really that while others were engineering skyscrapers we would endeavor to build Datta neighborhoods with a diverse kapala G of database configuration this is where our journey at Domo really gets under way without any purposeful intent to define our destination not necessarily thinking about database as a service or anything like that we had planned this ecosystem of clusters capable of efficiently performing varied workloads we achieve this with custom configurations for node count resource pool configuration parameters etc but it also meant concerning ourselves with the unattended consequences of our ambition the impact of increased DDL activities on the catalog system overhead in general what would be the management requirements of an ever-evolving infrastructure we would be introducing multiple points of failure what are the advantages the disadvantages those types of discussions and considerations really help to define what would be the basic characteristics of our system the database itself needed to be trivial redundant potentially ephemeral customizable and above all scalable and we'll get more into that later with this knowledge of what we were getting into automation would have to be an integral part of development one might even say automation will become the first point of interest on our journey now using popular DevOps tools like saltstack terraform ServiceNow everything would be automated I mean it discluded everything from larger multi-step tasks like database designs database cluster creation and reboots to smaller routine tasks like license updates move-out and projection refreshes all of this cool automation certainly made it easier for us to respond to problems within the ecosystem these methods alone still if our database administration reactionary and reacting to an unpredictable stream of slow query complaints is not a good way to manage a database in fact that's exactly how three a.m. Boogie Nights happen and again I understand there was a certain appeal to them but ultimately managing that level of instability is not sustainable earlier I mentioned an elephant in the room which brings us to the second point of interest on our road to autonomy analytics more specifically analytic database administration why our analytics so important not just in this case but generally speaking I mean we have a whole conference set up to discuss it domo itself is self-service analytics the answer is curiosity analytics is the method in which we feed the insatiable human curiosity and that really is the impetus for analytic database administration analytics is also the part of the road I like to think of as a bridge the bridge if you will from automation to autonomy and with that in mind I say to you my fellow engineers developers administrators that as conductors of the symphony of data we call analytics we have proven to be capable producers of analytic capacity you take pride in that and rightfully so the challenge now is to become more conscientious consumers in some way shape or form many of you already employ some level of analytics to inform your decisions far too often we are using data that would be categorized as nagging perhaps you're monitoring slow queries in the management console better still maybe you consult the workflows analyzing how about a logging and alerting system like sumo logic if you're lucky you do have demo where you monitor and alert on query metrics like this all examples of analytics that help inform our decisions being a Domo the incorporation of analytics into database administration is very organic in other words pretty much company mandated as a company that provides BI leverage a cloud scale it makes sense that we would want to use our own product could be better at the business of doma adoption of stretches across the entire company and everyone uses demo to deliver insights into the hands of the people that need it when they need it most so it should come as no surprise that we have from the very beginning use our own product to make informed decisions as it relates to the application back engine in engineering we call it our internal system demo for Domo Domo for Domo in its current iteration uses a rules-based engine with elements through machine learning to identify and eliminate conditions that cause slow query performance pulling data from a number of sources including our own we could identify all sorts of issues like global query performance actual query count success rate for instance as a function of query count and of course environment timeout errors this was a foundation right this recognition that we should be using analytics to be better conductors of curiosity these types of real-time alerts were a legitimate step in the right direction for the engineering team though we saw ourselves in an interesting position as far as demo for demo we started exploring the dynamics of using the platform to not only monitor an alert of course but to also triage and remediate just how much economy could we give the application what were the pros and cons of that Trust is a big part of that equation trust in the decision-making process trust that we can mitigate any negative impacts and Trust in the very data itself still much of the data comes from systems that interacted directly and in some cases in directly with the database by its very nature much of the data was past tense and limited you know things that had already happened without any reference or correlation to the condition the mayor to those events fortunately the vertical platform holds a tremendous amount of information about the transaction it had performed its configurations the characteristics of its objects like tables projections containers resource pools etc this treasure trove of metadata is collected in the vertical system tables and the appropriately named data collector tables as a version 9 3 there are over 190 tables that define the system tables while the data collector is the collection of 215 components a rich collection can be found in the vertical system tables these tables provide a robust stable set of views that let you monitor information about your system resources background processes workload and performance allowing you to more efficiently profile diagnose and correlate historical data such as low streams query profiles to pool mover operations and more here you see a simple query to retrieve the names and descriptions of the system tables and an example of some of the tables you'll find the system tables are divided into two schemas the catalog schema contains information about persistent objects and the monitor schema tracks transient system States most of the tables you find there can be grouped into the following areas system information system resources background processes and workload and performance the Vertica data collector extends system table functionality by gathering and retaining aggregating information about your database collecting the data collector mixes information available in system table a moment ago I show you how you get a list of the system tables in their description but here we see how to get that information for the data collector tables with data from the data collecting tables in the system tables we now have enough data to analyze that we would describe as conditional or leading data that will allow us to be proactive in our system management this is a big deal for Domo and particularly Domo for demo because from here we took the critical next step where we analyze this data for conditions we know or suspect lead to poor performance and then we can suggest the recommended remediation really for the first time we were using conditional data to be proactive in a database management in record time we track many of the same conditions the Vertica support analyzes via scrutinize like tables with too many production or non partition fact tables which can negatively affect query performance and life in vertical in viral suggests if the table has a data a time step column you recommend the partitioning by the month we also can track catalog sizes percentage of total memory and alert thresholds and trigger remediations requests per hour is a very important metric in determining when a trigger are scaling solution tracking memory usage over time allows us to adjust resource pool parameters to achieve the optimal performance for the workload of course the workload analyzer is a great example of analytic database administration I mean from here one can easily see the logical next step where we were able to execute these recommendations manually or automatically be of some configuration parameter now when I started preparing for this discussion this slide made a lot of sense as far as the logical next iteration for the workload analyzing now I left it in because together with the next slide it really illustrates how firmly Vertica has its finger on the pulse of the database engineering community in 10 that OS management console tada we have the updated work lies will load analyzer we've added a column to show tuning commands the management console allows the user to select to run certain recommendations currently tuning commands that are louder and alive statistics but you can see where this is going for us using Domo with our vertical connector we were able to then pull the metadata from all of our clusters we constantly analyze that data for any number of known conditions we build these recommendations into script that we can then execute immediately the actions or we can save it to a later time for manual execution and as you would expect those actions are triggered by thresholds that we can set from the moment nyan mode was released to beta our team began working on a serviceable auto-scaling solution the elastic nature of AI mode separated store that compute clearly lent itself to our ecosystems requirement for scalability in building our system we worked hard to overcome many of the obstacles they came with the more rigid architecture of enterprise mode but with the introduction is CRM mode we now have a practical way of giving our ecosystem at Domo the architectural elasticity our model requires using analytics we can now scale our environment to match demand what we've built is a system that scales without adding management overhead or our necessary cost all the while maintaining optimal performance well we're really this is just our journey up to now and which begs the question what's next for us we expand the use of Domo for Domo within our own application stack maybe more importantly we continue to build logic into the tools we have by bringing machine learning and artificial intelligence to our analysis and decision making really do to further illustrate those priorities we announced the support for Amazon sage maker autopilot at our demo collusive conference just a couple of weeks ago for vertical the future must include in database economy the enhanced capabilities in the new management console to me are clear nod to that future in fact with a streamline and lightweight database design process all the pieces should be in place versions deliver economists database management itself we'll see well I would like to thank you for listening and now of course we will have a Q&A session hopefully very robust thank you [Applause]