hi everybody thank you for joining us today for the virtual Vertica BBC 2020 today's breakout session is entitled Vertica and en mode at the trade desk my name is su LeClair director of marketing at Vertica and I'll be your host for this webinar joining me is Ron Cormier senior Vertica database engineer at the trade desk before we begin I encourage you to submit questions or comments during the virtual session you don't have to wait just type your question or comment in the question box below the slides and click submit there will be a Q&A session at the end of the presentation we'll answer as many questions as we're able to during that time any questions that we don't address we'll do our best to answer them offline alternatively you can visit vertical forums to post your questions there after the session our engineering team is planning to join the forums to keep the conversation going also a quick reminder that you can maximize your screen by clicking the double arrow button in the lower right corner of the slide and yes this virtual session is being recorded and will be available to view on demand this week we'll send you a notification as soon as it's ready so let's get started over to you run thanks - before I get started I'll just mention that my slide template was created before social distancing was a thing so hopefully some of the images will harken us back to a time when we could actually all be in the same room but with that I want to get started uh the date before I get started in thinking about the technology I just wanted to cover my background real quick because I think it's peach to where we're coming from with vertically on at the trade desk and I'll start out just by pointing out that prior to my time in the trade desk I was a tech consultant at HP HP America and so I traveled the world working with Vertica customers helping them configure install tune set up their verdict and databases and get them working properly so I've seen the biggest and the smallest implementations and everything in between and and so now I'm actually principal database engineer straight desk and and the reason I mentioned this is to let you know that I'm a practitioner I'm working with with the product every day or most days this is a marketing material so hopefully the the technical details in this presentation are are helpful I work with Vertica of course and that is most relative or relevant to our ETL and reporting stack and so what we're doing is we're taking about the data in the Vertica and running reports for our customers and we're an ad tech so I did want to just briefly describe what what that means and how it affects our implementation so I'm not going to cover the all the details of this slide but basically I want to point out that the trade desk is a DSP it's a demand-side provider and so we place ads on behalf of our customers or agencies and ad agencies and their customers that are advertised as brands themselves and the ads get placed on to websites and mobile applications and anywhere anywhere digital advertising happens so publishers are what we think ocean like we see here espn.com msn.com and so on and so every time a user goes to one of these sites or one of these digital places and an auction takes place and what people are bidding on is the privilege of showing and add one or more ads to users and so this is this is really important because it helps fund the internet ads can be annoying sometimes but they actually help help are incredibly helpful in how we get much much of our content and this is happening in real time at very high volumes so on the open Internet there is anywhere from seven to thirteen million auctions happening every second of those seven to thirteen million auctions happening every second the trade desk bids on hundreds of thousands per second um so that gives it and anytime we did we have an event that ends up in Vertica that's that's one of the main drivers of our data volume and certainly other events make their way into Vertica as well but that wanted to give you a sense of the scale of the data and sort of how it's impacting or how it is impacted by sort of real real people in the world so um the uh let's let's take a little bit more into the workload and and we have the three B's in spades late like many many people listening to a massive volume velocity and variety in terms of the data sizes I've got some information here some stats on on the raw data sizes that we deal with on a daily basis per day so we ingest 85 terabytes of raw data per day and then once we get it into Vertica we do some transformations we do matching which is like joins basically and we do some aggregation group buys to reduce the data and make it clean it up make it so it's more efficient to consume buy our reporting layer so that matching in aggregation produces about ten new terabytes of raw data per day it all comes from the it all comes from the data that was ingested but it's new data and so that's so it is reduced quite a bit but it's still pretty pretty high high volume and so we have this aggregated data that we then run reports on on behalf of our customers so we have about 40,000 reports per day oh that's probably that's actually a little bit old and older number it's probably closer to 50 or 55,000 reports per day at this point so it's I think probably a pretty common use case for for Vertica customers it's maybe a little different in the sense that most of the reports themselves are >> reports so they're not it's not a user sitting at a keyboard waiting for the result basically we have we we have a workflow where we do the ingest we do this transform and then and then once once all the data is available for a day we run reports on behalf of our customer to let me have our customers on that that daily data and then we send the reports out you via email or we drop them in a shared location and then they they look at the reports at some later point of time so it's up until yawn we did all this work on on enterprise Vertica at our peak we had four production enterprise clusters each which held two petabytes of raw data and I'll give you some details on on how those enterprise clusters were configured in the hardware but before I do that I want to talk about the reporting workload specifically so the the reporting workload is particularly lumpy and what I mean by that is there's a bunch of work that becomes available bunch of queries that we need to run in a short period of time after after the days just an aggregation is completed and then the clusters are relatively quiet for the remaining portion of the day that's not to say they are they're not doing anything as far as read workload but they certainly are but it's much less reactivity after that big spike so what I'm showing here is our reporting queue and the spike is is when all those reports become a bit sort of ailable to be processed we can't we can't process we can't run the report until we've done the full ingest and matching and aggregation for the day and so right around 1:00 or 2:00 a.m. UTC time every day that's when we get this spike and the spike we affectionately called the UTC hump but basically it's a huge number of queries that need to be processed sort of as soon as possible and we have service levels that dictate what as soon as possible means but I think the spike illustrates our use case pretty pretty accurately and um it really as we'll see it's really well suited for pervert icky on and we'll see what that means so we've got our we had our enterprise clusters that I mentioned earlier and just to give you some details on what they look like there they were independent and mirrored and so what that means is all four clusters held the same data and we did this intentionally because we wanted to be able to run our report anywhere we so so we've got this big queue over port is big a number of reports that need to be run and we've got these we started we started with one cluster and then we got we found that it couldn't keep up so we added a second and we found the number of reports went up that we needed to run that short period of time and and so on so we eventually ended up with four Enterprise clusters basically with this with the and we'd say they were mirrored they all had the same data they weren't however synchronized they were independent and so basically we would run the the tailpipe line so to speak we would run ingest and the matching and the aggregation on all the clusters in parallel so they it wasn't as if each cluster proceeded to the next step in sync with which dump the other clusters they were run independently so it was sort of like each each cluster would eventually get get consistent and so this this worked pretty well for for us but it created some imbalances and there was some cost concerns that will dig into but just to tell you about each of these each of these clusters they each had 50 nodes they had 72 logical CPU cores a half half a terabyte of RAM a bunch of raid rated disk drives and 2 petabytes of raw data as I stated before so pretty big beefy nodes that are physical physical nodes that we held we had in our data centers we actually reached these nodes so so it was on our data center providers data centers and the these were these these were what we built our business on basically but there was a number of challenges that we ran into as we as we continue to build our business and add data and add workload and and the first one is is some in ceremony can relate to his capacity planning so we had to prove think about the future and try to predict the amount of work that was going to need to be done and how much hardware we were going to need to satisfy that work to meet that demand and that's that's just generally a hard thing to do it's very difficult to verdict the future as we can probably all attest to and how much the world has changed and even in the last month so it's a it's a very difficult thing to do to look six twelve eighteen eighteen months into the future and sort of get it right and and and what people what we tended to do is we reach or we tried to our art plans our estimates were very conservative so we overbought in a lot of cases and not only that we had to plan for the peak so we're planning for that that that point in time that those number of hours in the early morning when we had to we had all those reports to run and so that so so we ended up buying a lot of hardware and we actually sort of overbought at times and then and then as the hardware were days it would kind of come into it would come into maturity and we have our our our workload would sort of come approach matching the demand so that was one of the big challenges the next challenge is that we were running on disk you can we wanted to add data in sort of two dimensions the only dimensions that everybody can think about we wanted to add more columns to our big aggregates and we wanted to keep our big aggregates for for longer periods of time so both horizontally and vertically we wanted to expand the datasets but we basically were running out of disk there was no more disk in and it's hard to add a disc to Vertica in enterprise mode not not impossible but certainly hard and and one cannot add discs without adding compute because enterprise mode the disk is all local to each of the nodes for most most people you can do not exchange with sands and other external rays but that's there are a number of other challenges with that so um adding in order to add disk we had to add compute and that basically meant kept us out of balance we're adding more compute than we needed for the amount of disk so that was the problem certainly physical nodes getting them the order delivered racked cables even before we even start such Vertica there's lead times there and and so it's also long commitment since we like I mentioned me Lisa hardware so we were committing to these nodes these physical servers for two or three years at a time and I mentioned that can be a hard thing to do but we wanted to least to keep our capex down so we wanted to keep our aggregates for a long period of time we could have done crazy things or more exotic things to to help us with this if we had to in enterprise mode we could have started to like daisy chain clusters together and that would have been sort of a non-trivial engineering effort because we would need to then figure out how to migrate data source first to recharge the data across all the clusters and we had to migrate data from one cluster to another cluster hesitation and we would have to think about how to aggregate run queries across clusters so if you assured data set spans two clusters it would have had to sort of aggregated within each cluster maybe and then build something on top the aggregated the data from each of those clusters so not impossible things but certainly not easy things and luckily for us we started talking about two Vertica about separation of compute and storage and I know other customers were talking to Vertica as we were people had had these problems and so Vertica inyeon mode came to the rescue and what I want to do is just talk about nyan mode really briefly for for those in the audience who aren't familiar but it's basically Vertigo's answered to the separation of computing storage it allows one to scale compute and or storage separately and and this there's a number of advantages to doing that whereas in the old enterprise days when you add a compute you added stores and vice-versa now we can now we can add one or the other or both according to how we want to and so really briefly how this works this slide this figure was taken directly from the verdict and documentation and so just just to talk really briefly about how it works the taking advantage of the cloud and so in this case Amazon Web Services the elasticity in the cloud and basically we've got you seen two instances so elastic cloud compute servers that access data that's in an s3 bucket and so three three ec2 nodes and in a bucket or the the blue objects in this diagram and the difference is a couple of a couple of big differences one the data no longer the persistent storage of the data the data where the data lives is no longer on each of the notes the persistent stores of the data is in s3 bucket and so what that does is it basically solves one of our first big problems which is we were running out of disk the s3 has for all intensive purposes infinite storage so we can keep much more data there and that mostly solved one of our big problems so the persistent data lives on s3 now what happens is when a query runs it runs on one of the three nodes that you see here and assuming we'll talk about depo in a second but what happens in a brand new cluster where it's just just spun up the hardware is the query will will run on those ec2 nodes but there will be no data so those nodes will reach out to s3 and run the query on remote storage so that so the query that the nodes are literally reaching out to the communal storage for the data and processing it entirely without using any data on on the nodes themselves and so that that that works pretty well it's not as fast as if the data was local to the nodes but um what Vertica did is they built a caching layer on on each of the node and that's what the depot represents so the depot is some amount of disk that is relatively local to the ec2 node and so when the query runs on remote stores on the on the s3 data it then queues up the data for download to the nodes and so the data will get will reside in the Depot so that the next query or the subsequent subsequent queries can run on local storage instead of remote stores and that speeds things up quite a bit so that that's that's what the role of the Depot is the depot is basically a caching layer and we'll talk about the details of how we can see your in our Depot the other thing that I want to point out is that since this is the cloud another problem that helps us solve is the concurrency problem so you can imagine that these three nodes are one sort of cluster and what we can do is we can spit up another three nodes and have it point to the same s3 communal storage bucket so now we've got six nodes pointing to the same data but we've you isolated each of the three nodes so that they act as if they are their own cluster and so vertical calls them sub-clusters so we've got two sub clusters each of which has three nodes and what this has essentially done it is it doubled the concurrency doubled the number of queries that can run at any given time because we've now got this new place which new this new chunk of compute which which can answer queries and so that has given us the ability to add concurrency much faster and I'll point out that for since it's cloud and and there are on-demand pricing models we can have significant savings because when a sub cluster is not needed we can stop it and we pay almost nothing for it so that's that's really really important really helpful especially for our workload which I pointed out before was so lumpy so those hours of the day when it's relatively quiet I can go and stop a bunch of sub clusters and and I will pay for them so that that yields nice cost savings let's be on in a nutshell obviously engineers and the documentation can use a lot more information and I'm happy to field questions later on as well but I want to talk about how how we implemented beyond at the trade desk and so I'll start on the left hand side at the top the the what we're representing here is some clusters so there's some cluster 0 r e t l sub cluster and it is a our primary sub cluster so when you get into the world of eon there's primary Club questions and secondary sub classes and it has to do with quorum so primary sub clusters are the sub clusters that we always expect to be up and running and they they contribute to quorum they decide whether there's enough instances number a number of enough nodes to have the database start up and so these this is where we run our ETL workload which is the ingest the match in the aggregate part of the work that I talked about earlier so these nodes are always up and running because our ETL pipeline is always on we're internet ad tech company like I mentioned and so we're constantly getting costly running ad and there's always data flowing into the system and the matching is happening in the aggregation so that part happens 24/7 and we wanted so that those nodes will always be up and running and we need this we need that those process needs to be super efficient and so what that is reflected in our instance type so each of our sub clusters is sixty four nodes we'll talk about how we came at that number but the infant type for the ETL sub cluster the primary subclusters is I 3x large so that is one of the instance types that has quite a bit of nvme stores attached and we'll talk about that but on 32 cores 240 four gigs of ram on each node and and that what that allows us to do I should have put the amount of nvme but I think it's seven terabytes for anything me storage what that allows us to do is to basically ensure that our ETL everything that this sub cluster does is always in Depot and so that that makes sure that it's always fast now when we get to the secondary subclusters these are as mentioned secondary so they can stop and start and it won't affect the cluster going up or down so they're they're sort of independent and we've got four what we call Rhian subclusters and and they're not read by definition or technically they're not read only any any sub cluster can ingest and create your data within the database and that'll all get that'll all get pushed to the s3 bucket but logically for us they're read only like these we just most of these the work that they happen to do is read only which it is which is nice because if it's read only it doesn't need to worry about commits and we let we let the primary subclusters or ETL so close to worry about committing data and we don't have to we don't have to have the all nodes in the database participating in transaction commits so we've got a for read subclusters and we've got one EP also cluster so a total of five sub clusters each so plus they're running sixty-four nodes so that gives us a 320 node database all things counted and not all those nodes are up at the same time as I mentioned but often often for big chunks of the days most of the read nodes are down but they do all spin up during our during our busy time so for the reading so clusters we've got I three for Excel so again the I three incidents family type which has nvme stores these notes have I think three and a half terabytes of nvme per node we just rate it to nvme drives we raid zero them together and 16 cores 122 gigs of ram so these are smaller you'll notice but it works out well for us because the the read workload is is typically dealing with much smaller data sets than then the ingest or the aggregation workbook so we can we can run these workloads on on smaller instances and leave a little bit of money and get more granularity with how many sub clusters are stopped and started at any given time the nvme doesn't persist the data on it isn't persisted remember you stop and start this is an important detail but it's okay because the depot does a pretty good job in that in that algorithm where it pulls data in that's recently used and the that gets pushed out a victim is the data that's least reasons use so it was used a long time ago so it's probably not going to be used to get so we've got um five sub-clusters and we have actually got to two of those so we've got a 320 node cluster in u.s. East and a 320 node cluster in u.s. West so we've got a high availability region diversity so and their peers like I talked about before they're they're independent but but yours they are each run 128 shards and and so with that what that which shards are is basically the it's similar to segmentation when you take those dataset you divide it into chunks and though and each sub cluster can concede want the data set in its entirety and so each sub cluster is dealing with 128 shards it shows 128 because it'll give us even distribution of the data on 64 node subclusters 60 120 might evenly by 64 and so there's so there's no data skew and and we chose 128 because the sort of ginger proof in case we wanted to double the size of any of the questions we can double the number of notes and we still have no excuse the data would be distributed evenly the disk what we've done is so we've got a couple of raid arrays we've got an EBS based array that they're catalog uses so the catalog storage location and I think we take for for EBS volumes and raid 0 them together and come up with 128 gigabyte Drive and we wanted an EPS for the catalog because it we can stop and start nodes and that data will persist it will come back when the node comes up so we don't have to run a bunch of configuration when the node starts up basically the node starts it automatically joins the cluster and and very strongly there after it starts processing work let's catalog and EBS now the nvme is another raid zero as I mess with this data and is ephemeral so let me stop and start it goes away but basically we take 512 gigabytes of the nvme and we give it to the data temp storage location and then we take whatever is remaining and give it to the depot and since the ETL and the reading clusters are different instance types they the depot is is side differently but otherwise it's the same across small clusters also it all adds up what what we have is now we we stopped the purging data for some of our big a grits we added bunch more columns and what basically we at this point we have 8 petabytes of raw data in each Jian cluster and it is obviously about 4 times what we can hold in our enterprise classes and we can continue to add to this maybe we need to add compute maybe we don't but the the amount of data that can can be held there against can obviously grow much more we've also built in auto scaling tool or service that basically monitors the queue that I showed you earlier monitors for those spikes I want to see as low spikes it then goes and starts up instances one sub-collector any of the sub clusters so that's that's how that's how we we have compute match the capacity match that's the demand also point out that we actually have one sub cluster is a specialized nodes it doesn't actually it's not strictly a customer reports sub clusters so we had this this tool called planner which basically optimizes ad campaigns for for our customers and we built it it runs on Vertica uses data and Vertica runs vertical queries and it was it was wildly successful um so we wanted to have some dedicated compute and beyond witty on it made it really easy to basically spin up one of these sub clusters or new sub cluster and say here you go planner team do what you want you can you can completely maximize the resources on these nodes and it won't affect any of the other operations that were doing the ingest the matching the aggregation or the reports up so it gave us a great deal of flexibility and agility which is super helpful so the question is has it been worth it and without a doubt the answer is yes we're doing things that we never could have done before sort of with reasonable cost we have lots more data specialized nodes and more agility but how do you quantify that because I don't want to try to quantify it for you guys but it's difficult because each eon we still have some enterprise nodes by the way cost as you have two of them but we also have these Eon clusters and so they're there they're running different workloads the aggregation is different the ingest is running more on eon does the number of nodes is different the hardware is different so there are significant differences between enterprise and and beyond and when we combine them together to do the entire workload but eon is definitely doing the majority of the workload it has most of the data it has data that goes is much older so it handles the the heavy heavy lifting now the query performance is more anecdotal still but basically when the data is in the Depot the query performance is very similar to enterprise quite close when the data is not in Depot and it needs to run our remote storage the the query performance is is is not as good it can be multiples it's not an order not orders of magnitude worse but certainly multiple the amount of time that it takes to run on enterprise but the good news is after the data downloads those young clusters quickly catch up as the cache populates there of cost I'd love to be able to tell you that we're running to X the number of reports or things are finishing 8x faster but it's not that simple as you Iran is that you it is me I seem to have gotten to thank you you hear me okay I can hear you now yeah we're still recording but that's fine we can edit this so if I'm just talking to the person the support person he will extend our recording time so if you want to maybe pick back up from the beginning of the slide and then we'll just edit out this this quiet period that we have sir okay great I'm going to go back on mute and why don't you just go back to the previous slide and then come into this one again and I'll make sure that I tell the person who yep perfect and then we'll continue from there is that okay yeah sound good all right all right I'm going back on yet so the question is has it been worth it and for us the answer has been a resounding yes we're doing things that we never could have done at reasonable cost before and we got more data we've got this Y note this law has nodes and in work we're much more agile so how to quantify that um well it's not quite as simple and straightforward as you might hope I mean we still have enterprise clusters we've got to update the the four that we had at peak so we've still got two of those around and we got our two yawn clusters but they're running different workloads and they're comprised of entirely different hardware the dependence has I've covered the number of nodes is different for sub-clusters so 64 versus 50 is going to have different performance the the workload itself the aggregation is aggregating more columns on yon because that's where we have disk available the queries themselves are different they're running more more queries on more intensive data intensive queries on yon because that's where the data is available so in a sense it is Jian is doing the heavy lifting for the cluster for our workload in terms of query performance still a little anecdotal but like when the queries that run on the enterprise cluster the performance matches that of the enterprise cluster quite closely when the data is in the Depot when the data is not in a Depot and Vertica has to go out to the f32 to get the data performance degrades as you might expect it can but it depends on the curious all things like counts counts are is really fast but if you need lots of the data from the material others to realize lots of columns that can run slower I'm not orders of magnitude slower but certainly multiple of the amount of time in terms of costs anecdotal will give a little bit more quantifying here so what I try to do is I try to figure out multiply it out if I wanted to run the entire workload on enterprise and I wanted to run the entire workload on e on with all the data we have today all the queries everything and to try to get it to the Apple tab so for enterprise the the and estimate that we do need approximately 18,000 cores CPU cores all together and that's a big number but that's doesn't even cover all the non-trivial engineering work that would need to be required that I kind of referenced earlier things like starting the data among multiple clusters migrating the data from one culture to another the daisy chain type stuff so that's that's the data point now for eon is to run the entire workload estimate we need about twenty thousand four hundred and eighty CPU cores so more CPU cores uh then then enterprise however about half of those and partly ten thousand of both CPU cores would only run for about six hours per day and so with the on demand and elasticity of the cloud that that is a huge advantage and so we are definitely moving as fast as we can to being on all Aeon we have we have time left on our contract with the enterprise clusters or not we're not able to get rid of them quite yet but Eon is certainly the way of the future for us I also want to point out that uh I mean yawn is we found to be the most efficient MPP database on the market and what that refers to is for a given dollar of spend of cost we get the most from that zone we get the most out of Vertica for that dollar compared to other cloud and MPP database platforms so our business is really happy with what we've been able to deliver with Yan Yan has also given us the ability to begin a new use case which is probably this case is probably pretty familiar to folks on the call where it's UI based so we'll have a website that our customers can log into and on that website they'll be able to run reports on queries through the website and have that run directly on a separate row to get beyond cluster and so much more latent latency sensitive and concurrency sensitive so the workflow that I've described up until this point has been pretty steady throughout the day and then we get our spike and then and then it goes back to normal for the rest of the day this workload it will be potentially more variable we don't know exactly when our engineers are going to deliver some huge feature that is going to make a 1-1 make a lot of people want to log into the website and check how their campaigns are doing so we but Yohn really helps us with this because we can add a capacity so easily we cannot compute and we can add so we can scale that up and down as needed and it allows us to match the concurrency so beyond the concurrency is much more variable we don't need a big long lead time so we're really excited about about this so last slide here I just want to leave you with some things to think about if you're about to embark or getting started on your journey with vertically on one of the things that you'll have to think about is the no account in the shard count so they're kind of tightly coupled the node count we determined by figuring like spinning up some instances in a single sub cluster and getting performance smaller to finding an acceptable performance considering current workload future workload for the queries that we had when we started and so we went with 64 we wanted to you want to certainly want to increase over 50 but we didn't want to have them be too big because of course it costs money and so what you like to do things in power to so 64 nodes and then the shard count for the shards again is like the data segmentation is a new type of segmentation on the data and the start out we went with 128 it began the reason is so that we could have no skew but you know could process the same same amount of data and we wanted to future-proof it so that's probably it's probably a nice general recommendation doubleness account for the nodes the instance type and and how much people space those are certainly things you're going to consider like I was talking about we went for they I three for Excel I 3/8 Excel because they offer good good Depot stores which gives us a really consistent good performance and it is all in Depot the pretty good mud presentation and some information on on I think we're going to use our r5 or the are for instance types for for our UI cluster so much less the data smaller so much less enter this on Depot so we don't need on that nvm you stores the reader we're going to want to have a reserved a mix of reserved and on-demand instances if you're if you're 24/7 shop like we are like so our ETL subclusters those are reserved instances because we know we're going to run those 24 hours a day 365 days a year so there's no advantage of having them be on-demand on demand cost more than reserve so we get cost savings on on figuring out what we're going to run and have keep running and it's the read subclusters that are for the most part on on demand we have one of our each sub Buster's is actually on 24/7 because we keep it up for ad-hoc queries your analyst queries that we don't know when exactly they're going to hit and they want to be able to continue working whenever they want to in terms of the initial data load the initial data ingest what we had to do and now how it works till today is you've got to basically load all your data from scratch there isn't a great tooling just yet for data populate or moving from enterprise to Aeon so what we did is we exported all the data in our enterprise cluster into park' files and put those out on s3 and then we ingested them into into our first Eon cluster so it's kind of a pain we script it out a bunch of stuff obviously but they worked and the good news is that once you do that like the second yon cluster is just a bucket copy in it and so there's tools missions that can help help with that you're going to want to manage your fetches and addiction so this is the data that's in the cache is what I'm referring to here the data that's in the default and so like I talked about we have our ETL cluster which has the most recent data that's just an injected and the most difficult data that's been aggregated so this really recent data so we wouldn't want anybody logging into that ETL cluster and running queries on big aggregates to go back one three years because that would invalidate the cache the depot would start pulling in that historical data and it was our assessing that historical data and evicting the recent data which would slow things out flow down that ETL pipelines so we didn't want that so we need to make sure that users whether their service accounts or human users are connecting to the right phone cluster and I mean we just created the adventure users with IPS and target groups to palm those pretty-pretty it was definitely something to think about lastly if you're like us and you're going to want to stop and start nodes you're going to have to have a service that does that for you we're where we built this very simple tool that basically monitors the queue and stops and starts subclusters accordingly we're hoping that that we can work with Vertica to have it be a little bit more driven by the cloud configuration itself so for us all amazon and we love it if we could have it have a scale with the with the with the eight of us can take through points do things to watch out for when when you're working with Eon is the first is system table queries on storage layer or metadata and the thing to be careful of is that the storage layer metadata is replicated it's caught as a copy for each of the sub clusters that are out there so we have the ETL sub cluster and our resources so for each of the five sub clusters there is a copy of all the data in storage containers system table all the data and partitions system table so when you want to use this new system tables for analyzing how much data you have or any other analysis make sure that you filter your query with a node name and so for us the node name is less than or equal to 64 because each of our sub clusters at 64 so we limit we limit the nodes to the to the 64 et 64 node ETL collector otherwise if we didn't have this filter we would get 5x the values for counts and some sort of stuff and lastly there is a problem that we're kind of working on and thinking about is a DC table data for sub clusters that are our stops when when the instances stopped literally the operating system is down and there's no way to access it so it takes the DC table DC table data with it and so I cannot after after my so close to scale up in the morning and then they scale down I can't run DC table queries on how what performed well and where and that sort of stuff because it's local to those nodes so we're working on something so something to be aware of and we're working on a solution or an implementation to try to suck that data out of all the notes you can those read only knows that stop and start all the time and bring it in to some other kind of repository perhaps another vertical cluster so that we can run analysis and monitoring even you want those those are down that's it um thanks for taking the time to look into my presentation really do it thank you Ron that was a tremendous amount of information thank you for sharing that with everyone um we have some questions come in that I would like to present to you Ron if you have a couple min it your first let's jump right in the first one a loading 85 terabytes per day of data is pretty significant amount what format does that data come in and what does that load process look like yeah a great question so the format is a tab separated files that are Jesus compressed and the reason for that could basically historical we don't have much tabs in our data and this is how how the data gets compressed and moved off of our our bidders the things that generate most of this data so it's a PSD gzip compressed and how you kind of we kind of have how we load it I would say we have actually kind of a Cadillac loader in a couple of different perspectives one is um we've got this autist raishin layer that's homegrown managing the logs is the data that gets loaded into Vertica and so we accumulate data and then we take we take some some files and we push them to redistribute them along the ETL nodes in the cluster and so we're literally pushing the file to through the nodes and we then run a copy statement to to ingest data in the database and then we remove the file from from the nodes themselves and so it's a little bit extra data movement which you may think about changing in the future assisting we move more and more to be on well the really nice thing about this especially for for the enterprise clusters is that the copy' statements are really fast and so we the coffee statements use memory but let's pick any other query but the performance of the cautery statement is really sensitive to the amount of available memory and so since the data is local to the nodes literally in the data directory that I referenced earlier it can access that data from the nvme stores and the kabhi statement runs very fast and then that memory is available to do something else and so we pay a little bit of cost in terms of latency and in terms of downloading the data to the nose we might as we move more and more PC on we might start ingesting it directly from s3 not copying the nodes first we'll see about that what's there that's how that's how we read the data interesting works great thanks Ron um another question what was the biggest challenge you found when migrating from on-prem to AWS uh yeah so um a couple of things that come to mind the first was the baculum the data load it was kind of a pain I mean like I referenced in that last slide only because I mean we didn't have tools built to do this so I mean we had to script some stuff out and it wasn't overly complex but yes it's just a lot of data to move I mean even with starting with with two petabytes so making sure that there there is no missed data no gaps making and moving it from the enterprise cluster so what we did is we exported it to the local disk on the enterprise buses and we then we push this history and then we ingested it in ze on again Allspark X oh so it's a lot of days to move around and I mean we have to you have to take an outage at some point stop loading data while we do that final kiss-up phase and so that was that was a challenge a sort of a one-time challenge the other saying that I mean we've been dealing with a week not that we're dealing with but with his challenge was is I mean it's relatively you can still throw totally new product for vertical and so we are big advantages of beyond is allow us to stop and start nodes and recently Vertica has gotten quite good at stopping in part starting nodes for a while there it was it was it took a really long time to start to Noah back up and it could be invasive but we worked with with the engineering team with Yan Zi and others to really really reduce that and now it's not really an issue that we think that we think too much about hey thanks towards the end of the presentation you had said that you've got 128 shards but you have your some clusters are usually around 64 nodes and you had talked about a ratio of two to one why is that and if you were to do it again would you use 128 shards ah good question so that is a reference the reason why is because we wanted to future professionals so basically we wanted to make sure that the number of stars was evenly divisible by the number of nodes and you could I could have done that was 64 I could have done that with 128 or any other multiple entities for but we went with 128 is to try to protect ourselves in the future so that if we wanted to double the number of nodes in the ECL phone cluster specifically we could have done that so that was double from 64 to 128 and then each node would have happened just one chart that it had would have to deal with so so no skew um the second part of question if I had to do it if I had to do it over again I think I would have done I think I would have stuck with 128 we still have I mean so we either running this cluster for more than 18 months now I think especially in USC and we haven't needed to increase the number of nodes so in that sense like it's been a little bit extra overhead having more shards but it gives us the peace of mind that we can easily double that and not have to worry about it so I think I think everyone is a nice place to start and you may even consider a three to one or four to one if if you're if you're expecting really rapid growth that you were just getting started with you on and your business and your gates that's a small now but what you expect to have them grow up significantly less powerful green thank you Ron that's with all the questions that we have out there for today if you do have others please feel free to send them in and we will get back to you and we'll respond directly via email and again our engineers will be available on the vertical forums where you can continue the discussion with them there I want to thank Ron for the great presentation and also the audience for your participation in questions please note that a replay of today's event and a copy of the slides will be available on demand shortly and of course we invite you to share this information with your colleagues as well again thank you and this concludes this webinar and have a great day you

(upbeat techno music) >> Live from Las Vegas, it's theCUBE. Covering Boomi World 2018. Brought to you by Dell Boomi. >> Good afternoon, welcome back to theCUBE's continuing coverage of Boomi World 2018 from Las Vegas. I'm Lisa Martin with John Furrier and we're welcoming back to theCUBE one of our alumni Nima Badiey, Head of Technology Ecosystems from Pivotal. Nima, welcome back. >> Thank you for having me back. >> So Pivotal, part of the Dell technologies part of the companies, >> Yeah. >> You guys IPOd recently. And I did read that of the first half 2018, eight of the 10 tech IPOs were powered by Boomi. >> Well, I don't know about that specific. I know that tech IPOs are making a big comeback. We did IPO on the 20th of April, so we've passed out six-month anniversary if you can say. But it's been a distinct privilege to be part of the overall Dell family of businesses. I think what you have in Michael as a leader, who, he has a specific vision, but he's left the independent operating units to work on their own, to find their path through that journey, and to help each other as brethren, as like sisters and brothers. And the fact that Pivotal is here supporting Boomi. That Boomi is within our conference of supporting our customers that we're working together really speaks volumes. I think if you take a look at it, a lot of things happened this week, right? So a couple weeks ago, IBM's acquiring RedHat, this morning VMWare's acquiring Heptio. That's a solid signal that the enterprise transformation and adoption of cloud native model is really taking off. So the new middleware is really all about the cloud native polyglock, multiglock environment. >> And what's interesting, I want to get your thoughts on this because first of all congratulations on the IP, some are saying Pivotal's never going to go public, and they did, you guys were spectacular, great success. But what's going on now is interesting. We're hearing here at this show, as other shows is, cloud scale and data are really at the center of this horizontally scalable cloud poly proposition. Okay great, you mention Kubernetes and Heptio and VM where, that's all great. The question that is how do you compete when ecosystems become the most important thing. You worked at VMware you're at Pivotal. Dell knows ecosystems. Boomi's got an ecosystem. Partners, which is also suppliers and integrators. >> Yeah. >> They integrate and also developers. This is a key competitive advantage. What's your take on that here? >> So I think you touched on the right point. You compete because of your ecosystem, not despite your ecosystem. We can't be completely hedgemonic like Microsoft or Cisco or Amazon can afford to be. And I don't think customers really want that. Customers actually want choice. They want the best options but from a variety of sources. And that's why one of the reasons that we not only invest Dell ecosystem but also in Pivotal's own ecosystem is to cultivate the right technologies that will help our customers on that journey. And our philosophy's always find the leaders in the quadrant. The Cadillac vendors, the Lexus vendors onboard them and the most important thing you can do is, to ensure a pristine customer experience. We're not measuring whether feature A from one partner is better than feature B from another partner. We really don't care. What we care about is we can hand wire and automate what would have been a very manual process for customers, so that, let's say Boomi with Cloud Foundry works perfectly out of the box. So the customers doesn't have to go through and hire consultants and additional external resources just to figure out how two pieces of software should work together, they just should. So when they make that buying decision they know that the day after that buying decision, everything's going to be installed and their developers and their app dev teams and their ops teams can be productive. So that's the power of the ecosystem. >> Can you talk about the relationship between Pivotal and Boomi, because Boomi's been born in the Cloud as start up. Acquired eight years ago. You're part of the Dell Technologies family. VMware's VMware, we know about VMware doing great. You guys doing great. Now Boomi's out there. So how do they factor into and what's the relationship you have with them and how does that work, how do you guys work together? >> Perfect question. So, in my primary role at Pivotal is to manage all of our partner ecosystems, specifically the technology partners. And what I look for are any force multipliers. Any essentially ISVs who can help us accomplish more together than we could on our own. Boomi's a classic example of that. What do they enable? So take your classic customer. Classic customer has, let's say, 100 applications in inventory that they have built, managed, and purchased procured off from shelf-to-shelf components. And roughly 20 or 30% are newish, green field applications, perfect for the cloud native transformation. Most 80% of them or 70% are going to be older, ground field applications that will have to be refactored. But there's always going to be that 15% towards the end that's legacy mainframe. It can't be changed, you cannot afford to modernize it, to restructure it, to refactor it. You're going to have to leave it alone, but you need it. Your inventory systems are there. >> These are critical systems, those people who think legacy as outdated, but they're actually just valued. >> No, they're critically valuable. >> Yes. >> We just cannot be modernized. >> Bingo. >> So a partner like Boomi will allow you to access the full breadth of those resources without having to change them. So I could potentially put Boomi in front of any number of older business applications and effectively modernize them by bridging those older legacy systems with the new systems that I want to build. So let's do an example. I am the Gap and I want to build a new version of our in-store procurement system that runs on my iPhone, that I can just point to a garment and it will automatically put it in my, ya know, check out box. How do I do that? Well I can build all the intelligence. And I can use AI and functions and I can build everything it's out of containers, that's great. But I still have to connect to the inventory system. Inventory system... >> Which is a database. All these systems are out there. >> Somewhere, something. And my developers don't know enough about the old legacy database to be able to use it. But if I put a restful interface using Boomi in front of it and a business connector that's not older XML or kind of inflexible, whatever, solo gateways. Then I have enabled my developer to actually build something that is real. That is customer focused. It is appropriate for that market without being hamstrung by my existing legacy infrastructure. And now my legacy infrastructure is not an anchor that's holding me back. >> You had mentioned force, me and Lisa talk about this all the time on theCUBE, where that scenario's totally legit and relevant because in the old version of IT you have to essentially build inventory management into the new app. You'd have to essentially kill the old to bring in the new. I think with containers and cloud native has shown is you can keep the old and sunset it if you want on your own time table or keep it there and make it productive. Make the data exposeble, but you can bring the cool relevant new stuff in. >> Yeah. >> I think that is what I see and we see from customers, like OK cool, I don't have to kill the old. I'll take care of it on my own timetable versus a complete switching cost analysis. Take down a production system. >> Exactly. >> Build something new, will it work. Ya know cross your fingers. Okay, again and this is a key IT different dynamic. >> It is and it's a realization that there are things you can move and those are immutable. They're simply just monolithic that will never move. And you're going to work within those confines. You can have the best of both worlds. You can maintain your legacy applications. They're still fine, they run most of your business. And still invent the new and explore new markets and new industries and new verticals. And just new capabilities all through and through without having to touch in your back end systems. Without having to bring the older vendors in and say can you please modernize your stuff because my business is dependent and I am going to lose that. I'm going to become the new Sears, I going to become the new Woolworth or whoever. Blockbuster that has missed an opportunity to vector into a new way of delivering their services. >> When you're having customer conversations, Nima, I'm curious, talking with enterprise organizations who have tons of data, all the systems including the legacy, which I'm glad that you brought up that that's not just old systems. There's a lot of business critical, mission critical application running on 'em. Where do you start that conversation with the large enterprise, who doesn't want to become a Blockbuster to your point, and going this is the suite of applications we have, where do we start? Talk to us about that customer journey that you help enable. >> That's great 'cause in most cases the customers already know exactly what they want. It's not the what that you have to have the conversation around, it's the how do I get there. I know what I want, I know what I want to be, I know what I want to design. And it's how do I transform my business fundamentally do an app transformation, enterprise transformation, digital transformation? Where do I begin? And so, ya know, our perspective at Pivotal is, ya know, we're diehard adopters of agile methodology. We truly, truly believe that you can be an agile development organization. We truly believe in Marc Andreessen's vision of software eating the world. Which let's unpack what that means. It just means that if you're going to survive the next 10 years you have to fundamentally become a software company, right? So look at all the companies we work with. Are you an insurance company or are you delivering an insurance product through software? Are you a bank or are you delivering banking product through software? Well, when was the last time you talked to a bank teller? Or the atm, most of your banking's done online. Your computer or your mobile device. Even my check cashing, I don't have to talk to anyone. It's wonderful. Ford Motor Company, do they bend sheet metal and put wheels on it or are they a software company? Well consider that your modern pickup truck has... >> They're an IOT company now. (laughing) (crosstalking) Manufacturing lines. >> That's what's crazy. You have a 150 million lines of code in your pickup truck. Your car, your pickup truck, your whatever is more software than it is anything else. >> But also data's key. I want to get your thoughts since this is super important Michael Dell brought up on the keynote today here at Boomi World was, okay the data's got to stay in the car. I don't need to have a latency issue of hey, I need to know nanosecond results. With data, cloud has become a great use case. With multicloud on the horizon, some people are going to throw data in multiple clouds and that's clear use case, and everyone can see the benefits of that. How do you guys look at this? 'Cause now data needs to be addressable across horizontal systems. You mentioned the Gap and the Gap example. >> That's great, so, one of the biggest trends we see in data is really event streaming. Is the idea that the ability to generate data far out exceeds the ability to consume it. So, what if we treated data as just a river? And I'm going to cast my line and only pick up what I want out of that stream. And this is where CAFCA and companies like Solice and any venturing networks and spring cloud functions and spring cloud data are really coming into play, is acknowledgement that yes we are not in a world where we can store all of the data all the time and figure out what to do with it after the fact. We need timely, and timely is within milliseconds, if not seconds. Action taken on an event or data even coming through. So why don't we modernize around, ya know, that type of data structure and data event and data horizon. So that's one of the trends we see. The second is that there is no one database to rule them all anymore. I can't get away with having oracle and that's my be all, end all. I now have my ESQL and SQL and Mongo and Cassandra and Redis and any other number of databases that are form, fit and function specific for a utility and they're perfect for that. I see graph databases, I see key value stores, I see distributed data warehouse. And so my options as a developer, as a user is really expanding, which means the total types of data components that I can use are also expanding exponentially. And that gives me a lot more flexibility on the types of products that I can build and the services that I can ultimately deliver. >> And that highlights micro services trend, because you have now a multitude of databases, it's not the one database rules them all. They'll be literally thousands of database on censors, so micro service has become the key element to connect all these systems. >> All of it together. And micro services really a higher level of abstraction. So we started with virtual machines and then we went to containers and then we went to functions and micro services. It's on an upward trend necessarily as it is an expansion. Into different ways of being able to do work. So some of my work products are going to be very, very small. They can afford to be ephemeral, but there may be many of them. How do I manage a cluster of millions of these potential work loads? Backing off I can have an ephemeral applications that run inside of containers or I can have ridged fixed applications that have to run inside a virtual machines. I'm going to have all of them. What I need is a platform that delivers all of this for me without me having to figure out how to hand wire these bits and pieces from various different either proprietary or open source kits just to make it work. I'm going to need a 60 to 100 or 200 person team just to maintain this very bespoke thing that I have developed. I'll just pull it off the shelf 'cause this is a solved problem. Right, Pivotal has already solved this problem. Other companies have already solved this problem. Let me start there and so now I'm here. I don't have to worry about all this left over plumbing. Now I can actually build on top of my business. The analogy I'd use is you don't bring furniture with you every time you check into a hotel. And we're telling customers every time you want to move to a different city just for business meeting or for work trip we're going to build you a house and you need to furnish it. Well, that's ridiculous. I'm going to check into a hotel and my expectation is I can check out of any other room and they'll all be the same, it doesn't really matter what floor I'm on, what room I'm in. But they'll have the same facilities, the same bed, the same, ya know, restroom facilities. That's what I want. That's what containers are. Eventually all the services surrounding that hotel room experience will be micro services. >> And we're the work load, the people. >> And we are the work load and we're the most important thing, we are the application, you're right. >> I love that. That's probably best analogy I've heard of containers. Nima, thanks so much for stopping by theCUBE, joining John and me today. And talking to us about what's going on with Pivotal and how you guys are really helping as part of Dell business dramatically transform. >> Been my pleasure. Thank you both. >> Thank you. >> Thank you. Thank you for watching theCUBE. I'm Lisa Martin with John Furrier. We are in Las Vegas at Boomi World '18. Stick around, John and I will be right back with our next guest. (light techno music)

>> Live from Las Vegas, it's theCUBE! Covering VMworld 2018, brought to you by VMware and its ecosystem partners. >> Welcome back everyone. It's the live CUBE coverage here in Las Vegas for VMworld 2018, three days of wall-to-wall coverage. We got two sets. I'm John Furrier, my co-host Stu Miniman. Our next guest is Adam Rasner, who is Vice-President of Technology Operation of AutoNation. Welcome to theCUBE. Thanks for joining us. >> Yeah thanks for having me. >> So you guys are a customer of all this virtualization stuff. What's going on in your company? Tell us what's happening at AutoNation. What are you guys at now with IT operations? Where you guys going? How you guys building into the Cloud? What's the strategy? >> Sure, so AutoNation is exploding. We have 280 new car dealerships. We have 80 collision centers. We just launched our own precision parts line. We're also looking at other technologies to automate the car buying experience. So we want to make like an Amazon-like car buying experience online, so that requires a lot new technology and digitalization. >> Yeah, talk a little bit about that. 'Cause I know, I've looked at cars in the last couple of years and now you know, I do so much of it online. I feel like I could do the whole experience from my phone if I wanted. So how much are you a technology company? And how much of that's cloud? And what are those dynamics that you've been going through the last couple of years? >> Yeah I think the millennials this day, they're willing to go online and do the whole car buying experience end-to-end, from the buying of the car to the financing of the car all online. And we can roll a flat-bed up to their house, and deliver a car, and they sign on an iPad, and they're good to go. And I think that's where things are going. So to do all that requires a lot of technology on the back-end. So we have a lot of on-prem infrastructure. I'd say we're still 90% on-prem, 10% in an Azure, AWS infrastructure. But that's going to change in time as a lot of these new applications are written. >> As you guys are doing the digital transformation, and it sounds like there's a lot of action going on, new things happening, you're in the app business. You got to build apps for user experience. So you've got to make the infrastructure work for you, and make it be failover, fall-tolerant, all that good stuff, recovery, how do you look at that? How do you run at the speed you need to run at? What are some of they key things you guys have to do to keep on that treadmill, but yet not drop the ball in delivering apps to the users that drive the business? >> I think there's a few things. I think one is, we have to be able to keep the lights on with our existing infrastructure, our existing apps while we build these next generation of applications. We have to be able to scale up as needed and scale down, be able to support some of the new mobile platforms that we're going to be working on. So there's a lot of work going on and DR is a big part of this too. >> Yeah, I'm glad you brought that up. Because data is at the core here. So, can you tell us that role of data, and then you say data protection. How is that changing, what was it like before you went through this transformation? Then we'll of course get into what you're using. >> Sure, so we actually, were using an old Microsoft data protection manager product and just didn't scale the way we needed to, we were having some performance issues. And so, data protection, while not very sexy, it's something you have to do. It's table stakes in IT. It doesn't innovate, it doesn't make me sell more cars, it doesn't help the business sell more cars, but it's something we have to do. So we looked out there at what I call the legacy players and also the nextgen players and went through a full proof of concept with several of them. >> All right, and what were you looking for? What was kind of the key objective you said? Data protection doesn't make you money, or didn't make you money. We've talked to some customers, that's like, wait, might do some cool snapshotting, I can leverage that data, I can do some more things with my developers, and everything. So what was the goal of this transformation and then what was the criteria that you went through to make a decision? >> Yeah, so the data protection was the initial piece and we just needed a rock solid backup and recovery solution. And we started off with just a simple, hey, we wanted an integrated hardware software solution, we wanted something that could scale infinitely, we wanted a predictive cost model. And so a lot of those older legacy players don't play well in that space, they're expensive to support, eventually you hit a wall on hardware limitations and you have to use forklift upgrades. So we wanted something that was a little bit more nimble and then down the road, as we got into it, once the backup and recovery piece was kind of under control we started using our new solution for other things and secondary storage which was an added bonus. >> So you haven't mentioned, what is the solution that you chose and what were the key things that led to that? >> Yeah, so after going through several POCs with you know, NetBackup, Rubric and Cohesity, we ultimately chose Cohesity for performance, cost, ease of implementation, ease of the user interface, ease of management. >> And what was the comparison, on the floor here you see Rubric and Cohesity next in the huge booths. What's the difference between those two? >> Yeah, so we actually put them side by side in our data center, full blown POCs, and there was some performance differences, there were some technical challenges that we had with some of the other products. And ultimately the team, our engineering team felt most comfortable with Cohesity after spending six or eight months in a really in-depth POC. >> Big bake-off. I love the bake-offs. It's the only way to have the answer like that. So when you look at the solutions, are you guys mostly interested in the software side of the business that they had? What was they key piece of it? >> I think we're interested in the whole thing. I had been at other places where we had done the NetBackup and data domain story and you know, you're having a problem at three o'clock in the morning and you got the finger pointing, is it a software issue, is it a hardware issue? We wanted the one throat to choke kind of solution, and so, you know, that was a requirement right off the bat. Whatever we chose was going to be an integrated hardware software platform. >> Adam, walk us through from the deployment to the day two action. How did it go? What surprised you? What, you know, thrilled you? You know, what challenges did you have? >> Yeah, we've been a customer for- I think we were very early customer, probably almost about two years now. So, there's a lot we didn't know. There was a lot of things in the product that actually weren't fully mature when we first started the POC. And so we went through a full, a full blown bake-off, and one of the things we noticed it was much easier to implement, we didn't require any professional services to get it up and running and the technical support we were super impressed with. So I think, you know, the team, after going through the motions, really felt like this was the product for us. And again, really mainly around backup and recovery, but ultimately decided that we were going to use it for other things too. >> Adam, I was walking through the hallways yesterday, Stu and I were both checking out the booths. And I hear a lot of conversations and it comes up around the Cohesity, Rubric, all these different cloud solutions. Some are rinsed and repeat old models that just have, you know, not mostly those guys are, but the customers are concerned about I don't want the old way, I want the new way, I want to be cloud native, I want to work with cloud, One choke to throw, I need software, I need to have agility, and I need to have auto, you know, healing, all this kind of stuff. How do you sort through that? I know you've been through the POC but your peers that are out here at VMworld, they're squinting through the noise going okay, I got to really dig in here. What's your advice to those guys and gals? >> I think it's really challenging for the people that are, you know, neck deep in some of these other legacy products because it's a little bit hard to move. You know, it's costly, it's expensive, and it's a significant effort. I was in a rare position where I was able to start net new, and so that made it a little bit easier. But I think you start with a slow migration, start setting up your new infrastructure on a nextgen platform and then slowly migrate off. These next, these legacy players are very expensive, and they don't scale very well. That's probably one of our biggest challenges. >> One of the things you said, you started with a couple of use cases but you're now doing a bunch more. Talk about that, what more, what are the new things you're doing and what's the road map look forward at AutoNation? >> Sure. So we had a, a lot of apps, that we're probably not needing. Tier one, NetApp, all SSD, high performance SAN. I call it my Cadillac of storage, you know. It's our highest performance applications and we were having some apps that the hardware was starting to, you know, just go bad. And so the only place I could put it was either on my NetApp, or I didn't have any place else. So the story changed over time. Cohesity became not only our backup and recovery data protection appliance, we started landing some of our tier two storage on Cohesity. So moving things that we would normally put on NetApp, putting it on Cohesity for 40 percent of the cost and it's a win-win. >> All right, so, Adam, I couldn't help noticing you've got the Drive Pink pin on. >> Yes. >> So, maybe tell our audience a little bit about the, you know, AutoNation Drive Pink initiative and you know, do you have relationships with the suppliers here? Pat Gelsinger this morning talked about you know, we need to be as a technology community more doing good. It's foundational to what we're doing. >> Autonation, it's one of our core charities is cancer awareness. I think we've donated almost 30 million dollars. Every car that you buy, we try to put the Drive Pink license plate. And I think not only for business, I think in IT we also have to have a lens to some of these charities and some of these things that need our help. >> Issue driven businesses are doing well now, people expect that. Not just for profit, but the people involved. >> Yeah. >> Anyone can work anywhere these days, talent, it's also good. I mean, it's one of those things. >> Yeah, yeah, absolutely. >> All right, so, takeaway from this show, so far, your impression as a practitioner in the IT footprint space, looking at a cloud on the horizon, we just had Andy Bechtolsheim just on, been part of the early days. Cloud's coming fast, networking's got to get better, you got to, you know, seeing what solutions, integrating well together. How do you make sense of all this content coming out of VMworld? >> Yeah, I think what I get out of this and kind of AWS, all of these conferences, is that everything we buy has to be extendable to the cloud. You know, we still have a lot of on-premise infrastructure but everything we implement has to be cloudable, it has to be able to be used in our future use cases. >> I would love, we're talking a lot here in the keynote this morning it's like, right, this move, we know it's going to take time and Amazon's doing some things, VMware's doing some things, how's the industry doing, how do you see the progression, what would you like to see them do more better if we come back in a year, if I kind of give you that magic wand? >> Yeah. You know, I always leave a lot of these conferences and I feel like I'm behind the eight ball, in our cloud migration, but, companies like us that have a lot of legacy apps, they're slow to move. And so, I leave the conference, I feel like I'm behind the eight ball, but I get back and I talk to my peers and many of them are in the same situation I am. They're still maturing, but I think, yes, I think the net new generation apps that we're going to build are going to be in the cloud because the capabilities to autoscale and so I think that anything we buy, anything we implement we have to have a lens to that going forward. >> Well, thanks for coming on theCUBE, we really appreciate, sounds like you're happy with Cohesity? >> They've done a great job, we're really happy customers. >> How long was that bake-off by the way, that you ran that? >> We did it about six months. >> That's pretty good and long. >> Yeah, we actually had some, again we were very early to the game so there were features in the product that we needed that they didn't have yet and our agreement was we'll proceed after you can meet these requirements and they did. >> Yeah. And Pat Gelsinger and Andy Jassy on the stage, one of the things Andy Jassy, who's been on theCUBE talks about all the time is listening to customers. Sounds like they're listening to you guys. >> Absolutely, absolutely. You have to, it's such a competitive environment now. You know, if you can't meet the customer's minimal requirements, there's somebody else that can. >> You got to be cloud compatible. AutoNation breaking it down here, here at Vmworld bringing the practitioner perspective, the customer perspective, all of these suppliers try to bring cloud and on-premises together. It's theCUBE bringing you all the action here at Vmworld 2018. I'm John Furrier. Stu Miniman. Stay with us for more coverage after this short break.

>> Narrator: From Nice, France, it's theCUBE covering .NEXT Conference 2017 Europe, brought to you by Nutanix. (electronic music) >> Welcome back. I'm Stu Miniman, and this is SiliconANGLE Media's Production of theCUBE live broadcast from Nutanix .NEXT in Nice, France. To help me wrap up for today's coverage, I'm happy to have Jens Soldner, who is a consultant, does media, writes for some organizations, someone I've gotten to know at some industry events over the last couple of years. So Jens, thanks so much for joining me. >> Thanks to you. Thanks for inviting me, and happy to be here at Nutanix .NEXT Conference. Awesome event and I think the vendor has a bright future in front of him. >> You know, we're a year after Nutanix IPOed. The attendance at this show doubled absolutely. You know, every Nutanix show, and I've been at all five of them, you know, customers are usually, they're enthusiastic. It's self-selecting, right? You go to a VMworld, you come to a Nutanix show, Veritas, some of these, there's VEEMs show, usually the customers, it's a good measure of how the company's doing and barometer is customers are happy, they like what they're doing. One thing I like about Nutanix customers is they aren't just oh, we love everything and we believe everything that Nutanix says. It was like hey, calm, I talked to a couple customers that use it and they're like it's great. But everybody else is like yeah, I'm waiting to get my hands on it and really beat it up and we'll see if it does what it is because Nutanix has proven themselves and needs to continually prove themselves time and again. Really I think something that reminds me of the Cloud Era because if I'm buying from public cloud and I'm buying from a consumption basis, if I don't like something, I'll move, I'll stop paying. What's been catching your ear and eye at the show so far, what have you liked, what are you still questioning and want to learn more about? >> I think there's a lot of good things coming actually, like the new version 5.5 which will be out end of this year also we hear, so that's like brings some good incremental innovation, nothing overly massive, but some good stuff in there. But I of course like the future looking stuff, calm, sigh and so on and so that looks pretty promising, but as you said it's not available right now and we want to get my hands dirty, that's the thing. >> What you like is if you talk to the customers that are deeply involved, they've probably been beta testing a lot of the stuff in 5.5, some of the features get out in the community edition. In the keynote, they talked about MVME was one that had been tested, anything in particular that you've been hearing, customers that chomping at the bit, interested, 5.5 oh great, finally we have that? >> I think the generic compute platform is a good thing so in order to enable new use cases like SAP stuff, HANA, maybe you need for Oracle licensing, this kind of stuff for anything. >> Are you talking about I think they call it AC2? >> Yeah. >> Which that's not in 5.5 I don't believe, no? I think that's a future one. >> Oh then I got the wrong impression, okay. >> To be honest, that's one of the critiques sometimes, if you go look at this and you get the keynote and it's a deluge of so much stuff, you need the cheat sheet so I look through, Nutanix, they did a press release, there's like four blog posts, it takes a little while for those of us that look at this to sort through and be like oh wait, which of this, as you said, 5.5 oh, does that mean I can do it today well it's coming real soon and if you're a beta person, you can do it, as opposed to the object storage and AC2 type pieces, if I remember right and I'm sure Nutanix will watch and tell us if we got it wrong though, was a coming future, we'll let you know when we have a date. Little bit different now as a public company too, a lot of these, they don't pin down on certain quarters or dates because then that impacts financial reporting. Talk to us about you were obviously at the keynote, what else have you been doing, what sessions have you been going to or what's been happening? >> We had a lot of background talks with the Nutanix executive leadership like Sumir Pati, right before our talk here and we talked actually in depth about questions that other journalists and I had about calm, when it will be available, how it will be priced, what can you do, can you move apps from one cloud to the other, not in the near future, maybe in the more distant future. It looks pretty promising and for a cloud management person that is one of my main jobs out there in the normal life, then it looks pretty good actually, but as you said, getting the hands dirty is an essential part and maybe that's coming a little bit too short here to really see what's happening and not just announcements and announcements. >> Absolutely and if you were to place bets on what are the important pieces in the future, calm and Zi, absolutely something Nutanix have been talking a lot, super important that they get it right. You've been tracking calm since the acquisition, any nuances, what do they need to do, what's going to be ready, what do they need to have in the future to really make that work? >> I think actually going to get it right. I think in the grand scheme of things, like delivering it two weeks early or later in five years race with the competition is not making such a huge difference, so rather than delivering a immature and unready product, how do you say, like filing the edges off and making it smooth should take some time, however me as a technical person, I like to get my hands on the stuff and really see it, so that's the downside. >> Getting your hands dirty is something that a lot of customers here like to do, do you get to play with the community edition and the like? >> Not yet, but I have a Nutanix folder trust waiting in our data center, ready for installation and we want to compare it like how it runs with Vsphere and how it runs with AHV so that AHV I think is unlikely work load actually. >> We've been hearing the last couple of shows, AHV has really been front and center, it's an interesting mix for them to balance because even if about a third of customers of Nutanix are running AHV, that means two thirds of customers still are running one of the other hyper visors out there. I put the question to Nutanix and I said, what is victory, what is the ultimate goal and it's not 100% AHV, they're not looking to become a hyper visor company, they want to be a platform, work in the multicloud world, so when you talk to companies, how does that discussion go? Is AHV a central discussion point or is it some of the features that come along with it that help? >> I would say it's rather on the sidelines, I think it makes sense from an economic point of view, not having to pay additional licenses obviously and getting the impression, getting the right kind of experience with the product and even Nutanix, I think they say if the customer wants this and this and this extra, go get Vsphere. We are offering you a standard path, like with 80% of the features that you really really need and those 20 super esoteric stuff, like fault tolerance that nobody is really using, Vsphere they're not bringing it to AHV, they're keeping their product clean, simple, easy. >> You said cloud management, kind of a main focus area of you, what does Nutanix have to do to be a strong player in that market over the next two years? >> I think they are actually on a good vein already with the calm stuff, the thing is we need to see it, if it can compete with the other players out there, Vmware and Red Hat and you name it basically. Then to see if it gets accepted in the market, how the marketplace, the calm marketplace takes off and so on, I think the adoption, if it gains significant adoption, if there is traction in the market, in the blogosphere and so on, I think that's crucial. >> You mentioned Vmware and Red Hat, big companies, gigantic ecosystems. We all know the Vmware ecosystem and Red Hat open source, everybody's there, been at Red Hat summit for many years now. Any others that you'd say who they should be matching up as customers will be? >> I think computer associates has a good valid offering, but we personally see in the German market, most of the time, we realize automation, we've written three books on it, my brothers and I so that's maybe we are opinionated and biased here. In this case, but VMware's doing a good job in this cloud management space and of course they have a tight integration with the other products, like L6 that they have and I think Nutanix is very eager to catch up in these areas where they have gaps. >> One of the underlying, simmering conversations at a Nutanix event is that kind of Vmware, Nutanix relationship and we talked about still, lots of Nutanix deployments are using Vmware, didn't feel that they were bashing Vmware at this event, but what are you seeing when you talk to customers and you use a lot of Vmware, how's that relationship? Are there any challenges there or things that are concerning? >> At the end of the day, it's the customer's decision what they are going for, I think most of the customers might not go all in Nutanix, but only place it in certain use cases and so on. Of course Vmware is not happy, why should they be and they are positioning their VSAM product which is running quite well pretty aggressively but Nutanix has a different storyline, I think it's not only about the IOPS and it's about the simplicity of the whole thing and offering the customer a real, simple path to manage it in a cloud enabled fashion and that's where they're really doing a good job. However, Vmware, they can cover everything and they can figure so many little things and that makes the whole thing huge and complicated and of course you can, any use case can somehow be tailored to but also if you have a vendor who has a real good storyline of simplicity like Nutanix, they have a good chance here. >> Yeah, there was a lot of discussion, it was interesting, we were talking to Nutanix, they were talking about they want to get one click, it's about simplicity, then there's all of this learning from what other customers have done, I've got artificial intelligence starting to help in there. How do you see that trend going as to how, if I'm an administrator, is it reducing the number of clicks or am I going to be able to let go of the reins some and allow some other tooling and knowledge bases really drive some of that decision making? >> I think it's pretty helpful to have these expert knowledge bases built in, there's also a startup that does a similar thing in the Vmware space, Runecast, great guys and so on, so that's good but it's also challenging I would say for partners that they really need to see that with Nutanix of course we are going to sell as a partner, you are going to sell less wrecking and stacking of servers, you as a partner, you really need to refocus, learn the orchestration, learn the automation, get into container stuff in order to offer to your customers a valuable offering, a value proposition. Everybody needs to learn and I think Nutanix makes your life easier in these mundane, day to day activities, so that's I would say a good benefit of getting such an environment. >> Again, we're about at the halfway mark of the event, any other key takeaways, customer conversations that you'd want to share? >> I talked to a couple of partners, friends of mine from the Vmware instructor community and they say we are going all in Nutanix, so that was pretty impressive here and it's also what I heard not only from those who are actually doing it but of course from the Nutanix management, easy to understand why they say this, so I think there is a huge traction, some partners seem to have got the message and seem to say yeah, we are going all in. That was one of the things and of course I'll go a little bit more technical tomorrow so today was really packed with the official schedule, tomorrow is a little bit more free, so I'll have a couple more conversations with actual customers from a large Swiss bank, where we'll be doing the Vmware's implementation soon but they are also into Nutanix, so they are both a Vmware and a Nutanix partner so we'll meet up later on and yeah, that's pretty much the schedule. >> All right I never do this, but you got any questions for me for the wrap? >> What I would like to know is what's your take on the microsegmentation part of Nutanix, can it compete with the other offerings and I have not really looked at it so far. It looked pretty impressive to me in the keynote. >> Look, I'll say two pieces, one is, it's one of the top items that I've heard from users that they are super excited about. There was a bank I just interviewed earlier today, I think financial services and service providers were really excited for the microsegmentation. That being said, I've also talked to a bunch of the partner community and of course it's the typical, well how much is Nutanix doing versus what the department, oh we've had this and ours is much more future rich and the like, so it's good to see Nutanix moving down this line, they need to balance how much they'll do versus what some of their partners that are especially deeper in the networking space can do there. It's definitely one that you talk to customers that are getting into it, digging into it, but yeah it's a good one and definitely when you talk about those features coming out, one that customers have been asking for a bit. Like Vmware has done in the past, there's probably a lot of customers that what's built in is going to be good enough but then if I really need the Cadillac of it, I might need to pull in some best of breed partner to be able to compete it, so cool. >> That could also happen this year for us, if you look at the partner ecosystem, it's also pretty impressive and most of their named guys are here. >> All right well Jens Soldner, a pleasure catching up with you, thanks for helping us here on the Cube. We're wrapping up day one of two days of live coverage on the Cube. I'm Stu Miniman, you're watching the Cube. (dramatic music) (acoustic music)