>>Okay, we're back. I'm Dave Valante with The Cube and you're watching Evolving Influx DB into the smart data platform made possible by influx data. Anna East Otis Georgio is here. She's a developer advocate for influx data and we're gonna dig into the rationale and value contribution behind several open source technologies that Influx DB is leveraging to increase the granularity of time series analysis analysis and bring the world of data into realtime analytics. Anna is welcome to the program. Thanks for coming on. >>Hi, thank you so much. It's a pleasure to be here. >>Oh, you're very welcome. Okay, so IO X is being touted as this next gen open source core for Influx db. And my understanding is that it leverages in memory, of course for speed. It's a kilo store, so it gives you compression efficiency, it's gonna give you faster query speeds, it gonna use store files and object storages. So you got very cost effective approach. Are these the salient points on the platform? I know there are probably dozens of other features, but what are the high level value points that people should understand? >>Sure, that's a great question. So some of the main requirements that IOCs is trying to achieve and some of the most impressive ones to me, the first one is that it aims to have no limits on cardinality and also allow you to write any kind of event data that you want, whether that's lift tag or a field. It also wants to deliver the best in class performance on analytics queries. In addition to our already well served metrics queries, we also wanna have operator control over memory usage. So you should be able to define how much memory is used for buffering caching and query processing. Some other really important parts is the ability to have bulk data export and import, super useful. Also, broader ecosystem compatibility where possible we aim to use and embrace emerging standards in the data analytics ecosystem and have compatibility with things like sql, Python, and maybe even pandas in the future. >>Okay, so a lot there. Now we talked to Brian about how you're using Rust and and which is not a new programming language and of course we had some drama around Russ during the pandemic with the Mozilla layoffs, but the formation of the Russ Foundation really addressed any of those concerns. You got big guns like Amazon and Google and Microsoft throwing their collective weights behind it. It's really, adoption is really starting to get steep on the S-curve. So lots of platforms, lots of adoption with rust, but why rust as an alternative to say c plus plus for example? >>Sure, that's a great question. So Rust was chosen because of his exceptional performance and rebi reliability. So while rust is synt tactically similar to c c plus plus and it has similar performance, it also compiles to a native code like c plus plus. But unlike c plus plus, it also has much better memory safety. So memory safety is protection against bugs or security vulnerabilities that lead to excessive memory usage or memory leaks. And rust achieves this memory safety due to its like innovative type system. Additionally, it doesn't allow for dangling pointers and dangling pointers are the main classes of errors that lead to exploitable security vulnerabilities in languages like c plus plus. So Russ like helps meet that requirement of having no limits on card for example, because it's, we're also using the Russ implementation of Apache Arrow and this control over memory and also Russ, Russ Russ's packaging system called crates IO offers everything that you need out of the box to have features like AY and a weight to fixed race conditions to protect against buffering overflows and to ensure thread safe ay caching structures as well. So essentially it's just like has all the control, all the fine grain control, you need to take advantage of memory and all your resources as well as possible so that you can handle those really, really high ity use cases. >>Yeah, and the more I learned about the the new engine and the, and the platform IOCs et cetera, you know, you, you see things like, you know, the old days not even to even today you do a lot of garbage collection in these, in these systems and there's an inverse, you know, impact relative to performance. So it looks like you're really, you know, the community is modernizing the platform, but I wanna talk about Apache Arrow for a moment. It's designed to address the constraints that are associated with analyzing large data sets. We, we know that, but please explain why, what, what is Arrow and and what does it bring to Influx db? >>Sure, yeah. So Arrow is a, a framework for defining in memory calmer data and so much of the efficiency and performance of IOCs comes from taking advantage of calmer data structures. And I will, if you don't mind, take a moment to kind of illustrate why calmer data structures are so valuable. Let's pretend that we are gathering field data about the temperature in our room and also maybe the temperature of our stove. And in our table we have those two temperature values as well as maybe a measurement value, timestamp value, maybe some other tag values that describe what room and what house, et cetera we're getting this data from. And so you can picture this table where we have like two rows with the two temperature values for both our room and the stove. Well usually our room temperature is regulated so those values don't change very often. >>So when you have calm oriented st calm oriented storage, essentially you take each row, each column and group it together. And so if that's the case and you're just taking temperature values from the room and a lot of those temperature values are the same, then you'll, you might be able to imagine how equal values will then neighbor each other and when they neighbor each other in the storage format. This provides a really perfect opportunity for cheap compression. And then this cheap compression enables high cardinality use cases. It also enables for faster scan rates. So if you wanna define like the min and max value of the temperature in the room across a thousand different points, you only have to get those a thousand different points in order to answer that question and you have those immediately available to you. But let's contrast this with a row oriented storage solution instead so that we can understand better the benefits of calmer oriented storage. >>So if you had a row oriented storage, you'd first have to look at every field like the temperature in, in the room and the temperature of the stove. You'd have to go across every tag value that maybe describes where the room is located or what model the stove is. And every timestamp you'd then have to pluck out that one temperature value that you want at that one times stamp and do that for every single row. So you're scanning across a ton more data and that's why row oriented doesn't provide the same efficiency as calmer and Apache Arrow is in memory calmer data, calmer data fit framework. So that's where a lot of the advantages come >>From. Okay. So you've basically described like a traditional database, a row approach, but I've seen like a lot of traditional databases say, okay, now we've got, we can handle colo format versus what you're talking about is really, you know, kind of native it, is it not as effective as the, is the form not as effective because it's largely a, a bolt on? Can you, can you like elucidate on that front? >>Yeah, it's, it's not as effective because you have more expensive compression and because you can't scan across the values as quickly. And so those are, that's pretty much the main reasons why, why RO row oriented storage isn't as efficient as calm, calmer oriented storage. >>Yeah. Got it. So let's talk about Arrow data fusion. What is data fusion? I know it's written in rust, but what does it bring to to the table here? >>Sure. So it's an extensible query execution framework and it uses Arrow as its in memory format. So the way that it helps influx DB IOx is that okay, it's great if you can write unlimited amount of cardinality into influx cbis, but if you don't have a query engine that can successfully query that data, then I don't know how much value it is for you. So data fusion helps enable the, the query process and transformation of that data. It also has a PANDAS API so that you could take advantage of PDA's data frames as well and all of the machine learning tools associated with pandas. >>Okay. You're also leveraging par K in the platform course. We heard a lot about Par K in the middle of the last decade cuz as a storage format to improve on Hadoop column stores. What are you doing with Par K and why is it important? >>Sure. So Par K is the calm oriented durable file format. So it's important because it'll enable bulk import and bulk export. It has compatibility with Python and pandas so it supports a broader ecosystem. Parque files also take very little disc disc space and they're faster to scan because again they're column oriented in particular, I think PAR K files are like 16 times cheaper than CSV files, just as kind of a point of reference. And so that's essentially a lot of the, the benefits of par k. >>Got it. Very popular. So and these, what exactly is influx data focusing on as a committer to these projects? What is your focus? What's the value that you're bringing to the community? >>Sure. So Influx DB first has contributed a lot of different, different things to the Apache ecosystem. For example, they contribute an implementation of Apache Arrow and go and that will support clearing with flux. Also, there has been a quite a few contributions to data fusion for things like memory optimization and supportive additional SQL features like support for timestamp, arithmetic and support for exist clauses and support for memory control. So yeah, Influx has contributed a a lot to the Apache ecosystem and continues to do so. And I think kind of the idea here is that if you can improve these upstream projects and then the long term strategy here is that the more you contribute and build those up, then the more you will perpetuate that cycle of improvement and the more we will invest in our own project as well. So it's just that kind of symbiotic relationship and appreciation of the open source community. >>Yeah. Got it. You got that virtuous cycle going, the people call it the flywheel. Give us your last thoughts and kind of summarize, you know, where what, what the big takeaways are from your perspective. >>So I think the big takeaway is that influx data is doing a lot of really exciting things with Influx DB IOCs and I really encourage if you are interested in learning more about the technologies that Influx is leveraging to produce IOCs, the challenges associated with it and all of the hard work questions and I just wanna learn more, then I would encourage you to go to the monthly tech talks and community office hours and they are on every second Wednesday of the month at 8:30 AM Pacific time. There's also a community forums and a community Slack channel. Look for the influx D DB underscore IAC channel specifically to learn more about how to join those office hours and those monthly tech tech talks as well as ask any questions they have about IOCs, what to expect and what you'd like to learn more about. I as a developer advocate, I wanna answer your questions. So if there's a particular technology or stack that you wanna dive deeper into and want more explanation about how influx TB leverages it to build IOCs, I will be really excited to produce content on that topic for you. >>Yeah, that's awesome. You guys have a really rich community, collaborate with your peers, solve problems, and you guys super responsive, so really appreciate that. All right, thank you so much and East for explaining all this open source stuff to the audience and why it's important to the future of data. >>Thank you. I really appreciate it. >>All right, you're very welcome. Okay, stay right there and in a moment I'll be back with Tim Yokum. He's the director of engineering for Influx Data and we're gonna talk about how you update a SaaS engine while the plane is flying at 30,000 feet. You don't wanna miss this.

>>This past May, The Cube in collaboration with Influx data shared with you the latest innovations in Time series databases. We talked at length about why a purpose built time series database for many use cases, was a superior alternative to general purpose databases trying to do the same thing. Now, you may, you may remember the time series data is any data that's stamped in time, and if it's stamped, it can be analyzed historically. And when we introduced the concept to the community, we talked about how in theory, those time slices could be taken, you know, every hour, every minute, every second, you know, down to the millisecond and how the world was moving toward realtime or near realtime data analysis to support physical infrastructure like sensors and other devices and IOT equipment. A time series databases have had to evolve to efficiently support realtime data in emerging use cases in iot T and other use cases. >>And to do that, new architectural innovations have to be brought to bear. As is often the case, open source software is the linchpin to those innovations. Hello and welcome to Evolving Influx DB into the smart Data platform, made possible by influx data and produced by the Cube. My name is Dave Valante and I'll be your host today. Now, in this program, we're going to dig pretty deep into what's happening with Time series data generally, and specifically how Influx DB is evolving to support new workloads and demands and data, and specifically around data analytics use cases in real time. Now, first we're gonna hear from Brian Gilmore, who is the director of IOT and emerging technologies at Influx Data. And we're gonna talk about the continued evolution of Influx DB and the new capabilities enabled by open source generally and specific tools. And in this program, you're gonna hear a lot about things like Rust, implementation of Apache Arrow, the use of par k and tooling such as data fusion, which powering a new engine for Influx db. >>Now, these innovations, they evolve the idea of time series analysis by dramatically increasing the granularity of time series data by compressing the historical time slices, if you will, from, for example, minutes down to milliseconds. And at the same time, enabling real time analytics with an architecture that can process data much faster and much more efficiently. Now, after Brian, we're gonna hear from Anna East Dos Georgio, who is a developer advocate at In Flux Data. And we're gonna get into the why of these open source capabilities and how they contribute to the evolution of the Influx DB platform. And then we're gonna close the program with Tim Yokum, he's the director of engineering at Influx Data, and he's gonna explain how the Influx DB community actually evolved the data engine in mid-flight and which decisions went into the innovations that are coming to the market. Thank you for being here. We hope you enjoy the program. Let's get started. Okay, we're kicking things off with Brian Gilmore. He's the director of i t and emerging Technology at Influx State of Bryan. Welcome to the program. Thanks for coming on. >>Thanks Dave. Great to be here. I appreciate the time. >>Hey, explain why Influx db, you know, needs a new engine. Was there something wrong with the current engine? What's going on there? >>No, no, not at all. I mean, I think it's, for us, it's been about staying ahead of the market. I think, you know, if we think about what our customers are coming to us sort of with now, you know, related to requests like sql, you know, query support, things like that, we have to figure out a way to, to execute those for them in a way that will scale long term. And then we also, we wanna make sure we're innovating, we're sort of staying ahead of the market as well and sort of anticipating those future needs. So, you know, this is really a, a transparent change for our customers. I mean, I think we'll be adding new capabilities over time that sort of leverage this new engine, but you know, initially the customers who are using us are gonna see just great improvements in performance, you know, especially those that are working at the top end of the, of the workload scale, you know, the massive data volumes and things like that. >>Yeah, and we're gonna get into that today and the architecture and the like, but what was the catalyst for the enhancements? I mean, when and how did this all come about? >>Well, I mean, like three years ago we were primarily on premises, right? I mean, I think we had our open source, we had an enterprise product, you know, and, and sort of shifting that technology, especially the open source code base to a service basis where we were hosting it through, you know, multiple cloud providers. That was, that was, that was a long journey I guess, you know, phase one was, you know, we wanted to host enterprise for our customers, so we sort of created a service that we just managed and ran our enterprise product for them. You know, phase two of this cloud effort was to, to optimize for like multi-tenant, multi-cloud, be able to, to host it in a truly like sass manner where we could use, you know, some type of customer activity or consumption as the, the pricing vector, you know, And, and that was sort of the birth of the, of the real first influx DB cloud, you know, which has been really successful. >>We've seen, I think, like 60,000 people sign up and we've got tons and tons of, of both enterprises as well as like new companies, developers, and of course a lot of home hobbyists and enthusiasts who are using out on a, on a daily basis, you know, and having that sort of big pool of, of very diverse and very customers to chat with as they're using the product, as they're giving us feedback, et cetera, has has, you know, pointed us in a really good direction in terms of making sure we're continuously improving that and then also making these big leaps as we're doing with this, with this new engine. >>Right. So you've called it a transparent change for customers, so I'm presuming it's non-disruptive, but I really wanna understand how much of a pivot this is and what, what does it take to make that shift from, you know, time series, you know, specialist to real time analytics and being able to support both? >>Yeah, I mean, it's much more of an evolution, I think, than like a shift or a pivot. You know, time series data is always gonna be fundamental and sort of the basis of the solutions that we offer our customers, and then also the ones that they're building on the sort of raw APIs of our platform themselves. You know, the time series market is one that we've worked diligently to lead. I mean, I think when it comes to like metrics, especially like sensor data and app and infrastructure metrics, if we're being honest though, I think our, our user base is well aware that the way we were architected was much more towards those sort of like backwards looking historical type analytics, which are key for troubleshooting and making sure you don't, you know, run into the same problem twice. But, you know, we had to ask ourselves like, what can we do to like better handle those queries from a performance and a, and a, you know, a time to response on the queries, and can we get that to the point where the results sets are coming back so quickly from the time of query that we can like limit that window down to minutes and then seconds. >>And now with this new engine, we're really starting to talk about a query window that could be like returning results in, in, you know, milliseconds of time since it hit the, the, the ingest queue. And that's, that's really getting to the point where as your data is available, you can use it and you can query it, you can visualize it, and you can do all those sort of magical things with it, you know? And I think getting all of that to a place where we're saying like, yes to the customer on, you know, all of the, the real time queries, the, the multiple language query support, but, you know, it was hard, but we're now at a spot where we can start introducing that to, you know, a a limited number of customers, strategic customers and strategic availability zones to start. But you know, everybody over time. >>So you're basically going from what happened to in, you can still do that obviously, but to what's happening now in the moment? >>Yeah, yeah. I mean, if you think about time, it's always sort of past, right? I mean, like in the moment right now, whether you're talking about like a millisecond ago or a minute ago, you know, that's, that's pretty much right now, I think for most people, especially in these use cases where you have other sort of components of latency induced by the, by the underlying data collection, the architecture, the infrastructure, the, you know, the, the devices and you know, the sort of highly distributed nature of all of this. So yeah, I mean, getting, getting a customer or a user to be able to use the data as soon as it is available is what we're after here. >>I always thought, you know, real, I always thought of real time as before you lose the customer, but now in this context, maybe it's before the machine blows up. >>Yeah, it's, it's, I mean it is operationally or operational real time is different, you know, and that's one of the things that really triggered us to know that we were, we were heading in the right direction, is just how many sort of operational customers we have. You know, everything from like aerospace and defense. We've got companies monitoring satellites, we've got tons of industrial users, users using us as a processes storing on the plant floor, you know, and, and if we can satisfy their sort of demands for like real time historical perspective, that's awesome. I think what we're gonna do here is we're gonna start to like edge into the real time that they're used to in terms of, you know, the millisecond response times that they expect of their control systems. Certainly not their, their historians and databases. >>I, is this available, these innovations to influx DB cloud customers only who can access this capability? >>Yeah. I mean, commercially and today, yes. You know, I think we want to emphasize that's a, for now our goal is to get our latest and greatest and our best to everybody over time. Of course. You know, one of the things we had to do here was like we double down on sort of our, our commitment to open source and availability. So like anybody today can take a look at the, the libraries in on our GitHub and, you know, can ex inspect it and even can try to, you know, implement or execute some of it themselves in their own infrastructure. You know, we are, we're committed to bringing our sort of latest and greatest to our cloud customers first for a couple of reasons. Number one, you know, there are big workloads and they have high expectations of us. I think number two, it also gives us the opportunity to monitor a little bit more closely how it's working, how they're using it, like how the system itself is performing. >>And so just, you know, being careful, maybe a little cautious in terms of, of, of how big we go with this right away. Just sort of both limits, you know, the risk of, of, you know, any issues that can come with new software rollouts. We haven't seen anything so far, but also it does give us the opportunity to have like meaningful conversations with a small group of users who are using the products, but once we get through that and they give us two thumbs up on it, it'll be like, open the gates and let everybody in. It's gonna be exciting time for the whole ecosystem. >>Yeah, that makes a lot of sense. And you can do some experimentation and, you know, using the cloud resources. Let's dig into some of the architectural and technical innovations that are gonna help deliver on this vision. What, what should we know there? >>Well, I mean, I think foundationally we built the, the new core on Rust. You know, this is a new very sort of popular systems language, you know, it's extremely efficient, but it's also built for speed and memory safety, which goes back to that us being able to like deliver it in a way that is, you know, something we can inspect very closely, but then also rely on the fact that it's going to behave well. And if it does find error conditions, I mean, we, we've loved working with Go and, you know, a lot of our libraries will continue to, to be sort of implemented in Go, but you know, when it came to this particular new engine, you know, that power performance and stability rust was critical. On top of that, like, we've also integrated Apache Arrow and Apache Parque for persistence. I think for anybody who's really familiar with the nuts and bolts of our backend and our TSI and our, our time series merged Trees, this is a big break from that, you know, arrow on the sort of in MI side and then Par K in the on disk side. >>It, it allows us to, to present, you know, a unified set of APIs for those really fast real time inquiries that we talked about, as well as for very large, you know, historical sort of bulk data archives in that PARQUE format, which is also cool because there's an entire ecosystem sort of popping up around Parque in terms of the machine learning community, you know, and getting that all to work, we had to glue it together with aero flight. That's sort of what we're using as our, our RPC component. You know, it handles the orchestration and the, the transportation of the Coer data. Now we're moving to like a true Coer database model for this, this version of the engine, you know, and it removes a lot of overhead for us in terms of having to manage all that serialization, the deserialization, and, you know, to that again, like blurring that line between real time and historical data. It's, you know, it's, it's highly optimized for both streaming micro batch and then batches, but true streaming as well. >>Yeah. Again, I mean, it's funny you mentioned Rust. It is, it's been around for a long time, but it's popularity is, is, you know, really starting to hit that steep part of the S-curve. And, and we're gonna dig into to more of that, but give us any, is there anything else that we should know about Bryan? Give us the last word? >>Well, I mean, I think first I'd like everybody sort of watching just to like, take a look at what we're offering in terms of early access in beta programs. I mean, if, if, if you wanna participate or if you wanna work sort of in terms of early access with the, with the new engine, please reach out to the team. I'm sure you know, there's a lot of communications going out and, you know, it'll be highly featured on our, our website, you know, but reach out to the team, believe it or not, like we have a lot more going on than just the new engine. And so there are also other programs, things we're, we're offering to customers in terms of the user interface, data collection and things like that. And, you know, if you're a customer of ours and you have a sales team, a commercial team that you work with, you can reach out to them and see what you can get access to because we can flip a lot of stuff on, especially in cloud through feature flags. >>But if there's something new that you wanna try out, we'd just love to hear from you. And then, you know, our goal would be that as we give you access to all of these new cool features that, you know, you would give us continuous feedback on these products and services, not only like what you need today, but then what you'll need tomorrow to, to sort of build the next versions of your business. Because, you know, the whole database, the ecosystem as it expands out into to, you know, this vertically oriented stack of cloud services and enterprise databases and edge databases, you know, it's gonna be what we all make it together, not just, you know, those of us who were employed by Influx db. And then finally, I would just say please, like watch in ice in Tim's sessions, Like these are two of our best and brightest. They're totally brilliant, completely pragmatic, and they are most of all customer obsessed, which is amazing. And there's no better takes, like honestly on the, the sort of technical details of this, then there's, especially when it comes to like the value that these investments will, will bring to our customers and our communities. So encourage you to, to, you know, pay more attention to them than you did to me, for sure. >>Brian Gilmore, great stuff. Really appreciate your time. Thank you. >>Yeah, thanks Dave. It was awesome. Look forward to it. >>Yeah, me too. Looking forward to see how the, the community actually applies these new innovations and goes, goes beyond just the historical into the real time, really hot area. As Brian said in a moment, I'll be right back with Anna East Dos Georgio to dig into the critical aspects of key open source components of the Influx DB engine, including Rust, Arrow, Parque, data fusion. Keep it right there. You don't want to miss this.

>>This past May, The Cube in collaboration with Influx data shared with you the latest innovations in Time series databases. We talked at length about why a purpose built time series database for many use cases, was a superior alternative to general purpose databases trying to do the same thing. Now, you may, you may remember the time series data is any data that's stamped in time, and if it's stamped, it can be analyzed historically. And when we introduced the concept to the community, we talked about how in theory, those time slices could be taken, you know, every hour, every minute, every second, you know, down to the millisecond and how the world was moving toward realtime or near realtime data analysis to support physical infrastructure like sensors and other devices and IOT equipment. A time series databases have had to evolve to efficiently support realtime data in emerging use cases in iot T and other use cases. >>And to do that, new architectural innovations have to be brought to bear. As is often the case, open source software is the linchpin to those innovations. Hello and welcome to Evolving Influx DB into the smart Data platform, made possible by influx data and produced by the Cube. My name is Dave Valante and I'll be your host today. Now in this program we're going to dig pretty deep into what's happening with Time series data generally, and specifically how Influx DB is evolving to support new workloads and demands and data, and specifically around data analytics use cases in real time. Now, first we're gonna hear from Brian Gilmore, who is the director of IOT and emerging technologies at Influx Data. And we're gonna talk about the continued evolution of Influx DB and the new capabilities enabled by open source generally and specific tools. And in this program you're gonna hear a lot about things like Rust, implementation of Apache Arrow, the use of par k and tooling such as data fusion, which powering a new engine for Influx db. >>Now, these innovations, they evolve the idea of time series analysis by dramatically increasing the granularity of time series data by compressing the historical time slices, if you will, from, for example, minutes down to milliseconds. And at the same time, enabling real time analytics with an architecture that can process data much faster and much more efficiently. Now, after Brian, we're gonna hear from Anna East Dos Georgio, who is a developer advocate at In Flux Data. And we're gonna get into the why of these open source capabilities and how they contribute to the evolution of the Influx DB platform. And then we're gonna close the program with Tim Yokum, he's the director of engineering at Influx Data, and he's gonna explain how the Influx DB community actually evolved the data engine in mid-flight and which decisions went into the innovations that are coming to the market. Thank you for being here. We hope you enjoy the program. Let's get started. Okay, we're kicking things off with Brian Gilmore. He's the director of i t and emerging Technology at Influx State of Bryan. Welcome to the program. Thanks for coming on. >>Thanks Dave. Great to be here. I appreciate the time. >>Hey, explain why Influx db, you know, needs a new engine. Was there something wrong with the current engine? What's going on there? >>No, no, not at all. I mean, I think it's, for us, it's been about staying ahead of the market. I think, you know, if we think about what our customers are coming to us sort of with now, you know, related to requests like sql, you know, query support, things like that, we have to figure out a way to, to execute those for them in a way that will scale long term. And then we also, we wanna make sure we're innovating, we're sort of staying ahead of the market as well and sort of anticipating those future needs. So, you know, this is really a, a transparent change for our customers. I mean, I think we'll be adding new capabilities over time that sort of leverage this new engine, but you know, initially the customers who are using us are gonna see just great improvements in performance, you know, especially those that are working at the top end of the, of the workload scale, you know, the massive data volumes and things like that. >>Yeah, and we're gonna get into that today and the architecture and the like, but what was the catalyst for the enhancements? I mean, when and how did this all come about? >>Well, I mean, like three years ago we were primarily on premises, right? I mean, I think we had our open source, we had an enterprise product, you know, and, and sort of shifting that technology, especially the open source code base to a service basis where we were hosting it through, you know, multiple cloud providers. That was, that was, that was a long journey I guess, you know, phase one was, you know, we wanted to host enterprise for our customers, so we sort of created a service that we just managed and ran our enterprise product for them. You know, phase two of this cloud effort was to, to optimize for like multi-tenant, multi-cloud, be able to, to host it in a truly like sass manner where we could use, you know, some type of customer activity or consumption as the, the pricing vector, you know, And, and that was sort of the birth of the, of the real first influx DB cloud, you know, which has been really successful. >>We've seen, I think like 60,000 people sign up and we've got tons and tons of, of both enterprises as well as like new companies, developers, and of course a lot of home hobbyists and enthusiasts who are using out on a, on a daily basis, you know, and having that sort of big pool of, of very diverse and very customers to chat with as they're using the product, as they're giving us feedback, et cetera, has has, you know, pointed us in a really good direction in terms of making sure we're continuously improving that and then also making these big leaps as we're doing with this, with this new engine. >>Right. So you've called it a transparent change for customers, so I'm presuming it's non-disruptive, but I really wanna understand how much of a pivot this is and what, what does it take to make that shift from, you know, time series, you know, specialist to real time analytics and being able to support both? >>Yeah, I mean, it's much more of an evolution, I think, than like a shift or a pivot. You know, time series data is always gonna be fundamental and sort of the basis of the solutions that we offer our customers, and then also the ones that they're building on the sort of raw APIs of our platform themselves. You know, the time series market is one that we've worked diligently to lead. I mean, I think when it comes to like metrics, especially like sensor data and app and infrastructure metrics, if we're being honest though, I think our, our user base is well aware that the way we were architected was much more towards those sort of like backwards looking historical type analytics, which are key for troubleshooting and making sure you don't, you know, run into the same problem twice. But, you know, we had to ask ourselves like, what can we do to like better handle those queries from a performance and a, and a, you know, a time to response on the queries, and can we get that to the point where the results sets are coming back so quickly from the time of query that we can like limit that window down to minutes and then seconds. >>And now with this new engine, we're really starting to talk about a query window that could be like returning results in, in, you know, milliseconds of time since it hit the, the, the ingest queue. And that's, that's really getting to the point where as your data is available, you can use it and you can query it, you can visualize it, and you can do all those sort of magical things with it, you know? And I think getting all of that to a place where we're saying like, yes to the customer on, you know, all of the, the real time queries, the, the multiple language query support, but, you know, it was hard, but we're now at a spot where we can start introducing that to, you know, a a limited number of customers, strategic customers and strategic availability zones to start. But you know, everybody over time. >>So you're basically going from what happened to in, you can still do that obviously, but to what's happening now in the moment? >>Yeah, yeah. I mean if you think about time, it's always sort of past, right? I mean, like in the moment right now, whether you're talking about like a millisecond ago or a minute ago, you know, that's, that's pretty much right now, I think for most people, especially in these use cases where you have other sort of components of latency induced by the, by the underlying data collection, the architecture, the infrastructure, the, you know, the, the devices and you know, the sort of highly distributed nature of all of this. So yeah, I mean, getting, getting a customer or a user to be able to use the data as soon as it is available is what we're after here. >>I always thought, you know, real, I always thought of real time as before you lose the customer, but now in this context, maybe it's before the machine blows up. >>Yeah, it's, it's, I mean it is operationally or operational real time is different, you know, and that's one of the things that really triggered us to know that we were, we were heading in the right direction, is just how many sort of operational customers we have. You know, everything from like aerospace and defense. We've got companies monitoring satellites, we've got tons of industrial users, users using us as a processes storing on the plant floor, you know, and, and if we can satisfy their sort of demands for like real time historical perspective, that's awesome. I think what we're gonna do here is we're gonna start to like edge into the real time that they're used to in terms of, you know, the millisecond response times that they expect of their control systems, certainly not their, their historians and databases. >>I, is this available, these innovations to influx DB cloud customers only who can access this capability? >>Yeah. I mean commercially and today, yes. You know, I think we want to emphasize that's a, for now our goal is to get our latest and greatest and our best to everybody over time. Of course. You know, one of the things we had to do here was like we double down on sort of our, our commitment to open source and availability. So like anybody today can take a look at the, the libraries in on our GitHub and, you know, can ex inspect it and even can try to, you know, implement or execute some of it themselves in their own infrastructure. You know, we are, we're committed to bringing our sort of latest and greatest to our cloud customers first for a couple of reasons. Number one, you know, there are big workloads and they have high expectations of us. I think number two, it also gives us the opportunity to monitor a little bit more closely how it's working, how they're using it, like how the system itself is performing. >>And so just, you know, being careful, maybe a little cautious in terms of, of, of how big we go with this right away, just sort of both limits, you know, the risk of, of, you know, any issues that can come with new software rollouts. We haven't seen anything so far, but also it does give us the opportunity to have like meaningful conversations with a small group of users who are using the products, but once we get through that and they give us two thumbs up on it, it'll be like, open the gates and let everybody in. It's gonna be exciting time for the whole ecosystem. >>Yeah, that makes a lot of sense. And you can do some experimentation and, you know, using the cloud resources. Let's dig into some of the architectural and technical innovations that are gonna help deliver on this vision. What, what should we know there? >>Well, I mean, I think foundationally we built the, the new core on Rust. You know, this is a new very sort of popular systems language, you know, it's extremely efficient, but it's also built for speed and memory safety, which goes back to that us being able to like deliver it in a way that is, you know, something we can inspect very closely, but then also rely on the fact that it's going to behave well. And if it does find error conditions, I mean we, we've loved working with Go and, you know, a lot of our libraries will continue to, to be sort of implemented in Go, but you know, when it came to this particular new engine, you know, that power performance and stability rust was critical. On top of that, like, we've also integrated Apache Arrow and Apache Parque for persistence. I think for anybody who's really familiar with the nuts and bolts of our backend and our TSI and our, our time series merged Trees, this is a big break from that, you know, arrow on the sort of in MI side and then Par K in the on disk side. >>It, it allows us to, to present, you know, a unified set of APIs for those really fast real time inquiries that we talked about, as well as for very large, you know, historical sort of bulk data archives in that PARQUE format, which is also cool because there's an entire ecosystem sort of popping up around Parque in terms of the machine learning community, you know, and getting that all to work, we had to glue it together with aero flight. That's sort of what we're using as our, our RPC component. You know, it handles the orchestration and the, the transportation of the Coer data. Now we're moving to like a true Coer database model for this, this version of the engine, you know, and it removes a lot of overhead for us in terms of having to manage all that serialization, the deserialization, and, you know, to that again, like blurring that line between real time and historical data. It's, you know, it's, it's highly optimized for both streaming micro batch and then batches, but true streaming as well. >>Yeah. Again, I mean, it's funny you mentioned Rust. It is, it's been around for a long time, but it's popularity is, is you know, really starting to hit that steep part of the S-curve. And, and we're gonna dig into to more of that, but give us any, is there anything else that we should know about Bryan? Give us the last word? >>Well, I mean, I think first I'd like everybody sort of watching just to like take a look at what we're offering in terms of early access in beta programs. I mean, if, if, if you wanna participate or if you wanna work sort of in terms of early access with the, with the new engine, please reach out to the team. I'm sure you know, there's a lot of communications going out and you know, it'll be highly featured on our, our website, you know, but reach out to the team, believe it or not, like we have a lot more going on than just the new engine. And so there are also other programs, things we're, we're offering to customers in terms of the user interface, data collection and things like that. And, you know, if you're a customer of ours and you have a sales team, a commercial team that you work with, you can reach out to them and see what you can get access to because we can flip a lot of stuff on, especially in cloud through feature flags. >>But if there's something new that you wanna try out, we'd just love to hear from you. And then, you know, our goal would be that as we give you access to all of these new cool features that, you know, you would give us continuous feedback on these products and services, not only like what you need today, but then what you'll need tomorrow to, to sort of build the next versions of your business. Because you know, the whole database, the ecosystem as it expands out into to, you know, this vertically oriented stack of cloud services and enterprise databases and edge databases, you know, it's gonna be what we all make it together, not just, you know, those of us who were employed by Influx db. And then finally I would just say please, like watch in ICE in Tim's sessions, like these are two of our best and brightest, They're totally brilliant, completely pragmatic, and they are most of all customer obsessed, which is amazing. And there's no better takes, like honestly on the, the sort of technical details of this, then there's, especially when it comes to like the value that these investments will, will bring to our customers and our communities. So encourage you to, to, you know, pay more attention to them than you did to me, for sure. >>Brian Gilmore, great stuff. Really appreciate your time. Thank you. >>Yeah, thanks Dave. It was awesome. Look forward to it. >>Yeah, me too. Looking forward to see how the, the community actually applies these new innovations and goes, goes beyond just the historical into the real time really hot area. As Brian said in a moment, I'll be right back with Anna East dos Georgio to dig into the critical aspects of key open source components of the Influx DB engine, including Rust, Arrow, Parque, data fusion. Keep it right there. You don't wanna miss this >>Time series Data is everywhere. The number of sensors, systems and applications generating time series data increases every day. All these data sources producing so much data can cause analysis paralysis. Influx DB is an entire platform designed with everything you need to quickly build applications that generate value from time series data influx. DB Cloud is a serverless solution, which means you don't need to buy or manage your own servers. There's no need to worry about provisioning because you only pay for what you use. Influx DB Cloud is fully managed so you get the newest features and enhancements as they're added to the platform's code base. It also means you can spend time building solutions and delivering value to your users instead of wasting time and effort managing something else. Influx TVB Cloud offers a range of security features to protect your data, multiple layers of redundancy ensure you don't lose any data access controls ensure that only the people who should see your data can see it. >>And encryption protects your data at rest and in transit between any of our regions or cloud providers. InfluxDB uses a single API across the entire platform suite so you can build on open source, deploy to the cloud and then then easily query data in the cloud at the edge or on prem using the same scripts. And InfluxDB is schemaless automatically adjusting to changes in the shape of your data without requiring changes in your application. Logic. InfluxDB Cloud is production ready from day one. All it needs is your data and your imagination. Get started today@influxdata.com slash cloud. >>Okay, we're back. I'm Dave Valante with a Cube and you're watching evolving Influx DB into the smart data platform made possible by influx data. Anna ETOs Georgio is here, she's a developer advocate for influx data and we're gonna dig into the rationale and value contribution behind several open source technologies that Influx DB is leveraging to increase the granularity of time series analysis analysis and bring the world of data into real-time analytics and is welcome to the program. Thanks for coming on. >>Hi, thank you so much. It's a pleasure to be here. >>Oh, you're very welcome. Okay, so IX is being touted as this next gen open source core for Influx db. And my understanding is that it leverages in memory of course for speed. It's a kilo store, so it gives you a compression efficiency, it's gonna give you faster query speeds, you store files and object storage, so you got very cost effective approach. Are these the salient points on the platform? I know there are probably dozens of other features, but what are the high level value points that people should understand? >>Sure, that's a great question. So some of the main requirements that IOx is trying to achieve and some of the most impressive ones to me, the first one is that it aims to have no limits on cardinality and also allow you to write any kind of event data that you want, whether that's live tag or a field. It also wants to deliver the best in class performance on analytics queries. In addition to our already well served metrics queries, we also wanna have operator control over memory usage. So you should be able to define how much memory is used for buffering caching and query processing. Some other really important parts is the ability to have bulk data export and import super useful. Also broader ecosystem compatibility where possible we aim to use and embrace emerging standards in the data analytics ecosystem and have compatibility with things like sql, Python, and maybe even pandas in the future. >>Okay, so lot there. Now we talked to Brian about how you're using Rust and which is not a new programming language and of course we had some drama around Rust during the pandemic with the Mozilla layoffs, but the formation of the Rust Foundation really addressed any of those concerns. You got big guns like Amazon and Google and Microsoft throwing their collective weights behind it. It's really, the adoption is really starting to get steep on the S-curve. So lots of platforms, lots of adoption with rust, but why rust as an alternative to say c plus plus for example? >>Sure, that's a great question. So Russ was chosen because of his exceptional performance and reliability. So while Russ is synt tactically similar to c plus plus and it has similar performance, it also compiles to a native code like c plus plus. But unlike c plus plus, it also has much better memory safety. So memory safety is protection against bugs or security vulnerabilities that lead to excessive memory usage or memory leaks. And rust achieves this memory safety due to its like innovative type system. Additionally, it doesn't allow for dangling pointers. And dangling pointers are the main classes of errors that lead to exploitable security vulnerabilities in languages like c plus plus. So Russ like helps meet that requirement of having no limits on ality, for example, because it's, we're also using the Russ implementation of Apache Arrow and this control over memory and also Russ Russ's packaging system called crates IO offers everything that you need out of the box to have features like AY and a weight to fix race conditions, to protection against buffering overflows and to ensure thread safe async cashing structures as well. So essentially it's just like has all the control, all the fine grain control, you need to take advantage of memory and all your resources as well as possible so that you can handle those really, really high ity use cases. >>Yeah, and the more I learn about the, the new engine and, and the platform IOCs et cetera, you know, you, you see things like, you know, the old days not even to even today you do a lot of garbage collection in these, in these systems and there's an inverse, you know, impact relative to performance. So it looks like you really, you know, the community is modernizing the platform, but I wanna talk about Apache Arrow for a moment. It it's designed to address the constraints that are associated with analyzing large data sets. We, we know that, but please explain why, what, what is Arrow and and what does it bring to Influx db? >>Sure, yeah. So Arrow is a, a framework for defining in memory calmer data. And so much of the efficiency and performance of IOx comes from taking advantage of calmer data structures. And I will, if you don't mind, take a moment to kind of of illustrate why column or data structures are so valuable. Let's pretend that we are gathering field data about the temperature in our room and also maybe the temperature of our stove. And in our table we have those two temperature values as well as maybe a measurement value, timestamp value, maybe some other tag values that describe what room and what house, et cetera we're getting this data from. And so you can picture this table where we have like two rows with the two temperature values for both our room and the stove. Well usually our room temperature is regulated so those values don't change very often. >>So when you have calm oriented st calm oriented storage, essentially you take each row, each column and group it together. And so if that's the case and you're just taking temperature values from the room and a lot of those temperature values are the same, then you'll, you might be able to imagine how equal values will then enable each other and when they neighbor each other in the storage format, this provides a really perfect opportunity for cheap compression. And then this cheap compression enables high cardinality use cases. It also enables for faster scan rates. So if you wanna define like the men and max value of the temperature in the room across a thousand different points, you only have to get those a thousand different points in order to answer that question and you have those immediately available to you. But let's contrast this with a row oriented storage solution instead so that we can understand better the benefits of calmer oriented storage. >>So if you had a row oriented storage, you'd first have to look at every field like the temperature in, in the room and the temperature of the stove. You'd have to go across every tag value that maybe describes where the room is located or what model the stove is. And every timestamp you'd then have to pluck out that one temperature value that you want at that one time stamp and do that for every single row. So you're scanning across a ton more data and that's why Rowe Oriented doesn't provide the same efficiency as calmer and Apache Arrow is in memory calmer data, commoner data fit framework. So that's where a lot of the advantages come >>From. Okay. So you basically described like a traditional database, a row approach, but I've seen like a lot of traditional database say, okay, now we've got, we can handle colo format versus what you're talking about is really, you know, kind of native i, is it not as effective? Is the, is the foreman not as effective because it's largely a, a bolt on? Can you, can you like elucidate on that front? >>Yeah, it's, it's not as effective because you have more expensive compression and because you can't scan across the values as quickly. And so those are, that's pretty much the main reasons why, why RO row oriented storage isn't as efficient as calm, calmer oriented storage. Yeah. >>Got it. So let's talk about Arrow Data Fusion. What is data fusion? I know it's written in Rust, but what does it bring to the table here? >>Sure. So it's an extensible query execution framework and it uses Arrow as it's in memory format. So the way that it helps in influx DB IOCs is that okay, it's great if you can write unlimited amount of cardinality into influx Cbis, but if you don't have a query engine that can successfully query that data, then I don't know how much value it is for you. So Data fusion helps enable the, the query process and transformation of that data. It also has a PANDAS API so that you could take advantage of PANDAS data frames as well and all of the machine learning tools associated with Pandas. >>Okay. You're also leveraging Par K in the platform cause we heard a lot about Par K in the middle of the last decade cuz as a storage format to improve on Hadoop column stores. What are you doing with Parque and why is it important? >>Sure. So parque is the column oriented durable file format. So it's important because it'll enable bulk import, bulk export, it has compatibility with Python and Pandas, so it supports a broader ecosystem. Par K files also take very little disc disc space and they're faster to scan because again, they're column oriented in particular, I think PAR K files are like 16 times cheaper than CSV files, just as kind of a point of reference. And so that's essentially a lot of the, the benefits of par k. >>Got it. Very popular. So and he's, what exactly is influx data focusing on as a committer to these projects? What is your focus? What's the value that you're bringing to the community? >>Sure. So Influx DB first has contributed a lot of different, different things to the Apache ecosystem. For example, they contribute an implementation of Apache Arrow and go and that will support clearing with flux. Also, there has been a quite a few contributions to data fusion for things like memory optimization and supportive additional SQL features like support for timestamp, arithmetic and support for exist clauses and support for memory control. So yeah, Influx has contributed a a lot to the Apache ecosystem and continues to do so. And I think kind of the idea here is that if you can improve these upstream projects and then the long term strategy here is that the more you contribute and build those up, then the more you will perpetuate that cycle of improvement and the more we will invest in our own project as well. So it's just that kind of symbiotic relationship and appreciation of the open source community. >>Yeah. Got it. You got that virtuous cycle going, the people call the flywheel. Give us your last thoughts and kind of summarize, you know, where what, what the big takeaways are from your perspective. >>So I think the big takeaway is that influx data is doing a lot of really exciting things with Influx DB IOx and I really encourage, if you are interested in learning more about the technologies that Influx is leveraging to produce IOCs, the challenges associated with it and all of the hard work questions and you just wanna learn more, then I would encourage you to go to the monthly Tech talks and community office hours and they are on every second Wednesday of the month at 8:30 AM Pacific time. There's also a community forums and a community Slack channel look for the influx DDB unders IAC channel specifically to learn more about how to join those office hours and those monthly tech tech talks as well as ask any questions they have about iacs, what to expect and what you'd like to learn more about. I as a developer advocate, I wanna answer your questions. So if there's a particular technology or stack that you wanna dive deeper into and want more explanation about how INFLUX DB leverages it to build IOCs, I will be really excited to produce content on that topic for you. >>Yeah, that's awesome. You guys have a really rich community, collaborate with your peers, solve problems, and, and you guys super responsive, so really appreciate that. All right, thank you so much Anise for explaining all this open source stuff to the audience and why it's important to the future of data. >>Thank you. I really appreciate it. >>All right, you're very welcome. Okay, stay right there and in a moment I'll be back with Tim Yoakum, he's the director of engineering for Influx Data and we're gonna talk about how you update a SAS engine while the plane is flying at 30,000 feet. You don't wanna miss this. >>I'm really glad that we went with InfluxDB Cloud for our hosting because it has saved us a ton of time. It's helped us move faster, it's saved us money. And also InfluxDB has good support. My name's Alex Nada. I am CTO at Noble nine. Noble Nine is a platform to measure and manage service level objectives, which is a great way of measuring the reliability of your systems. You can essentially think of an slo, the product we're providing to our customers as a bunch of time series. So we need a way to store that data and the corresponding time series that are related to those. The main reason that we settled on InfluxDB as we were shopping around is that InfluxDB has a very flexible query language and as a general purpose time series database, it basically had the set of features we were looking for. >>As our platform has grown, we found InfluxDB Cloud to be a really scalable solution. We can quickly iterate on new features and functionality because Influx Cloud is entirely managed, it probably saved us at least a full additional person on our team. We also have the option of running InfluxDB Enterprise, which gives us the ability to even host off the cloud or in a private cloud if that's preferred by a customer. Influx data has been really flexible in adapting to the hosting requirements that we have. They listened to the challenges we were facing and they helped us solve it. As we've continued to grow, I'm really happy we have influx data by our side. >>Okay, we're back with Tim Yokum, who is the director of engineering at Influx Data. Tim, welcome. Good to see you. >>Good to see you. Thanks for having me. >>You're really welcome. Listen, we've been covering open source software in the cube for more than a decade, and we've kind of watched the innovation from the big data ecosystem. The cloud has been being built out on open source, mobile, social platforms, key databases, and of course influx DB and influx data has been a big consumer and contributor of open source software. So my question to you is, where have you seen the biggest bang for the buck from open source software? >>So yeah, you know, influx really, we thrive at the intersection of commercial services and open, so open source software. So OSS keeps us on the cutting edge. We benefit from OSS in delivering our own service from our core storage engine technologies to web services temping engines. Our, our team stays lean and focused because we build on proven tools. We really build on the shoulders of giants and like you've mentioned, even better, we contribute a lot back to the projects that we use as well as our own product influx db. >>You know, but I gotta ask you, Tim, because one of the challenge that that we've seen in particular, you saw this in the heyday of Hadoop, the, the innovations come so fast and furious and as a software company you gotta place bets, you gotta, you know, commit people and sometimes those bets can be risky and not pay off well, how have you managed this challenge? >>Oh, it moves fast. Yeah, that, that's a benefit though because it, the community moves so quickly that today's hot technology can be tomorrow's dinosaur. And what we, what we tend to do is, is we fail fast and fail often. We try a lot of things. You know, you look at Kubernetes for example, that ecosystem is driven by thousands of intelligent developers, engineers, builders, they're adding value every day. So we have to really keep up with that. And as the stack changes, we, we try different technologies, we try different methods, and at the end of the day, we come up with a better platform as a result of just the constant change in the environment. It is a challenge for us, but it's, it's something that we just do every day. >>So we have a survey partner down in New York City called Enterprise Technology Research etr, and they do these quarterly surveys of about 1500 CIOs, IT practitioners, and they really have a good pulse on what's happening with spending. And the data shows that containers generally, but specifically Kubernetes is one of the areas that has kind of, it's been off the charts and seen the most significant adoption and velocity particularly, you know, along with cloud. But, but really Kubernetes is just, you know, still up until the right consistently even with, you know, the macro headwinds and all, all of the stuff that we're sick of talking about. But, so what are you doing with Kubernetes in the platform? >>Yeah, it, it's really central to our ability to run the product. When we first started out, we were just on AWS and, and the way we were running was, was a little bit like containers junior. Now we're running Kubernetes everywhere at aws, Azure, Google Cloud. It allows us to have a consistent experience across three different cloud providers and we can manage that in code so our developers can focus on delivering services, not trying to learn the intricacies of Amazon, Azure, and Google and figure out how to deliver services on those three clouds with all of their differences. >>Just to follow up on that, is it, no. So I presume it's sounds like there's a PAs layer there to allow you guys to have a consistent experience across clouds and out to the edge, you know, wherever is that, is that correct? >>Yeah, so we've basically built more or less platform engineering, This is the new hot phrase, you know, it, it's, Kubernetes has made a lot of things easy for us because we've built a platform that our developers can lean on and they only have to learn one way of deploying their application, managing their application. And so that, that just gets all of the underlying infrastructure out of the way and, and lets them focus on delivering influx cloud. >>Yeah, and I know I'm taking a little bit of a tangent, but is that, that, I'll call it a PAs layer if I can use that term. Is that, are there specific attributes to Influx db or is it kind of just generally off the shelf paths? You know, are there, is, is there any purpose built capability there that, that is, is value add or is it pretty much generic? >>So we really build, we, we look at things through, with a build versus buy through a, a build versus by lens. Some things we want to leverage cloud provider services, for instance, Postgres databases for metadata, perhaps we'll get that off of our plate, let someone else run that. We're going to deploy a platform that our engineers can, can deliver on that has consistency that is, is all generated from code that we can as a, as an SRE group, as an ops team, that we can manage with very few people really, and we can stamp out clusters across multiple regions and in no time. >>So how, so sometimes you build, sometimes you buy it. How do you make those decisions and and what does that mean for the, for the platform and for customers? >>Yeah, so what we're doing is, it's like everybody else will do, we're we're looking for trade offs that make sense. You know, we really want to protect our customers data. So we look for services that support our own software with the most uptime, reliability, and durability we can get. Some things are just going to be easier to have a cloud provider take care of on our behalf. We make that transparent for our own team. And of course for customers you don't even see that, but we don't want to try to reinvent the wheel, like I had mentioned with SQL data stores for metadata, perhaps let's build on top of what of these three large cloud providers have already perfected. And we can then focus on our platform engineering and we can have our developers then focus on the influx data, software, influx, cloud software. >>So take it to the customer level, what does it mean for them? What's the value that they're gonna get out of all these innovations that we've been been talking about today and what can they expect in the future? >>So first of all, people who use the OSS product are really gonna be at home on our cloud platform. You can run it on your desktop machine, on a single server, what have you, but then you want to scale up. We have some 270 terabytes of data across, over 4 billion series keys that people have stored. So there's a proven ability to scale now in terms of the open source, open source software and how we've developed the platform. You're getting highly available high cardinality time series platform. We manage it and, and really as, as I mentioned earlier, we can keep up with the state of the art. We keep reinventing, we keep deploying things in real time. We deploy to our platform every day repeatedly all the time. And it's that continuous deployment that allows us to continue testing things in flight, rolling things out that change new features, better ways of doing deployments, safer ways of doing deployments. >>All of that happens behind the scenes. And like we had mentioned earlier, Kubernetes, I mean that, that allows us to get that done. We couldn't do it without having that platform as a, as a base layer for us to then put our software on. So we, we iterate quickly. When you're on the, the Influx cloud platform, you really are able to, to take advantage of new features immediately. We roll things out every day and as those things go into production, you have, you have the ability to, to use them. And so in the end we want you to focus on getting actual insights from your data instead of running infrastructure, you know, let, let us do that for you. So, >>And that makes sense, but so is the, is the, are the innovations that we're talking about in the evolution of Influx db, do, do you see that as sort of a natural evolution for existing customers? I, is it, I'm sure the answer is both, but is it opening up new territory for customers? Can you add some color to that? >>Yeah, it really is it, it's a little bit of both. Any engineer will say, well, it depends. So cloud native technologies are, are really the hot thing. Iot, industrial iot especially, people want to just shove tons of data out there and be able to do queries immediately and they don't wanna manage infrastructure. What we've started to see are people that use the cloud service as their, their data store backbone and then they use edge computing with R OSS product to ingest data from say, multiple production lines and downsample that data, send the rest of that data off influx cloud where the heavy processing takes place. So really us being in all the different clouds and iterating on that and being in all sorts of different regions allows for people to really get out of the, the business of man trying to manage that big data, have us take care of that. And of course as we change the platform end users benefit from that immediately. And, >>And so obviously taking away a lot of the heavy lifting for the infrastructure, would you say the same thing about security, especially as you go out to IOT and the Edge? How should we be thinking about the value that you bring from a security perspective? >>Yeah, we take, we take security super seriously. It, it's built into our dna. We do a lot of work to ensure that our platform is secure, that the data we store is, is kept private. It's of course always a concern. You see in the news all the time, companies being compromised, you know, that's something that you can have an entire team working on, which we do to make sure that the data that you have, whether it's in transit, whether it's at rest, is always kept secure, is only viewable by you. You know, you look at things like software, bill of materials, if you're running this yourself, you have to go vet all sorts of different pieces of software. And we do that, you know, as we use new tools. That's something that, that's just part of our jobs to make sure that the platform that we're running it has, has fully vetted software and, and with open source especially, that's a lot of work. And so it's, it's definitely new territory. Supply chain attacks are, are definitely happening at a higher clip than they used to, but that is, that is really just part of a day in the, the life for folks like us that are, are building platforms. >>Yeah, and that's key. I mean especially when you start getting into the, the, you know, we talk about IOT and the operations technologies, the engineers running the, that infrastructure, you know, historically, as you know, Tim, they, they would air gap everything. That's how they kept it safe. But that's not feasible anymore. Everything's >>That >>Connected now, right? And so you've gotta have a partner that is again, take away that heavy lifting to r and d so you can focus on some of the other activities. Right. Give us the, the last word and the, the key takeaways from your perspective. >>Well, you know, from my perspective I see it as, as a a two lane approach with, with influx, with Anytime series data, you know, you've got a lot of stuff that you're gonna run on-prem, what you had mentioned, air gaping. Sure there's plenty of need for that, but at the end of the day, people that don't want to run big data centers, people that want torus their data to, to a company that's, that's got a full platform set up for them that they can build on, send that data over to the cloud, the cloud is not going away. I think more hybrid approach is, is where the future lives and that's what we're prepared for. >>Tim, really appreciate you coming to the program. Great stuff. Good to see you. >>Thanks very much. Appreciate it. >>Okay, in a moment I'll be back to wrap up. Today's session, you're watching The Cube. >>Are you looking for some help getting started with InfluxDB Telegraph or Flux Check >>Out Influx DB University >>Where you can find our entire catalog of free training that will help you make the most of your time series data >>Get >>Started for free@influxdbu.com. >>We'll see you in class. >>Okay, so we heard today from three experts on time series and data, how the Influx DB platform is evolving to support new ways of analyzing large data sets very efficiently and effectively in real time. And we learned that key open source components like Apache Arrow and the Rust Programming environment Data fusion par K are being leveraged to support realtime data analytics at scale. We also learned about the contributions in importance of open source software and how the Influx DB community is evolving the platform with minimal disruption to support new workloads, new use cases, and the future of realtime data analytics. Now remember these sessions, they're all available on demand. You can go to the cube.net to find those. Don't forget to check out silicon angle.com for all the news related to things enterprise and emerging tech. And you should also check out influx data.com. There you can learn about the company's products. You'll find developer resources like free courses. You could join the developer community and work with your peers to learn and solve problems. And there are plenty of other resources around use cases and customer stories on the website. This is Dave Valante. Thank you for watching Evolving Influx DB into the smart data platform, made possible by influx data and brought to you by the Cube, your leader in enterprise and emerging tech coverage.

>>This past May, The Cube in collaboration with Influx data shared with you the latest innovations in Time series databases. We talked at length about why a purpose built time series database for many use cases, was a superior alternative to general purpose databases trying to do the same thing. Now, you may, you may remember the time series data is any data that's stamped in time, and if it's stamped, it can be analyzed historically. And when we introduced the concept to the community, we talked about how in theory, those time slices could be taken, you know, every hour, every minute, every second, you know, down to the millisecond and how the world was moving toward realtime or near realtime data analysis to support physical infrastructure like sensors and other devices and IOT equipment. A time series databases have had to evolve to efficiently support realtime data in emerging use cases in iot T and other use cases. >>And to do that, new architectural innovations have to be brought to bear. As is often the case, open source software is the linchpin to those innovations. Hello and welcome to Evolving Influx DB into the smart Data platform, made possible by influx data and produced by the Cube. My name is Dave Valante and I'll be your host today. Now in this program we're going to dig pretty deep into what's happening with Time series data generally, and specifically how Influx DB is evolving to support new workloads and demands and data, and specifically around data analytics use cases in real time. Now, first we're gonna hear from Brian Gilmore, who is the director of IOT and emerging technologies at Influx Data. And we're gonna talk about the continued evolution of Influx DB and the new capabilities enabled by open source generally and specific tools. And in this program you're gonna hear a lot about things like Rust, implementation of Apache Arrow, the use of par k and tooling such as data fusion, which powering a new engine for Influx db. >>Now, these innovations, they evolve the idea of time series analysis by dramatically increasing the granularity of time series data by compressing the historical time slices, if you will, from, for example, minutes down to milliseconds. And at the same time, enabling real time analytics with an architecture that can process data much faster and much more efficiently. Now, after Brian, we're gonna hear from Anna East Dos Georgio, who is a developer advocate at In Flux Data. And we're gonna get into the why of these open source capabilities and how they contribute to the evolution of the Influx DB platform. And then we're gonna close the program with Tim Yokum, he's the director of engineering at Influx Data, and he's gonna explain how the Influx DB community actually evolved the data engine in mid-flight and which decisions went into the innovations that are coming to the market. Thank you for being here. We hope you enjoy the program. Let's get started. Okay, we're kicking things off with Brian Gilmore. He's the director of i t and emerging Technology at Influx State of Bryan. Welcome to the program. Thanks for coming on. >>Thanks Dave. Great to be here. I appreciate the time. >>Hey, explain why Influx db, you know, needs a new engine. Was there something wrong with the current engine? What's going on there? >>No, no, not at all. I mean, I think it's, for us, it's been about staying ahead of the market. I think, you know, if we think about what our customers are coming to us sort of with now, you know, related to requests like sql, you know, query support, things like that, we have to figure out a way to, to execute those for them in a way that will scale long term. And then we also, we wanna make sure we're innovating, we're sort of staying ahead of the market as well and sort of anticipating those future needs. So, you know, this is really a, a transparent change for our customers. I mean, I think we'll be adding new capabilities over time that sort of leverage this new engine, but you know, initially the customers who are using us are gonna see just great improvements in performance, you know, especially those that are working at the top end of the, of the workload scale, you know, the massive data volumes and things like that. >>Yeah, and we're gonna get into that today and the architecture and the like, but what was the catalyst for the enhancements? I mean, when and how did this all come about? >>Well, I mean, like three years ago we were primarily on premises, right? I mean, I think we had our open source, we had an enterprise product, you know, and, and sort of shifting that technology, especially the open source code base to a service basis where we were hosting it through, you know, multiple cloud providers. That was, that was, that was a long journey I guess, you know, phase one was, you know, we wanted to host enterprise for our customers, so we sort of created a service that we just managed and ran our enterprise product for them. You know, phase two of this cloud effort was to, to optimize for like multi-tenant, multi-cloud, be able to, to host it in a truly like sass manner where we could use, you know, some type of customer activity or consumption as the, the pricing vector, you know, And, and that was sort of the birth of the, of the real first influx DB cloud, you know, which has been really successful. >>We've seen, I think like 60,000 people sign up and we've got tons and tons of, of both enterprises as well as like new companies, developers, and of course a lot of home hobbyists and enthusiasts who are using out on a, on a daily basis, you know, and having that sort of big pool of, of very diverse and very customers to chat with as they're using the product, as they're giving us feedback, et cetera, has has, you know, pointed us in a really good direction in terms of making sure we're continuously improving that and then also making these big leaps as we're doing with this, with this new engine. >>Right. So you've called it a transparent change for customers, so I'm presuming it's non-disruptive, but I really wanna understand how much of a pivot this is and what, what does it take to make that shift from, you know, time series, you know, specialist to real time analytics and being able to support both? >>Yeah, I mean, it's much more of an evolution, I think, than like a shift or a pivot. You know, time series data is always gonna be fundamental and sort of the basis of the solutions that we offer our customers, and then also the ones that they're building on the sort of raw APIs of our platform themselves. You know, the time series market is one that we've worked diligently to lead. I mean, I think when it comes to like metrics, especially like sensor data and app and infrastructure metrics, if we're being honest though, I think our, our user base is well aware that the way we were architected was much more towards those sort of like backwards looking historical type analytics, which are key for troubleshooting and making sure you don't, you know, run into the same problem twice. But, you know, we had to ask ourselves like, what can we do to like better handle those queries from a performance and a, and a, you know, a time to response on the queries, and can we get that to the point where the results sets are coming back so quickly from the time of query that we can like limit that window down to minutes and then seconds. >>And now with this new engine, we're really starting to talk about a query window that could be like returning results in, in, you know, milliseconds of time since it hit the, the, the ingest queue. And that's, that's really getting to the point where as your data is available, you can use it and you can query it, you can visualize it, and you can do all those sort of magical things with it, you know? And I think getting all of that to a place where we're saying like, yes to the customer on, you know, all of the, the real time queries, the, the multiple language query support, but, you know, it was hard, but we're now at a spot where we can start introducing that to, you know, a a limited number of customers, strategic customers and strategic availability zones to start. But you know, everybody over time. >>So you're basically going from what happened to in, you can still do that obviously, but to what's happening now in the moment? >>Yeah, yeah. I mean if you think about time, it's always sort of past, right? I mean, like in the moment right now, whether you're talking about like a millisecond ago or a minute ago, you know, that's, that's pretty much right now, I think for most people, especially in these use cases where you have other sort of components of latency induced by the, by the underlying data collection, the architecture, the infrastructure, the, you know, the, the devices and you know, the sort of highly distributed nature of all of this. So yeah, I mean, getting, getting a customer or a user to be able to use the data as soon as it is available is what we're after here. >>I always thought, you know, real, I always thought of real time as before you lose the customer, but now in this context, maybe it's before the machine blows up. >>Yeah, it's, it's, I mean it is operationally or operational real time is different, you know, and that's one of the things that really triggered us to know that we were, we were heading in the right direction, is just how many sort of operational customers we have. You know, everything from like aerospace and defense. We've got companies monitoring satellites, we've got tons of industrial users, users using us as a processes storing on the plant floor, you know, and, and if we can satisfy their sort of demands for like real time historical perspective, that's awesome. I think what we're gonna do here is we're gonna start to like edge into the real time that they're used to in terms of, you know, the millisecond response times that they expect of their control systems, certainly not their, their historians and databases. >>I, is this available, these innovations to influx DB cloud customers only who can access this capability? >>Yeah. I mean commercially and today, yes. You know, I think we want to emphasize that's a, for now our goal is to get our latest and greatest and our best to everybody over time. Of course. You know, one of the things we had to do here was like we double down on sort of our, our commitment to open source and availability. So like anybody today can take a look at the, the libraries in on our GitHub and, you know, can ex inspect it and even can try to, you know, implement or execute some of it themselves in their own infrastructure. You know, we are, we're committed to bringing our sort of latest and greatest to our cloud customers first for a couple of reasons. Number one, you know, there are big workloads and they have high expectations of us. I think number two, it also gives us the opportunity to monitor a little bit more closely how it's working, how they're using it, like how the system itself is performing. >>And so just, you know, being careful, maybe a little cautious in terms of, of, of how big we go with this right away, just sort of both limits, you know, the risk of, of, you know, any issues that can come with new software rollouts. We haven't seen anything so far, but also it does give us the opportunity to have like meaningful conversations with a small group of users who are using the products, but once we get through that and they give us two thumbs up on it, it'll be like, open the gates and let everybody in. It's gonna be exciting time for the whole ecosystem. >>Yeah, that makes a lot of sense. And you can do some experimentation and, you know, using the cloud resources. Let's dig into some of the architectural and technical innovations that are gonna help deliver on this vision. What, what should we know there? >>Well, I mean, I think foundationally we built the, the new core on Rust. You know, this is a new very sort of popular systems language, you know, it's extremely efficient, but it's also built for speed and memory safety, which goes back to that us being able to like deliver it in a way that is, you know, something we can inspect very closely, but then also rely on the fact that it's going to behave well. And if it does find error conditions, I mean we, we've loved working with Go and, you know, a lot of our libraries will continue to, to be sort of implemented in Go, but you know, when it came to this particular new engine, you know, that power performance and stability rust was critical. On top of that, like, we've also integrated Apache Arrow and Apache Parque for persistence. I think for anybody who's really familiar with the nuts and bolts of our backend and our TSI and our, our time series merged Trees, this is a big break from that, you know, arrow on the sort of in MI side and then Par K in the on disk side. >>It, it allows us to, to present, you know, a unified set of APIs for those really fast real time inquiries that we talked about, as well as for very large, you know, historical sort of bulk data archives in that PARQUE format, which is also cool because there's an entire ecosystem sort of popping up around Parque in terms of the machine learning community, you know, and getting that all to work, we had to glue it together with aero flight. That's sort of what we're using as our, our RPC component. You know, it handles the orchestration and the, the transportation of the Coer data. Now we're moving to like a true Coer database model for this, this version of the engine, you know, and it removes a lot of overhead for us in terms of having to manage all that serialization, the deserialization, and, you know, to that again, like blurring that line between real time and historical data. It's, you know, it's, it's highly optimized for both streaming micro batch and then batches, but true streaming as well. >>Yeah. Again, I mean, it's funny you mentioned Rust. It is, it's been around for a long time, but it's popularity is, is you know, really starting to hit that steep part of the S-curve. And, and we're gonna dig into to more of that, but give us any, is there anything else that we should know about Bryan? Give us the last word? >>Well, I mean, I think first I'd like everybody sort of watching just to like take a look at what we're offering in terms of early access in beta programs. I mean, if, if, if you wanna participate or if you wanna work sort of in terms of early access with the, with the new engine, please reach out to the team. I'm sure you know, there's a lot of communications going out and you know, it'll be highly featured on our, our website, you know, but reach out to the team, believe it or not, like we have a lot more going on than just the new engine. And so there are also other programs, things we're, we're offering to customers in terms of the user interface, data collection and things like that. And, you know, if you're a customer of ours and you have a sales team, a commercial team that you work with, you can reach out to them and see what you can get access to because we can flip a lot of stuff on, especially in cloud through feature flags. >>But if there's something new that you wanna try out, we'd just love to hear from you. And then, you know, our goal would be that as we give you access to all of these new cool features that, you know, you would give us continuous feedback on these products and services, not only like what you need today, but then what you'll need tomorrow to, to sort of build the next versions of your business. Because you know, the whole database, the ecosystem as it expands out into to, you know, this vertically oriented stack of cloud services and enterprise databases and edge databases, you know, it's gonna be what we all make it together, not just, you know, those of us who were employed by Influx db. And then finally I would just say please, like watch in ICE in Tim's sessions, like these are two of our best and brightest, They're totally brilliant, completely pragmatic, and they are most of all customer obsessed, which is amazing. And there's no better takes, like honestly on the, the sort of technical details of this, then there's, especially when it comes to like the value that these investments will, will bring to our customers and our communities. So encourage you to, to, you know, pay more attention to them than you did to me, for sure. >>Brian Gilmore, great stuff. Really appreciate your time. Thank you. >>Yeah, thanks Dave. It was awesome. Look forward to it. >>Yeah, me too. Looking forward to see how the, the community actually applies these new innovations and goes, goes beyond just the historical into the real time really hot area. As Brian said in a moment, I'll be right back with Anna East dos Georgio to dig into the critical aspects of key open source components of the Influx DB engine, including Rust, Arrow, Parque, data fusion. Keep it right there. You don't wanna miss this >>Time series Data is everywhere. The number of sensors, systems and applications generating time series data increases every day. All these data sources producing so much data can cause analysis paralysis. Influx DB is an entire platform designed with everything you need to quickly build applications that generate value from time series data influx. DB Cloud is a serverless solution, which means you don't need to buy or manage your own servers. There's no need to worry about provisioning because you only pay for what you use. Influx DB Cloud is fully managed so you get the newest features and enhancements as they're added to the platform's code base. It also means you can spend time building solutions and delivering value to your users instead of wasting time and effort managing something else. Influx TVB Cloud offers a range of security features to protect your data, multiple layers of redundancy ensure you don't lose any data access controls ensure that only the people who should see your data can see it. >>And encryption protects your data at rest and in transit between any of our regions or cloud providers. InfluxDB uses a single API across the entire platform suite so you can build on open source, deploy to the cloud and then then easily query data in the cloud at the edge or on prem using the same scripts. And InfluxDB is schemaless automatically adjusting to changes in the shape of your data without requiring changes in your application. Logic. InfluxDB Cloud is production ready from day one. All it needs is your data and your imagination. Get started today@influxdata.com slash cloud. >>Okay, we're back. I'm Dave Valante with a Cube and you're watching evolving Influx DB into the smart data platform made possible by influx data. Anna ETOs Georgio is here, she's a developer advocate for influx data and we're gonna dig into the rationale and value contribution behind several open source technologies that Influx DB is leveraging to increase the granularity of time series analysis analysis and bring the world of data into real-time analytics and is welcome to the program. Thanks for coming on. >>Hi, thank you so much. It's a pleasure to be here. >>Oh, you're very welcome. Okay, so IX is being touted as this next gen open source core for Influx db. And my understanding is that it leverages in memory of course for speed. It's a kilo store, so it gives you a compression efficiency, it's gonna give you faster query speeds, you store files and object storage, so you got very cost effective approach. Are these the salient points on the platform? I know there are probably dozens of other features, but what are the high level value points that people should understand? >>Sure, that's a great question. So some of the main requirements that IOx is trying to achieve and some of the most impressive ones to me, the first one is that it aims to have no limits on cardinality and also allow you to write any kind of event data that you want, whether that's live tag or a field. It also wants to deliver the best in class performance on analytics queries. In addition to our already well served metrics queries, we also wanna have operator control over memory usage. So you should be able to define how much memory is used for buffering caching and query processing. Some other really important parts is the ability to have bulk data export and import super useful. Also broader ecosystem compatibility where possible we aim to use and embrace emerging standards in the data analytics ecosystem and have compatibility with things like sql, Python, and maybe even pandas in the future. >>Okay, so lot there. Now we talked to Brian about how you're using Rust and which is not a new programming language and of course we had some drama around Rust during the pandemic with the Mozilla layoffs, but the formation of the Rust Foundation really addressed any of those concerns. You got big guns like Amazon and Google and Microsoft throwing their collective weights behind it. It's really, the adoption is really starting to get steep on the S-curve. So lots of platforms, lots of adoption with rust, but why rust as an alternative to say c plus plus for example? >>Sure, that's a great question. So Russ was chosen because of his exceptional performance and reliability. So while Russ is synt tactically similar to c plus plus and it has similar performance, it also compiles to a native code like c plus plus. But unlike c plus plus, it also has much better memory safety. So memory safety is protection against bugs or security vulnerabilities that lead to excessive memory usage or memory leaks. And rust achieves this memory safety due to its like innovative type system. Additionally, it doesn't allow for dangling pointers. And dangling pointers are the main classes of errors that lead to exploitable security vulnerabilities in languages like c plus plus. So Russ like helps meet that requirement of having no limits on ality, for example, because it's, we're also using the Russ implementation of Apache Arrow and this control over memory and also Russ Russ's packaging system called crates IO offers everything that you need out of the box to have features like AY and a weight to fix race conditions, to protection against buffering overflows and to ensure thread safe async cashing structures as well. So essentially it's just like has all the control, all the fine grain control, you need to take advantage of memory and all your resources as well as possible so that you can handle those really, really high ity use cases. >>Yeah, and the more I learn about the, the new engine and, and the platform IOCs et cetera, you know, you, you see things like, you know, the old days not even to even today you do a lot of garbage collection in these, in these systems and there's an inverse, you know, impact relative to performance. So it looks like you really, you know, the community is modernizing the platform, but I wanna talk about Apache Arrow for a moment. It it's designed to address the constraints that are associated with analyzing large data sets. We, we know that, but please explain why, what, what is Arrow and and what does it bring to Influx db? >>Sure, yeah. So Arrow is a, a framework for defining in memory calmer data. And so much of the efficiency and performance of IOx comes from taking advantage of calmer data structures. And I will, if you don't mind, take a moment to kind of of illustrate why column or data structures are so valuable. Let's pretend that we are gathering field data about the temperature in our room and also maybe the temperature of our stove. And in our table we have those two temperature values as well as maybe a measurement value, timestamp value, maybe some other tag values that describe what room and what house, et cetera we're getting this data from. And so you can picture this table where we have like two rows with the two temperature values for both our room and the stove. Well usually our room temperature is regulated so those values don't change very often. >>So when you have calm oriented st calm oriented storage, essentially you take each row, each column and group it together. And so if that's the case and you're just taking temperature values from the room and a lot of those temperature values are the same, then you'll, you might be able to imagine how equal values will then enable each other and when they neighbor each other in the storage format, this provides a really perfect opportunity for cheap compression. And then this cheap compression enables high cardinality use cases. It also enables for faster scan rates. So if you wanna define like the men and max value of the temperature in the room across a thousand different points, you only have to get those a thousand different points in order to answer that question and you have those immediately available to you. But let's contrast this with a row oriented storage solution instead so that we can understand better the benefits of calmer oriented storage. >>So if you had a row oriented storage, you'd first have to look at every field like the temperature in, in the room and the temperature of the stove. You'd have to go across every tag value that maybe describes where the room is located or what model the stove is. And every timestamp you'd then have to pluck out that one temperature value that you want at that one time stamp and do that for every single row. So you're scanning across a ton more data and that's why Rowe Oriented doesn't provide the same efficiency as calmer and Apache Arrow is in memory calmer data, commoner data fit framework. So that's where a lot of the advantages come >>From. Okay. So you basically described like a traditional database, a row approach, but I've seen like a lot of traditional database say, okay, now we've got, we can handle colo format versus what you're talking about is really, you know, kind of native i, is it not as effective? Is the, is the foreman not as effective because it's largely a, a bolt on? Can you, can you like elucidate on that front? >>Yeah, it's, it's not as effective because you have more expensive compression and because you can't scan across the values as quickly. And so those are, that's pretty much the main reasons why, why RO row oriented storage isn't as efficient as calm, calmer oriented storage. Yeah. >>Got it. So let's talk about Arrow Data Fusion. What is data fusion? I know it's written in Rust, but what does it bring to the table here? >>Sure. So it's an extensible query execution framework and it uses Arrow as it's in memory format. So the way that it helps in influx DB IOCs is that okay, it's great if you can write unlimited amount of cardinality into influx Cbis, but if you don't have a query engine that can successfully query that data, then I don't know how much value it is for you. So Data fusion helps enable the, the query process and transformation of that data. It also has a PANDAS API so that you could take advantage of PANDAS data frames as well and all of the machine learning tools associated with Pandas. >>Okay. You're also leveraging Par K in the platform cause we heard a lot about Par K in the middle of the last decade cuz as a storage format to improve on Hadoop column stores. What are you doing with Parque and why is it important? >>Sure. So parque is the column oriented durable file format. So it's important because it'll enable bulk import, bulk export, it has compatibility with Python and Pandas, so it supports a broader ecosystem. Par K files also take very little disc disc space and they're faster to scan because again, they're column oriented in particular, I think PAR K files are like 16 times cheaper than CSV files, just as kind of a point of reference. And so that's essentially a lot of the, the benefits of par k. >>Got it. Very popular. So and he's, what exactly is influx data focusing on as a committer to these projects? What is your focus? What's the value that you're bringing to the community? >>Sure. So Influx DB first has contributed a lot of different, different things to the Apache ecosystem. For example, they contribute an implementation of Apache Arrow and go and that will support clearing with flux. Also, there has been a quite a few contributions to data fusion for things like memory optimization and supportive additional SQL features like support for timestamp, arithmetic and support for exist clauses and support for memory control. So yeah, Influx has contributed a a lot to the Apache ecosystem and continues to do so. And I think kind of the idea here is that if you can improve these upstream projects and then the long term strategy here is that the more you contribute and build those up, then the more you will perpetuate that cycle of improvement and the more we will invest in our own project as well. So it's just that kind of symbiotic relationship and appreciation of the open source community. >>Yeah. Got it. You got that virtuous cycle going, the people call the flywheel. Give us your last thoughts and kind of summarize, you know, where what, what the big takeaways are from your perspective. >>So I think the big takeaway is that influx data is doing a lot of really exciting things with Influx DB IOx and I really encourage, if you are interested in learning more about the technologies that Influx is leveraging to produce IOCs, the challenges associated with it and all of the hard work questions and you just wanna learn more, then I would encourage you to go to the monthly Tech talks and community office hours and they are on every second Wednesday of the month at 8:30 AM Pacific time. There's also a community forums and a community Slack channel look for the influx DDB unders IAC channel specifically to learn more about how to join those office hours and those monthly tech tech talks as well as ask any questions they have about iacs, what to expect and what you'd like to learn more about. I as a developer advocate, I wanna answer your questions. So if there's a particular technology or stack that you wanna dive deeper into and want more explanation about how INFLUX DB leverages it to build IOCs, I will be really excited to produce content on that topic for you. >>Yeah, that's awesome. You guys have a really rich community, collaborate with your peers, solve problems, and, and you guys super responsive, so really appreciate that. All right, thank you so much Anise for explaining all this open source stuff to the audience and why it's important to the future of data. >>Thank you. I really appreciate it. >>All right, you're very welcome. Okay, stay right there and in a moment I'll be back with Tim Yoakum, he's the director of engineering for Influx Data and we're gonna talk about how you update a SAS engine while the plane is flying at 30,000 feet. You don't wanna miss this. >>I'm really glad that we went with InfluxDB Cloud for our hosting because it has saved us a ton of time. It's helped us move faster, it's saved us money. And also InfluxDB has good support. My name's Alex Nada. I am CTO at Noble nine. Noble Nine is a platform to measure and manage service level objectives, which is a great way of measuring the reliability of your systems. You can essentially think of an slo, the product we're providing to our customers as a bunch of time series. So we need a way to store that data and the corresponding time series that are related to those. The main reason that we settled on InfluxDB as we were shopping around is that InfluxDB has a very flexible query language and as a general purpose time series database, it basically had the set of features we were looking for. >>As our platform has grown, we found InfluxDB Cloud to be a really scalable solution. We can quickly iterate on new features and functionality because Influx Cloud is entirely managed, it probably saved us at least a full additional person on our team. We also have the option of running InfluxDB Enterprise, which gives us the ability to even host off the cloud or in a private cloud if that's preferred by a customer. Influx data has been really flexible in adapting to the hosting requirements that we have. They listened to the challenges we were facing and they helped us solve it. As we've continued to grow, I'm really happy we have influx data by our side. >>Okay, we're back with Tim Yokum, who is the director of engineering at Influx Data. Tim, welcome. Good to see you. >>Good to see you. Thanks for having me. >>You're really welcome. Listen, we've been covering open source software in the cube for more than a decade, and we've kind of watched the innovation from the big data ecosystem. The cloud has been being built out on open source, mobile, social platforms, key databases, and of course influx DB and influx data has been a big consumer and contributor of open source software. So my question to you is, where have you seen the biggest bang for the buck from open source software? >>So yeah, you know, influx really, we thrive at the intersection of commercial services and open, so open source software. So OSS keeps us on the cutting edge. We benefit from OSS in delivering our own service from our core storage engine technologies to web services temping engines. Our, our team stays lean and focused because we build on proven tools. We really build on the shoulders of giants and like you've mentioned, even better, we contribute a lot back to the projects that we use as well as our own product influx db. >>You know, but I gotta ask you, Tim, because one of the challenge that that we've seen in particular, you saw this in the heyday of Hadoop, the, the innovations come so fast and furious and as a software company you gotta place bets, you gotta, you know, commit people and sometimes those bets can be risky and not pay off well, how have you managed this challenge? >>Oh, it moves fast. Yeah, that, that's a benefit though because it, the community moves so quickly that today's hot technology can be tomorrow's dinosaur. And what we, what we tend to do is, is we fail fast and fail often. We try a lot of things. You know, you look at Kubernetes for example, that ecosystem is driven by thousands of intelligent developers, engineers, builders, they're adding value every day. So we have to really keep up with that. And as the stack changes, we, we try different technologies, we try different methods, and at the end of the day, we come up with a better platform as a result of just the constant change in the environment. It is a challenge for us, but it's, it's something that we just do every day. >>So we have a survey partner down in New York City called Enterprise Technology Research etr, and they do these quarterly surveys of about 1500 CIOs, IT practitioners, and they really have a good pulse on what's happening with spending. And the data shows that containers generally, but specifically Kubernetes is one of the areas that has kind of, it's been off the charts and seen the most significant adoption and velocity particularly, you know, along with cloud. But, but really Kubernetes is just, you know, still up until the right consistently even with, you know, the macro headwinds and all, all of the stuff that we're sick of talking about. But, so what are you doing with Kubernetes in the platform? >>Yeah, it, it's really central to our ability to run the product. When we first started out, we were just on AWS and, and the way we were running was, was a little bit like containers junior. Now we're running Kubernetes everywhere at aws, Azure, Google Cloud. It allows us to have a consistent experience across three different cloud providers and we can manage that in code so our developers can focus on delivering services, not trying to learn the intricacies of Amazon, Azure, and Google and figure out how to deliver services on those three clouds with all of their differences. >>Just to follow up on that, is it, no. So I presume it's sounds like there's a PAs layer there to allow you guys to have a consistent experience across clouds and out to the edge, you know, wherever is that, is that correct? >>Yeah, so we've basically built more or less platform engineering, This is the new hot phrase, you know, it, it's, Kubernetes has made a lot of things easy for us because we've built a platform that our developers can lean on and they only have to learn one way of deploying their application, managing their application. And so that, that just gets all of the underlying infrastructure out of the way and, and lets them focus on delivering influx cloud. >>Yeah, and I know I'm taking a little bit of a tangent, but is that, that, I'll call it a PAs layer if I can use that term. Is that, are there specific attributes to Influx db or is it kind of just generally off the shelf paths? You know, are there, is, is there any purpose built capability there that, that is, is value add or is it pretty much generic? >>So we really build, we, we look at things through, with a build versus buy through a, a build versus by lens. Some things we want to leverage cloud provider services, for instance, Postgres databases for metadata, perhaps we'll get that off of our plate, let someone else run that. We're going to deploy a platform that our engineers can, can deliver on that has consistency that is, is all generated from code that we can as a, as an SRE group, as an ops team, that we can manage with very few people really, and we can stamp out clusters across multiple regions and in no time. >>So how, so sometimes you build, sometimes you buy it. How do you make those decisions and and what does that mean for the, for the platform and for customers? >>Yeah, so what we're doing is, it's like everybody else will do, we're we're looking for trade offs that make sense. You know, we really want to protect our customers data. So we look for services that support our own software with the most uptime, reliability, and durability we can get. Some things are just going to be easier to have a cloud provider take care of on our behalf. We make that transparent for our own team. And of course for customers you don't even see that, but we don't want to try to reinvent the wheel, like I had mentioned with SQL data stores for metadata, perhaps let's build on top of what of these three large cloud providers have already perfected. And we can then focus on our platform engineering and we can have our developers then focus on the influx data, software, influx, cloud software. >>So take it to the customer level, what does it mean for them? What's the value that they're gonna get out of all these innovations that we've been been talking about today and what can they expect in the future? >>So first of all, people who use the OSS product are really gonna be at home on our cloud platform. You can run it on your desktop machine, on a single server, what have you, but then you want to scale up. We have some 270 terabytes of data across, over 4 billion series keys that people have stored. So there's a proven ability to scale now in terms of the open source, open source software and how we've developed the platform. You're getting highly available high cardinality time series platform. We manage it and, and really as, as I mentioned earlier, we can keep up with the state of the art. We keep reinventing, we keep deploying things in real time. We deploy to our platform every day repeatedly all the time. And it's that continuous deployment that allows us to continue testing things in flight, rolling things out that change new features, better ways of doing deployments, safer ways of doing deployments. >>All of that happens behind the scenes. And like we had mentioned earlier, Kubernetes, I mean that, that allows us to get that done. We couldn't do it without having that platform as a, as a base layer for us to then put our software on. So we, we iterate quickly. When you're on the, the Influx cloud platform, you really are able to, to take advantage of new features immediately. We roll things out every day and as those things go into production, you have, you have the ability to, to use them. And so in the end we want you to focus on getting actual insights from your data instead of running infrastructure, you know, let, let us do that for you. So, >>And that makes sense, but so is the, is the, are the innovations that we're talking about in the evolution of Influx db, do, do you see that as sort of a natural evolution for existing customers? I, is it, I'm sure the answer is both, but is it opening up new territory for customers? Can you add some color to that? >>Yeah, it really is it, it's a little bit of both. Any engineer will say, well, it depends. So cloud native technologies are, are really the hot thing. Iot, industrial iot especially, people want to just shove tons of data out there and be able to do queries immediately and they don't wanna manage infrastructure. What we've started to see are people that use the cloud service as their, their data store backbone and then they use edge computing with R OSS product to ingest data from say, multiple production lines and downsample that data, send the rest of that data off influx cloud where the heavy processing takes place. So really us being in all the different clouds and iterating on that and being in all sorts of different regions allows for people to really get out of the, the business of man trying to manage that big data, have us take care of that. And of course as we change the platform end users benefit from that immediately. And, >>And so obviously taking away a lot of the heavy lifting for the infrastructure, would you say the same thing about security, especially as you go out to IOT and the Edge? How should we be thinking about the value that you bring from a security perspective? >>Yeah, we take, we take security super seriously. It, it's built into our dna. We do a lot of work to ensure that our platform is secure, that the data we store is, is kept private. It's of course always a concern. You see in the news all the time, companies being compromised, you know, that's something that you can have an entire team working on, which we do to make sure that the data that you have, whether it's in transit, whether it's at rest, is always kept secure, is only viewable by you. You know, you look at things like software, bill of materials, if you're running this yourself, you have to go vet all sorts of different pieces of software. And we do that, you know, as we use new tools. That's something that, that's just part of our jobs to make sure that the platform that we're running it has, has fully vetted software and, and with open source especially, that's a lot of work. And so it's, it's definitely new territory. Supply chain attacks are, are definitely happening at a higher clip than they used to, but that is, that is really just part of a day in the, the life for folks like us that are, are building platforms. >>Yeah, and that's key. I mean especially when you start getting into the, the, you know, we talk about IOT and the operations technologies, the engineers running the, that infrastructure, you know, historically, as you know, Tim, they, they would air gap everything. That's how they kept it safe. But that's not feasible anymore. Everything's >>That >>Connected now, right? And so you've gotta have a partner that is again, take away that heavy lifting to r and d so you can focus on some of the other activities. Right. Give us the, the last word and the, the key takeaways from your perspective. >>Well, you know, from my perspective I see it as, as a a two lane approach with, with influx, with Anytime series data, you know, you've got a lot of stuff that you're gonna run on-prem, what you had mentioned, air gaping. Sure there's plenty of need for that, but at the end of the day, people that don't want to run big data centers, people that want torus their data to, to a company that's, that's got a full platform set up for them that they can build on, send that data over to the cloud, the cloud is not going away. I think more hybrid approach is, is where the future lives and that's what we're prepared for. >>Tim, really appreciate you coming to the program. Great stuff. Good to see you. >>Thanks very much. Appreciate it. >>Okay, in a moment I'll be back to wrap up. Today's session, you're watching The Cube. >>Are you looking for some help getting started with InfluxDB Telegraph or Flux Check >>Out Influx DB University >>Where you can find our entire catalog of free training that will help you make the most of your time series data >>Get >>Started for free@influxdbu.com. >>We'll see you in class. >>Okay, so we heard today from three experts on time series and data, how the Influx DB platform is evolving to support new ways of analyzing large data sets very efficiently and effectively in real time. And we learned that key open source components like Apache Arrow and the Rust Programming environment Data fusion par K are being leveraged to support realtime data analytics at scale. We also learned about the contributions in importance of open source software and how the Influx DB community is evolving the platform with minimal disruption to support new workloads, new use cases, and the future of realtime data analytics. Now remember these sessions, they're all available on demand. You can go to the cube.net to find those. Don't forget to check out silicon angle.com for all the news related to things enterprise and emerging tech. And you should also check out influx data.com. There you can learn about the company's products. You'll find developer resources like free courses. You could join the developer community and work with your peers to learn and solve problems. And there are plenty of other resources around use cases and customer stories on the website. This is Dave Valante. Thank you for watching Evolving Influx DB into the smart data platform, made possible by influx data and brought to you by the Cube, your leader in enterprise and emerging tech coverage.

>> Okay, so we heard today from three experts on time series and data, how the InfluxDB platform is evolving to support new ways of analyzing large data sets very efficiently and effectively in realtime. And we learned that key open source components like Apache Arrow and the Rust Programming environment DataFusion parquet are being leveraged to support realtime data analytics at scale. We also learned about the contributions and importance of open source software and how the InfluxDB community is evolving the platform with minimal disruption to support new workloads, new use cases in the future of realtime data analytics. Now remember these sessions, they're all available on demand. You can go to thecube.net to find those. Don't forget to check out siliconangle.com for all the news related to things enterprise and emerging tech. And you should also check out influxdata.com. There you can learn about the company's products, you'll find developer resources like free courses, you can join the developer community and work with your peers to learn and solve problems, and there are plenty of other resources around use cases and customer stories on the website. This is Dave Vellante. Thank you for watching Evolving InfluxDB into the Smart Data Platform, made possible by InfluxData and brought to you by theCUBE, your leader in enterprise and emerging tech coverage.

(upbeat music) >> Okay, we're back. I'm Dave Vellante with The Cube and you're watching Evolving InfluxDB into the smart data platform made possible by influx data. Anais Dotis-Georgiou is here. She's a developer advocate for influx data and we're going to dig into the rationale and value contribution behind several open source technologies that InfluxDB is leveraging to increase the granularity of time series analysis and bring the world of data into realtime analytics. Anais welcome to the program. Thanks for coming on. >> Hi, thank you so much. It's a pleasure to be here. >> Oh, you're very welcome. Okay, so IOx is being touted as this next gen open source core for InfluxDB. And my understanding is that it leverages in memory, of course for speed. It's a kilometer store, so it gives you compression efficiency it's going to give you faster query speeds, it's going to see you store files and object storages so you got very cost effective approach. Are these the salient points on the platform? I know there are probably dozens of other features but what are the high level value points that people should understand? >> Sure, that's a great question. So some of the main requirements that IOx is trying to achieve and some of the most impressive ones to me the first one is that it aims to have no limits on cardinality and also allow you to write any kind of event data that you want whether that's lift tag or a field. It also wants to deliver the best in class performance on analytics queries. In addition to our already well served metric queries we also want to have operator control over memory usage. So you should be able to define how much memory is used for buffering caching and query processing. Some other really important parts is the ability to have bulk data export and import, super useful. Also, broader ecosystem compatibility where possible we aim to use and embrace emerging standards in the data analytics ecosystem and have compatibility with things like SQL, Python and maybe even Pandas in the future. >> Okay, so a lot there. Now we talked to Brian about how you're using Rust and which is not a new programming language and of course we had some drama around Rust during the pandemic with the Mozilla layoffs but the formation of the Rust Foundation really addressed any of those concerns and you got big guns like Amazon and Google and Microsoft throwing their collective weights behind it. It's really adoption is really starting to get steep on the S-curve. So lots of platforms, lots of adoption with Rust but why Rust as an alternative to say C++ for example? >> Sure, that's a great question. So Rust was chosen because of his exceptional performance and reliability. So while Rust is syntactically similar to C++ and it has similar performance it also compiles to a native code like C++ But unlike C++ it also has much better memory safety. So memory safety is protection against bugs or security vulnerabilities that lead to excessive memory usage or memory leaks. And Rust achieves this memory safety due to its like innovative type system. Additionally, it doesn't allow for dangling pointers and dangling pointers are the main classes of errors that lead to exploitable security vulnerabilities in languages like C++. So Rust like helps meet that requirement of having no limits on cardinality, for example, because it's we're also using the Rust implementation of Apache Arrow and this control over memory and also Rust's packaging system called Crates IO offers everything that you need out of the box to have features like async and await to fix race conditions to protect against buffering overflows and to ensure thread safe async caching structures as well. So essentially it's just like has all the control all the fine grain control, you need to take advantage of memory and all your resources as well as possible so that you can handle those really, really high cardinality use cases. >> Yeah, and the more I learn about the new engine and the platform IOx et cetera, you see things like the old days not even to even today you do a lot of garbage collection in these systems and there's an inverse, impact relative to performance. So it looks like you're really, the community is modernizing the platform but I want to talk about Apache Arrow for a moment. It's designed to address the constraints that are associated with analyzing large data sets. We know that, but please explain why, what is Arrow and what does it bring to InfluxDB? >> Sure. Yeah. So Arrow is a a framework for defining in memory column data. And so much of the efficiency and performance of IOx comes from taking advantage of column data structures. And I will, if you don't mind, take a moment to kind of illustrate why column data structures are so valuable. Let's pretend that we are gathering field data about the temperature in our room and also maybe the temperature of our store. And in our table we have those two temperature values as well as maybe a measurement value, timestamp value maybe some other tag values that describe what room and what house, et cetera we're getting this data from. And so you can picture this table where we have like two rows with the two temperature values for both our room and the store. Well, usually our room temperature is regulated so those values don't change very often. So when you have calm oriented storage essentially you take each row each column and group it together. And so if that's the case and you're just taking temperature values from the room and a lot of those temperature values are the same then you'll, you might be able to imagine how equal values will then enable each other and when they neighbor each other in the storage format this provides a really perfect opportunity for cheap compression. And then this cheap compression enables high cardinality use cases. It also enables for faster scan rates. So if you want to define like the min and max value of the temperature in the room across a thousand different points you only have to get those a thousand different points in order to answer that question and you have those immediately available to you. But let's contrast this with a row oriented storage solution instead so that we can understand better the benefits of column oriented storage. So if you had a row oriented storage, you'd first have to look at every field like the temperature in the room and the temperature of the store. You'd have to go across every tag value that maybe describes where the room is located or what model the store is. And every timestamp you then have to pluck out that one temperature value that you want at that one time stamp and do that for every single row. So you're scanning across a ton more data and that's why row oriented doesn't provide the same efficiency as column and Apache Arrow is in memory column data column data fit framework. So that's where a lot of the advantages come from. >> Okay. So you've basically described like a traditional database a row approach, but I've seen like a lot of traditional databases say, okay, now we've got we can handle Column format versus what you're talking about is really kind of native is it not as effective as the former not as effective because it's largely a bolt on? Can you like elucidate on that front? >> Yeah, it's not as effective because you have more expensive compression and because you can't scan across the values as quickly. And so those are, that's pretty much the main reasons why row oriented storage isn't as efficient as column oriented storage. >> Yeah. Got it. So let's talk about Arrow data fusion. What is data fusion? I know it's written in Rust but what does it bring to to the table here? >> Sure. So it's an extensible query execution framework and it uses Arrow as its in memory format. So the way that it helps InfluxDB IOx is that okay it's great if you can write unlimited amount of cardinality into InfluxDB, but if you don't have a query engine that can successfully query that data then I don't know how much value it is for you. So data fusion helps enable the query process and transformation of that data. It also has a Pandas API so that you could take advantage of Pandas data frames as well and all of the machine learning tools associated with Pandas. >> Okay. You're also leveraging Par-K in the platform course. We heard a lot about Par-K in the middle of the last decade cuz as a storage format to improve on Hadoop column stores. What are you doing with Par-K and why is it important? >> Sure. So Par-K is the column oriented durable file format. So it's important because it'll enable bulk import and bulk export. It has compatibility with Python and Pandas so it supports a broader ecosystem. Par-K files also take very little disc space and they're faster to scan because again they're column oriented, in particular I think Par-K files are like 16 times cheaper than CSV files, just as kind of a point of reference. And so that's essentially a lot of the benefits of Par-K. >> Got it. Very popular. So and these, what exactly is Influx data focusing on as a committer to these projects? What is your focus? What's the value that you're bringing to the community? >> Sure. So InfluxDB first has contributed a lot of different things to the Apache ecosystem. For example, they contribute an implementation of Apache Arrow and go and that will support clearing Influx. Also, there has been a quite a few contributions to data fusion for things like memory optimization and supportive additional SQL features like support for timestamp, arithmetic and support for exist clauses and support for memory control. So yeah, Influx has contributed a lot to the Apache ecosystem and continues to do so. And I think kind of the idea here is that if you can improve these upstream projects and then the long term strategy here is that the more you contribute and build those up then the more you will perpetuate that cycle of improvement and the more we will invest in our own project as well. So it's just that kind of symbiotic relationship and appreciation of the open source community. >> Yeah. Got it. You got that virtuous cycle going people call it the flywheel. Give us your last thoughts and kind of summarize, what the big takeaways are from your perspective. >> So I think the big takeaway is that, Influx data is doing a lot of really exciting things with InfluxDB IOx and I really encourage if you are interested in learning more about the technologies that Influx is leveraging to produce IOx the challenges associated with it and all of the hard work questions and I just want to learn more then I would encourage you to go to the monthly Tech talks and community office hours and they are on every second Wednesday of the month at 8:30 AM Pacific time. There's also a community forums and a community Slack channel. Look for the InfluxDB underscore IOx channel specifically to learn more about how to join those office hours and those monthly tech talks as well as ask any questions they have about IOx what to expect and what you'd like to learn more about. I as a developer advocate, I want to answer your questions. So if there's a particular technology or stack that you want to dive deeper into and want more explanation about how InfluxDB leverages it to build IOx, I will be really excited to produce content on that topic for you. >> Yeah, that's awesome. You guys have a really rich community collaborate with your peers, solve problems and you guys super responsive, so really appreciate that. All right, thank you so much Anais for explaining all this open source stuff to the audience and why it's important to the future of data. >> Thank you. I really appreciate it. >> All right, you're very welcome. Okay, stay right there and in a moment I'll be back with Tim Yoakam. He's the director of engineering for Influx Data and we're going to talk about how you update a SaaS engine while the plane is flying at 30,000 feet. You don't want to miss this. (upbeat music)

(soft upbeat music) >> Okay, we're kicking things off with Brian Gilmore. He's the director of IoT, an emerging technology at InfluxData. Brian, welcome to the program. Thanks for coming on. >> Thanks, Dave, great to be here. I appreciate the time. >> Hey, explain why InfluxDB, you know, needs a new engine. Was there something wrong with the current engine? What's going on there? >> No, no, not at all. I mean, I think, for us it's been about staying ahead of the market. I think, you know, if we think about what our customers are coming to us sort of with now, you know, related to requests like SQL query support, things like that, we have to figure out a way to execute those for them in a way that will scale long term. And then we also want to make sure we're innovating, we're sort of staying ahead of the market as well, and sort of anticipating those future needs. So, you know, this is really a transparent change for our customers. I mean, I think we'll be adding new capabilities over time that sort of leverage this new engine. But, you know, initially, the customers who are using us are going to see just great improvements in performance, you know, especially those that are working at the top end of the workload scale, you know, the massive data volumes and things like that. >> Yeah, and we're going to get into that today and the architecture and the like. But what was the catalyst for the enhancements? I mean, when and how did this all come about? >> Well, I mean, like three years ago, we were primarily on premises, right? I mean, I think we had our open source, we had an enterprise product. And sort of shifting that technology, especially the open source code base to a service basis where we were hosting it through, you know, multiple cloud providers. That was a long journey. (chuckles) I guess, you know, phase one was, we wanted to host enterprise for our customers, so we sort of created a service that we just managed and ran our enterprise product for them. You know, phase two of this cloud effort was to optimize for like multi-tenant, multi-cloud, be able to host it in a truly like SAS manner where we could use, you know, some type of customer activity or consumption as the pricing vector. And that was sort of the birth of the real first InfluxDB cloud, you know, which has been really successful. We've seen, I think, like 60,000 people sign up. And we've got tons and tons of both enterprises as well as like new companies, developers, and of course a lot of home hobbyists and enthusiasts who are using out on a daily basis. And having that sort of big pool of very diverse and varied customers to chat with as they're using the product, as they're giving us feedback, et cetera, has, you know, pointed us in a really good direction in terms of making sure we're continuously improving that, and then also making these big leaps as we're doing with this new engine. >> All right, so you've called it a transparent change for customers, so I'm presuming it's non-disruptive, but I really want to understand how much of a pivot this is, and what does it take to make that shift from, you know, time series specialist to real time analytics and being able to support both? >> Yeah, I mean, it's much more of an evolution, I think, than like a shift or a pivot. Time series data is always going to be fundamental in sort of the basis of the solutions that we offer our customers, and then also the ones that they're building on the sort of raw APIs of our platform themselves. The time series market is one that we've worked diligently to lead. I mean, I think when it comes to like metrics, especially like sensor data and app and infrastructure metrics. If we're being honest though, I think our user base is well aware that the way we were architected was much more towards those sort of like backwards-looking historical type analytics, which are key for troubleshooting and making sure you don't, you know, run into the same problem twice. But, you know, we had to ask ourselves like, what can we do to like better handle those queries from a performance and a time to response on the queries, and can we get that to the point where the result sets are coming back so quickly from the time of query that we can like, limit that window down to minutes and then seconds? And now with this new engine, we're really starting to talk about a query window that could be like returning results in, you know, milliseconds of time since it hit the ingest queue. And that's really getting to the point where, as your data is available, you can use it and you can query it, you can visualize it, you can do all those sort of magical things with it. And I think getting all of that to a place where we're saying like, yes to the customer on, you know, all of the real time queries, the multiple language query support. But, you know, it was hard, but we're now at a spot where we can start introducing that to, you know, a limited number of customers, strategic customers and strategic availabilities zones to start, but, you know, everybody over time. >> So you're basically going from what happened to, and you can still do that, obviously, but to what's happening now in the moment? >> Yeah. Yeah. I mean, if you think about time, it's always sort of past, right? I mean, like in the moment right now, whether you're talking about like a millisecond ago or a minute ago, you know, that's pretty much right now, I think for most people, especially in these use cases where you have other sort of components of latency induced by the underlying data collection, the architecture, the infrastructure, the devices, and you know, the sort of highly distributed nature of all of this. So, yeah, I mean, getting a customer or a user to be able to use the data as soon as it is available, is what we're after here. I always thought of real time as before you lose the customer, but now in this context, maybe it's before the machine blows up. >> Yeah, I mean, it is operationally, or operational real time is different. And that's one of the things that really triggered us to know that we were heading in the right direction is just how many sort of operational customers we have, you know, everything from like aerospace and defense. We've got companies monitoring satellites. We've got tons of industrial users using us as a process historian on the plant floor. And if we can satisfy their sort of demands for like real time historical perspective, that's awesome. I think what we're going to do here is we're going to start to like edge into the real time that they're used to in terms of, you know, the millisecond response times that they expect of their control systems, certainly not their historians and databases. >> Is this available, these innovations to InfluxDB cloud customers, only who can access this capability? >> Yeah, I mean, commercially and today, yes. I think we want to emphasize that for now our goal is to get our latest and greatest and our best to everybody over time of course. You know, one of the things we had to do here was like we doubled down on sort of our commitment to open source and availability. So, like, anybody today can take a look at the libraries on our GitHub and can inspect it and even can try to implement or execute some of it themselves in their own infrastructure. We are committed to bringing our sort of latest and greatest to our cloud customers first for a couple of reasons. Number one, you know, there are big workloads and they have high expectations of us. I think number two, it also gives us the opportunity to monitor a little bit more closely how it's working, how they're using it, like how the system itself is performing. And so just, you know, being careful, maybe a little cautious in terms of how big we go with this right away. Just sort of both limits, you know, the risk of any issues that can come with new software roll outs, we haven't seen anything so far. But also it does give us the opportunity to have like meaningful conversations with a small group of users who are using the products. But once we get through that and they give us two thumbs up on it, it'll be like, open the gates and let everybody in. It's going to be exciting time for the whole ecosystem. >> Yeah, that makes a lot of sense. And you can do some experimentation and, you know, using the cloud resources. Let's dig into some of the architectural and technical innovations that are going to help deliver on this vision. What should we know there? >> Well, I mean, I think, foundationally, we built the new core on Rust. This is a new very sort of popular systems language. It's extremely efficient, but it's also built for speed and memory safety, which goes back to that us being able to like deliver it in a way that is, you know, something we can inspect very closely, but then also rely on the fact that it's going to behave well, and if it does find error conditions. I mean, we've loved working with Go, and a lot of our libraries will continue to be sort of implemented in Go, but when it came to this particular new engine, that power performance and stability of Rust was critical. On top of that, like, we've also integrated Apache Arrow and Apache Parquet for persistence. I think, for anybody who's really familiar with the nuts and bolts of our backend and our TSI and our time series merge trees, this is a big break from that. You know, Arrow on the sort of in mem side and then Parquet in the on disk side. It allows us to present, you know, a unified set of APIs for those really fast real time queries that we talked about, as well as for very large, you know, historical sort of bulk data archives in that Parquet format, which is also cool because there's an entire ecosystem sort of popping up around Parquet in terms of the machine learning community. And getting that all to work, we had to glue it together with Arrow Flight. That's sort of what we're using as our RPC component. It handles the orchestration and the transportation of the columnar data now, we're moving to like a true columnar database model for this version of the engine. You know, and it removes a lot of overhead for us in terms of having to manage all that serialization, the deserialization, and, you know, to that again, like, blurring that line between real time and historical data, it's highly optimized for both streaming micro batch and then batches, but true streaming as well. >> Yeah, again, I mean, it's funny. You mentioned Rust. It's been around for a long time but it's popularity is, you know, really starting to hit that steep part of the S-curve. And we're going to dig into more of that, but give us, is there anything else that we should know about, Brian? Give us the last word. >> Well, I mean, I think first, I'd like everybody sort of watching, just to like, take a look at what we're offering in terms of early access in beta programs. I mean, if you want to participate or if you want to work sort of in terms of early access with the new engine, please reach out to the team. I'm sure, you know, there's a lot of communications going out and it'll be highly featured on our website. But reach out to the team. Believe it or not, like we have a lot more going on than just the new engine. And so there are also other programs, things we're offering to customers in terms of the user interface, data collection and things like that. And, you know, if you're a customer of ours and you have a sales team, a commercial team that you work with, you can reach out to them and see what you can get access to, because we can flip a lot of stuff on, especially in cloud through feature flags. But if there's something new that you want to try out, we'd just love to hear from you. And then, you know, our goal would be, that as we give you access to all of these new cool features that, you know, you would give us continuous feedback on these products and services, not only like what you need today, but then what you'll need tomorrow to sort of build the next versions of your business. Because, you know, the whole database, the ecosystem as it expands out into this vertically-oriented stack of cloud services, and enterprise databases, and edge databases, you know, it's going to be what we all make it together, not just those of us who are employed by InfluxDB. And then finally, I would just say, please, like, watch and Anais' and Tim's sessions. Like, these are two of our best and brightest. They're totally brilliant, completely pragmatic, and they are most of all customer-obsessed, which is amazing. And there's no better takes, like honestly, on the sort of technical details of this than theirs, especially when it comes to the value that these investments will bring to our customers and our communities. So, encourage you to, you know, pay more attention to them than you did to me, for sure. >> Brian Gilmore, great stuff. Really appreciate your time. Thank you. >> Yeah, thanks David, it was awesome. Looking forward to it. >> Yeah, me too. I'm looking forward to see how the community actually applies these new innovations and goes beyond just the historical into the real time. Really hot area. As Brian said, in a moment, I'll be right back with Anais Dotis-Georgiou to dig into the critical aspects of key open source components of the InfluxDB engine, including Rust, Arrow, Parquet, Data Fusion. Keep it right there. You don't want to miss this. (soft upbeat music)

>> Narrator: The cube presents, the Kubecon and Cloudnativecon Europe, 2022 brought to you by Red Hat, the cloud native computing foundation and its ecosystem partners. >> Welcome to Valencia, Spain in Kubecon and Cloudnativecon Europe, 2022. I'm your host Keith Townsend. It's been a amazing day, three days of coverage 7,500 people, 170 sponsors, a good mix of end user organizations, vendors, just people with open source at large. I've loved the conversations. We're not going to stop that coverage just because this is the last session of the conference. Colin Murphy, senior software engineer, Adobe, >> Adobe. >> Oh, wow. This is going to be fun. And then Liam Randall, the chair of CNCF Cloud Native WebAssembly Day. >> That's correct. >> And CNCF & CEO of Cosmonic. >> That's right. >> All right. First off, let's talk about the show. How has this been different than other, if at all of other Kubecons? >> Well, first I think we all have to do a tremendous round of applause, not only for the vendors, but the CNC staff and all the attendees for coming out. And you have to say, Kubecon is back. The online experiences have been awesome but this was the first one, where Hallwaycon was in full effect. And you had the opportunity to sit down and meet with so many intelligent and inspiring peers and really have a chance to learn about all the exciting innovations that have happened over the last year. >> Colin. >> Yeah, it's been my most enjoyable Kubecon I've ever been to. And I've been to a bunch of them over the last few years. Just the quality of people. The problems that we're solving right now, everywhere from this newer stuff that we're talking about today with WebAssembly but then all these big enterprises trying to getting involved in Kubernetes >> Colin, to your point about the problems that we're solving, in many ways the pandemic has dramatically accelerated the pace of innovation, especially inside the CNCF, which is by far the most critical repository of open source projects that enterprises, governments and individuals rely on around the world, in order to deliver new experiences and to have coped and scaled out within the pandemic over the last few years. >> Yeah, I'm getting this feel, this vibe of the overall show that feels like we're on the cuff for something. There's other shows throughout the year, that's more vendor focused that talk about cloud native. But I think this is going to be the industry conference where we're just getting together and talking about it and it's going to probably be, in the next couple of years, the biggest conference of the year, that's just my personal opinion. >> I actually really strongly agree with you. And I think that the reason for that is the diversity that we get from the open source focus of Kubecon Kubecon has started where the industry really started which was in shared community projects. And I was the executive at Capital One that led the donation of cloud custodian into the CNCF. And I've started and put many projects here. And one of the reasons that you do that is so that you can build real scalable communities, Vendors that oftentimes even have competing interest but it gives us a place where we can truly collaborate where we can set aside our personal agendas and our company's agendas. And we can focus on the problems at hand. And how do we really raise the bar for technology for everybody. >> Now you two are representing a project that, you know as we look at kind of, how the web has evolved the past few decades, there's standards, there's things that we know that work, there's things that we know that don't work and we're beyond cloud native, we're kind of resistant to change. Funny enough. >> That's right. >> So WebAssembly, talk to me about what problem is WebAssembly solving that need solving? >> I think it's fitting that here on the last day of Kubecon, we're starting with the newest standard for the web and for background, there's only four languages that make up what we think of as the modern web. There's JavaScript, there's HTML, there's CSS, and now there's a new idea that's WebAssembly. And it's maybe not a new idea but it's certainly a new standard, that's got massive adoption and acceleration. WebAssembly is best thought of as almost like a portable little virtual machine. And like a lot of great ideas like JavaScript, it was originally designed to bring new experiences to browsers everywhere. And as organizations looked at the portability and security value props that come from this tiny little virtual machine, it's made a wonderful addition to backend servers and as a platform for portability to bring solutions all the way out to the edge. >> So what are some of the business cases for WebAssembly? Like what problem, what business problem are we solving? >> So it, you know, we would not have been able to bring Photoshop to the web without WASM. >> Wow. >> And just to be clear, I had nothing to do with that effort. So I want to make sure everybody understands, but if you have a lot of C++ or C code and you want to bring that experience to the web browser which is a great cost savings, cause it's running on the client's machines, really low latency, high performance experiences in the browser, WASM, really the only way to go. >> So I'm getting hints of fruit berry, Java. >> Liam: Yeah, absolutely. >> Colin: Definitely. >> You know, the look, WebAssembly sounds similar to promises you've heard before, right ones, run anywhere. The difference is, is that WebAssembly is not driven by any one particular vendor. So there's no one vendor that's trying to bring a plug in to every single device. WebAssembly was a recognition, much like Kubecon, the point that we started with around the diversity of thought ideas and representation of shared interest, of how do we have a platform that's polyglot? Many people can bring languages to it, and solutions that we can share and then build from there. And it is unlocking some of the most amazing and innovative experiences, both on the web backend servers and all the way to the edge. Because WebAssembly is a tiny little virtual machine that runs everywhere. Adobe's leadership is absolutely incredible with the things that they're doing with WebAssembly. They did this awesome blog post with the Google Chrome team that talked about other performance improvements that were brought into Chrome and other browsers, in order to enable that kind of experience. >> So I get the general concept of WebAssembly and it's one of those things that I have to ask the question, and I appreciate that Adobe uses it but without the community, I mean, I've dedicated some of my team's resources over the years to some really cool projects and products that just died on the buying cause there was no community around. >> Yeah. >> Who else uses WebAssembly? >> Yeah, I think so. We actually, inside the CNCF now, have an entire day devoted just to WebAssembly and as the co-chair of the CNCF Cloud Native WebAssembly Day, we really focus on bringing those case studies to the forefront. So some of the more interesting talks that we had here and at some of the precursor weekend conferences were from BMW, for example, they talked about how they were excited about not only WebAssembly, but a framework that they use on WebAssembly called WASM cloud, that lets them a flexibly scale machine learning models from their own edge, in their own vehicles through to their developer's workstations and even take that data onto their regular cloud Kubernetes and scale analysis and analytics. They invested and they just released a machine learning framework for one of the many great WebAssembly projects called WASM cloud, which is a CNCF project, a member project here in the CNCF. >> So how does that fit in overall landscape? >> So think of WebAssembly, like you think of HTML. It's a technology that gives you a lot of concept and to accelerate your journey on those technologies, people create frameworks. For example, if you were going to write a UI, you would not very likely start with an empty document you'd start with a react or view. And in a similar vein, if you were going to start a new microservice or backend application, project for WebAssembly, you might use WASM cloud or you might use ATMO or you might use a Spin. Those are three different types of projects. They all have their own different value props and their own different opinions that they bring to them. But the point is is that this is a quickly evolving space and it's going to dramatically change the type of experiences that we bring, not only to web browsers but to servers and edges everywhere. >> So Colin, you mentioned C+ >> Colin: Yeah. >> And other coding. Well , talk to me about the ramp up. >> Oh, well, so, yeah, so, C++ there was a lot of work done in scripting, at Adobe. Taking our C++ code and bringing it into the browser. A lot of new instructions, Cimdi, that were brought to make a really powerful experience, but what's new now is the server side aspect of things. So, just what kind of, what Liam was talking about. Now we can run this stuff in the data center. It's not just for people's browsers anymore. And then we can also bring it out to the edge too, which is a new space that we can take advantage of really almost only through WebAssembly and some JavaScript. >> So wait, let me get this kind of under hook. Before, if I wanted a rich experience, I have to run a heavy VDI instance on the back end so that I'm basically getting remote desktop calls from a light thin client back to my backend server, that's heavy. >> That is heavy. >> WebAssembly is alternative to that? >> Yes, absolutely. Think of WebAssembly as a tiny little CPU that is a shim, that we can take the places that don't even traditionally have a concept of a processor. So inside the browser, for example, traditionally cloud native development on the backend has been dominated by things like Docker and Docker is a wonderful technology and Container is a wonderful technology that really drove the last 10 years of cloud native with the great lift and shift, if you will. Take our existing applications, package them up in this virtual desktop and then deliver them. But to deliver the next 10 years of experiences, we need solutions that let us have portability first and a security model that's portable across the entire landscape. So this isn't just browsers and servers on the back end, WebAssembly creates an a layer of equality from truly edge to edge. It's can transcend different CPUs, different operating systems. So where containers have this lower bound off you need to be running Linux and you need to be in a place where you're going to bring Kubernetes. WebAssembly is so small and portable, it transcends that lower bound. It can go to places like iOS. It can go to places like web browsers. It can even go to teeny tiny CPUs that don't even traditionally have a full on operating systems inside them. >> Colin: Right, places where you can't run Docker. >> So as I think about that, and I'm a developer and I'm running my back end and I'm running whatever web stack that I want, how does this work? Like, how do I get started with it? >> Well, there's some great stuff Liam already mentioned with WASM cloud and Frmion Spin. Microsoft is heavily involved now on providing cloud products that can take advantage of WebAssembly. So we've got a lot of languages, new languages coming in.net and Ruby, Rust is a big one, TinyGo, really just a lot of places to get involved. A lot of places to get started. >> At the highest level Finton Ryan, when he was at Gartner, he's a really well known analyst. He wrote something profound a few years ago. He said, WebAssembly is the one technology, You don't need a strategy to adopt. >> Mm. >> Because frankly you're already using it because there's so many wonderful experiences and products that are out there, like what Adobe's doing. This virtual CPU is not just a platform to run on cloud native and to build applications towards the edge. You can embed this virtual CPU inside of applications. So cases where you would want to allow your users to customize an application or to extend functionality. Give you an example, Shopify is a big believer in WebAssembly because while their platform covers, two standard deviations or 80% of the use cases, they have a wonderful marketplace of extensions that folks can use in order to customize the checkout process or apply specialized discounts or integrate into a partner ecosystem. So when you think about the requirements for those scenarios, they line up to the same requirements that we have in browsers and servers. I want real security. I want portability. I want reuseability. And ultimately I want to save money and go faster. So organizations everywhere should take a few minutes and do a heads up and think about one, where WebAssembly is already in their environment, inside of places like Envoy and Istio, some of the most popular projects in the cloud native ecosystem, outside of Kubernetes. And they should perhaps consider studying, how WebAssembly can help them to transform the experiences that they're delivering for their customers. This may be the last day of Kubecon, but this is certainly not the last time we're going to be talking about WebAssembly, I'll tell you that. >> So, last question, we've talked a lot about how to get started. How about day two, when I'm thinking about performance troubleshooting and ensuring clients have a great experience what's day two operation like? >> That's a really good question. So there's, I know that each language kind of brings their own tool chain and their, and you know we saw some great stuff on, on WASM day. You can look it up around the .net experience for debugging, They really tried to make it as seamless and the same as it was for native code. So, yeah, I think that's a great question. I mean, right now it's still trying to figure out server side, It's still, as Liam said, a shifting landscape. But we've got some great stuff out here already >> You know, I'd make an even bigger call than that. When I think about the last 20 years as computing has evolved, we've continued to move through these epics of tech that were dominated by a key abstraction. Think about the rise of virtualization with VMware and the transition to the cloud. The rise of containerization, we virtualized to OS. The rise of Kubernetes and CNCF itself, where we virtualize cloud APIs. I firmly believe that WebAssembly represents the next epic of tech. So I think that day two WebAssembly continues to become one of the dominant themes, not only across cloud native but across the entire technical computing landscape. And it represents a fundamentally gigantic opportunity for organizations such as Adobe, that are always market leading and at the cutting edge of tech, to bring new experiences to their customers and for vendors to bring new platforms and tools to companies that want to execute on that opportunity. >> Colin Murphy, Liam Randall, I want to thank you for joining the Cube at Kubecon Cloudnativecon 2022. I'm now having a JavaScript based app that I want to re-look at, and maybe re-platforming that to WebAssembly. It's some lot of good stuff there. We want to thank you for tuning in to our coverage of Kubecon Cloudnativecon. And we want to thank the organization for hosting us, here from Valencia, Spain. I'm Keith Townsend, and you're watching the Cube, the leader in high tech coverage. (bright music)

(gentle upbeat music) >> Hello, and welcome to theCUBE presentation of the AWS Startup Showcase Open Cloud Innovations. This is season two episode one of an ongoing series covering setting status from the AWS ecosystem. Talking about innovation, here it's open source for this theme. We do this every episode, we pick a theme and have a lot of fun talking to the leaders in the industry and the hottest startups. I'm your host John Furrier here with Lisa Martin in our Palo Alto studios. Lisa great series, great to see you again. >> Good to see you too. Great series, always such spirited conversations with very empowered and enlightened individuals. >> I love the episodic nature of these events, we get more stories out there than ever before. They're the hottest startups in the AWS ecosystem, which is dominating the cloud sector. And there's a lot of them really changing the game on cloud native and the enablement, the stories that are coming out here are pretty compelling, not just from startups they're actually penetrating the enterprise and the buyers are changing their architectures, and it's just really fun to catch the wave here. >> They are, and one of the things too about the open source community is these companies embracing that and how that's opening up their entry to your point into the enterprise. I was talking with several customers, companies who were talking about the 70% of their pipeline comes from the open source community. That's using the premium version of the technology. So, it's really been a very smart, strategic way into the enterprise. >> Yeah, and I love the format too. We get the keynote we're doing now, opening keynote, some great guests. We have Sir John on from AWS started program, he is the global startups lead. We got Swami coming on and then closing keynote with Deepak Singh. Who's really grown in the Amazon organization from containers now, compute services, which now span how modern applications are being built. And I think the big trend that we're seeing that these startups are riding on that big wave is cloud natives driving the modern architecture for software development, not just startups, but existing, large ISV and software companies are rearchitecting and the customers who buy their products and services in the cloud are rearchitecting too. So, it's a whole new growth wave coming in, the modern era of cloud some say, and it's exciting a small startup could be the next big name tomorrow. >> One of the things that kind of was a theme throughout the conversations that I had with these different guests was from a modern application security perspective is, security is key, but it's not just about shifting lab. It's about doing so empowering the developers. They don't have to be security experts. They need to have a developer brain and a security heart, and how those two organizations within companies can work better together, more collaboratively, but ultimately empowering those developers, which goes a long way. >> Well, for the folks who are watching this, the format is very simple. We have a keynote, editorial keynote speakers come in, and then we're going to have a bunch of companies who are going to present their story and their showcase. We've interviewed them, myself, you Dave Vallante and Dave Nicholson from theCUBE team. They're going to tell their stories and between the companies and the AWS heroes, 14 companies are represented and some of them new business models and Deepak Singh who leads the AWS team, he's going to have the closing keynote. He talks about the new changing business model in open source, not just the tech, which has a lot of tech, but how companies are being started around the new business models around open source. It's really, really amazing. >> I bet, and does he see any specific verticals that are taking off? >> Well, he's seeing the contribution from big companies like AWS and the Facebook's of the world and large companies, Netflix, Intuit, all contributing content to the open source and then startups forming around them. So Netflix does some great work. They donated to open source and next thing you know a small group of people get together entrepreneurs, they form a company and they create a platform around it with unification and scale. So, the cloud is enabling this new super application environment, superclouds as we call them, that's emerging and this new supercloud and super applications are scaling data-driven machine learning and AI that's the new formula for success. >> The new formula for success also has to have that velocity that developers expect, but also that the consumerization of tech has kind of driven all of us to expect things very quickly. >> Well, we're going to bring in Serge Shevchenko, AWS Global Startup program into the program. Serge is our partner. He is the leader at AWS who has been working on this program Serge, great to see you. Thanks for coming on. >> Yeah, likewise, John, thank you for having me very excited to be here. >> We've been working together on collaborating on this for over a year. Again, season two of this new innovative program, which is a combination of CUBE Media partnership, and AWS getting the stories out. And this has been a real success because there's a real hunger to discover content. And then in the marketplace, as these new solutions coming from startups are the next big thing coming. So, you're starting to see this going on. So I have to ask you, first and foremost, what's the AWS startup showcase about. Can you explain in your terms, your team's vision behind it, and why those startup focus? >> Yeah, absolutely. You know John, we curated the AWS Startup Showcase really to bring meaningful and oftentimes educational content to our customers and partners highlighting innovative solutions within these themes and ultimately to help customers find the best solutions for their use cases, which is a combination of AWS and our partners. And really from pre-seed to IPO, John, the world's most innovative startups build on AWS. From leadership downward, very intentional about cultivating vigorous AWS community and since 2019 at re:Invent at the launch of the AWS Global Startup program, we've helped hundreds of startups accelerate their growth through product development support, go to market and co-sell programs. >> So Serge question for you on the theme of today, John mentioned our showcases having themes. Today's theme is going to cover open source software. Talk to us about how Amazon thinks about opensource. >> Sure, absolutely. And I'll just touch on it briefly, but I'm very excited for the keynote at the end of today, that will be delivered by Deepak the VP of compute services at AWS. We here at Amazon believe in open source. In fact, Amazon contributes to open source in multiple ways, whether that's through directly contributing to third-party project, repos or significant code contributions to Kubernetes, Rust and other projects. And all the way down to leadership participation in organizations such as the CNCF. And supporting of dozens of ISV myself over the years, I've seen explosive growth when it comes to open source adoption. I mean, look at projects like Checkov, within 12 months of launching their open source project, they had about a million users. And another great example is Falco within, under a decade actually they've had about 37 million downloads and that's about 300% increase since it's become an incubating project in the CNCF. So, very exciting things that we're seeing here at AWS. >> So explosive growth, lot of content. What do you hope that our viewers and our guests are going to be able to get out of today? >> Yeah, great question, Lisa. I really hope that today's event will help customers understand why AWS is the best place for them to run open source, commercial and which partner solutions will help them along their journey. I think that today the lineup through the partner solutions and Deepak at the end with the ending keynote is going to present a very valuable narrative for customers and startups in selecting where and which projects to run on AWS. >> That's great stuff Serge would love to have you on and again, I want to just say really congratulate your team and we enjoy working with them. We think this showcase does a great service for the community. It's kind of open source in its own way if I can co contributing working on out there, but you're really getting the voices out at scale. We've got companies like Armory, Kubecost, Sysdig, Tidelift, Codefresh. I mean, these are some of the companies that are changing the game. We even had Patreon a customer and one of the partners sneak with security, all the big names in the startup scene. Plus AWS Deepak saying Swami is going to be on the AWS Heroes. I mean really at scale and this is really a great. So, thank you so much for participating and enabling all of this. >> No, thank you to theCUBE. You've been a great partner in this whole process, very excited for today. >> Thanks Serge really appreciate it. Lisa, what a great segment that was kicking off the event. We've got a great lineup coming up. We've got the keynote, final keynote fireside chat with Deepak Singh a big name at AWS, but Serge in the startup showcase really innovative. >> Very innovative and in a short time period, he talked about the launch of this at re:Invent 2019. They've helped hundreds of startups. We've had over 50 I think on the showcase in the last year or so John. So we really gotten to cover a lot of great customers, a lot of great stories, a lot of great content coming out of theCUBE. >> I love the openness of it. I love the scale, the storytelling. I love the collaboration, a great model, Lisa, great to work with you. We also Dave Vallante and Dave Nicholson interview. They're not here, but let's kick off the show. Let's get started with our next guest Swami. The leader at AWS Swami just got promoted to VP of the database, but also he ran machine learning and AI at AWS. He is a leader. He's the author of the original DynamoDB paper, which is celebrating its 10th year anniversary really impacted distributed computing and open source. Swami's introduced many opensource aspects of products within AWS and has been a leader in the engineering side for many, many years at AWS, from an intern to now an executive. Swami, great to see you. Thanks for coming on our AWS startup showcase. Thanks for spending the time with us. >> My pleasure, thanks again, John. Thanks for having me. >> I wanted to just, if you don't mind asking about the database market over the past 10 to 20 years cloud and application development as you see, has changed a lot. You've been involved in so many product launches over the years. Cloud and machine learning are the biggest waves happening to your point to what you're doing now. Software is under the covers it's powering it all infrastructure is code. Open source has been a big part of it and it continues to grow and change. Deepak Singh from AWS talks about the business model transformation of how like Netflix donates to the open source. Then a company starts around it and creates more growth. Machine learnings and all the open source conversations around automation as developers and builders, like software as cloud and machine learning become the key pistons in the engine. This is a big wave, what's your view on this? How how has cloud scale and data impacting the software market? >> I mean, that's a broad question. So I'm going to break it down to kind of give some of the back data. So now how we are thinking about it first, I'd say when it comes to the open source, I'll start off by saying first the longevity and by ability of open sources are very important to our customers and that is why we have been a significant contributor and supporter of these communities. I mean, there are several efforts in open source, even internally by actually open sourcing some of our key Amazon technologies like Firecracker or BottleRocket or our CDK to help advance the industry. For example, CDK itself provides some really powerful way to build and configure cloud services as well. And we also contribute to a lot of different open source projects that are existing ones, open telemetries and Linux, Java, Redis and Kubernetes, Grafana and Kafka and Robotics Operating System and Hadoop, Leucine and so forth. So, I think, I can go on and on, but even now I'd say the database and observability space say machine learning we have always started with embracing open source in a big material way. If you see, even in deep learning framework, we championed MX Linux and some of the core components and we open sourced our auto ML technology auto Glue on, and also be open sourced and collaborated with partners like Facebook Meta on Fighter showing some major components and there, and then we are open search Edge Compiler. So, I would say the number one thing is, I mean, we are actually are very, very excited to partner with broader community on problems that really mattered to the customers and actually ensure that they are able to get amazing benefit of this. >> And I see machine learning is a huge thing. If you look at how cloud group and when you had DynamoDB paper, when you wrote it, that that was the beginning of, I call the cloud surge. It was the beginning of not just being a resource versus building a data center, certainly a great alternative. Every startup did it. That's history phase one inning and a half, first half inning. Then it became a large scale. Machine learning feels like the same way now. You feel like you're seeing a lot of people using it. A lot of people are playing around with it. It's evolving. It's been around as a science, but combined with cloud scale, this is a big thing. What should people who are in the enterprise think about how should they think about machine learning? How has some of your top customers thought about machine learning as they refactor their applications? What are some of the things that you can share from your experience and journey here? >> I mean, one of the key things I'd say just to set some context on scale and numbers. More than one and a half million customers use our database analytics or ML services end-to-end. Part of which machine learning services and capabilities are easily used by more than a hundred thousand customers at a really good scale. However, I still think in Amazon, we tend to use the phrase, "It's day one in the age of internet," even though it's an, or the phrase, "Now, but it's a golden one," but I would say in the world of machine learning, yes it's day one but I also think we just woke up and we haven't even had a cup of coffee yet. That's really that early, so. And, but when you it's interesting, you've compared it to where cloud was like 10, 12 years ago. That's early days when I used to talk to engineering leaders who are running their own data center and then we talked about cloud and various disruptive technologies. I still used to get a sense about like why cloud and basic and whatnot at that time, Whereas now with machine learning though almost every CIO, CEO, all of them never asked me why machine learning. Instead, the number one question, I get is, how do I get started with it? What are the best use cases? which is great, and this is where I always tell them one of the learnings that we actually learned in Amazon. So again, a few years ago, probably seven or eight years ago, and Amazon itself realized as a company, the impact of what machine learning could do in terms of changing how we actually run our business and what it means to provide better customer experience optimize our supply chain and so far we realized that the we need to help our builders learn machine learning and the help even our business leaders understand the power of machine learning. So we did two things. One, we actually, from a bottom-up level, we built what I call as machine learning university, which is run in my team. It's literally stocked with professors and teachers who offer curriculum to builders so that they get educated on machine learning. And now from a top-down level we also, in our yearly planning process, we call it the operational planning process where we write Amazon style narratives six pages and then answer FAQ's. We asked everyone to answer one question around, like how do you plan to leverage machine learning in your business? And typically when someone says, I really don't play into our, it does not apply. It's usually it doesn't go well. So we kind of politely encourage them to do better and come back with a better answer. This kind of dynamic on top-down and bottom-up, changed the conversation and we started seeing more and more measurable growth. And these are some of the things you're starting to see more and more among our customers too. They see the business benefit, but this is where to address the talent gap. We also made machine learning university curriculum actually now open source and freely available. And we launched SageMaker Studio Lab, which is a no cost, no set up SageMaker notebook service for educating learner profiles and all the students as well. And we are excited to also announce AIMLE scholarship for underrepresented students as well. So, so much more we can do well. >> Well, congratulations on the DynamoDB paper. That's the 10 year anniversary, which is a revolutionary product, changed the game that did change the world and that a huge impact. And now as machine learning goes to the next level, the next intern out there is at school with machine learning. They're going to be writing that next paper, your advice to them real quick. >> My biggest advice is, always, I encourage all the builders to always dream big, and don't be hesitant to speak your mind as long as you have the right conviction saying you're addressing a real customer problem. So when you feel like you have an amazing solution to address a customer problem, take the time to articulate your thoughts better, and then feel free to speak up and communicate to the folks you're working with. And I'm sure any company that nurtures good talent and knows how to hire and develop the best they will be willing to listen and then you will be able to have an amazing impact in the industry. >> Swami, great to know you're CUBE alumni love our conversations from intern on the paper of DynamoDB to the technical leader at AWS and database analyst machine learning, congratulations on all your success and continue innovating on behalf of the customers and the industry. Thanks for spending the time here on theCUBE and our program, appreciate it. >> Thanks again, John. Really appreciate it. >> Okay, now let's kick off our program. That ends the keynote track here on the AWS startup showcase. Season two, episode one, enjoy the program and don't miss the closing keynote with Deepak Singh. He goes into great detail on the changing business models, all the exciting open source innovation. (gentle bright music)

(upbeat music) >> Welcome to this CUBE Conversation. This is part of the second season of the AWS startup showcase, season two, episode one. I'm Dave Nicholson, and I am joined with a very special guest, CEO and co-founder of Tidelift, Mr. Donald Fischer. Donald, welcome to the CUBE. >> Thanks David. Really glad to be here. >> So, first and foremost, tell us about Tidelift. >> Happy to, yeah, so, at Tidelift we're on a mission. Our mission is to make open source software work better for everyone, and when we say that, we mean, make it work better for all the organizations and governments and everybody that depends on open source software to build the applications that we all rely on. But also part of our mission, is making open source work better for the creators of open source. The independent open source maintainers, who are behind so many of those building blocks, technology building blocks that our commerce industry and society is comprised of these days. They've got a hard task to hold up all of that stuff and make sure that it meets, you know, professional grade standards and that we can all rely on it. And so, we want to do our part to help both sides of that equation. >> Fantastic, well, I want to double click on a few of the things that you said, but I think I want to format this by starting out with a little role play between the two of us, if you don't mind. I know you're CEO, but for the sake of this, you're going to be the CIO and I'm going to be the CEO, and we're going to play off some recent events here. So, hey Donald, come on in, sit down. Listen, I want to talk to you about this whole log shell, log for something, or another thing that's going on. So, let me get this straight. Our multinational Fortune 500 company is dependent upon software, that's free, and somehow we've been running this and the people who maintain it, do it for free, we don't pay for it, but somehow this has opened us up to a threat from people who can log into a system we're using to keep track of stuff, and then, what's going on? By the way, you're fired, but I want to know if, I want to know if you can stay on for the next 90 days to train your replacement, but, explain to me what's going on with this whole open-source nonsense? >> Yeah. Don't panic boss. Only about 70 or 80% of the software in our enterprise that is third-party open source software. So, there's definitely, like 20 or 30% that's not, and we're on top of it. Now, yeah, I think it's a, you know, you're right to say, we are completely dependent on this software, that's being created by these, you know, amazing folks on the internet. Boss, you told me that we had to have a global corporation here with modern digital customer experience. We're not going to be able to do it using Microsoft front page from 1997, and there's no other path to take than to build with modern building blocks. And today in, you know, the modern era, that means building on open source packages and technologies across a whole slew of language, ecosystems, like JavaScript and Java PHP, Ruby, Python, .NET, Rust, Go, we use all of it here, boss, and, we don't get to have a business unless we do. >> Okay, so, I didn't understand a word that you just said, but it was enough to convince me to let you keep your job. So, end-scene, we're not getting paid scale wages to do this, Donald, so I think we can go back to our normal personas. So, how does Tidelift play into all of this? I'd really want to hear about this concept of what an open source maintainer is, because these are largely volunteers, aren't they, in terms of the maintenance that they're doing? >> Yeah, so, I mean, open source, there's a lot of different models for open source software development. There certainly are a number of foundational open source projects, certainly at the infrastructure level, like operating systems, databases and things like that, that tend to be, you know, predominantly driven by vendors, software vendors, you know, like you can think of Red Hat, VMware organizations like that. But when you get up to the application development world, teams, building, you know, websites, web applications, mobile applications, most of the building blocks at that tier in these a programming language ecosystems, most of the software there is actually being created, that enterprise organizations use, is being created by individual, independent, open source maintainers, where it's not their day job, it's a side hustle for them. And it's a really interesting question, like, how did we get here? You know, why are these folks doing it? It sort of rhymes with the question I asked myself years ago, like, who's typing all this stuff into Wikipedia, and why? Like, it's amazing resource, I'm so glad it's there, but why are they doing this, right? And it turns out that there's a bunch of motivations there's some cynical motivations for the open source maintainers that people attribute that are practical too, you know, people say your GitHub repository is your resume in as a modern developer, things like that helps you get a reputation, you can use that to get a job. But, when we've talked to the maintainers of the most widely used open source packages, and by that, I mean, thousands of packages that every major organization that builds software relies on, the main reason why they do it is actually impact. We find we've actually done direct surveys of this audience and the reason why they spend their nights and weekends and carve out time, where they could be, you know, getting paid to do something else or going skiing or going to the beach, is it really feels good to have this activity that they put out into the world, and, you know, they know that folks use this stuff and rely on it, and there's a pride in their work and the impact that they're making. But the challenge with this model is that when it's only an impact and pride, and sort of a, you know, a good feeling driven effort, it means that maybe all of the things that organizations might want their standards that organizations might want their software to meet doesn't get done, right? Like it's one thing, if you've got a job as a software engineer, building corporate software, or even as a, you know, a maintainer at a corporate open source company, and you have a checklist of, you know, standard enterprise software development, commercial grade software development tasks that you need to be completing, if you're doing it as a side hustle for good reasons, like impact and, you know, releasing your creative juice, you might not get to some of the more boring aspects of commercial software engineering, like security engineering and some of the documentation and release engineering and, you know, making sure there's structured metadata around all the elements of it. And then that's the gap that we're really trying to fill at Tidelift, by connecting these two audiences. >> Yeah. How? How? You want to fill the gap, you want to connect the audiences, but, how do you do that? >> Yeah, perfect, so, we do it by paying the maintainers, paying the open source maintainers, actual dollars, or the currency of their preference, and what we're paying them for is not just to sort of hack on their projects, or hack on their projects more, we're asking them to help us ensure that the software that the organizations that we work with depend on meets certain specific concrete enterprise standards, and those standards fall into three categories, security, licensing, and maintenance. So, on the security front, you know, a baseline standard, there is making sure that we have known versions of the open source packages that are free of known defects, right? So there's like a catalog of known security defects that the industry uses called the National Vulnerability Database, you may have seen the terminology CVE referred to in passing, that's the identifier for these things. So, we work with the open-source maintainers to make sure that we've figured out, mapped out, which versions of software packages are impacted by known security vulnerabilities. And then we also look forward and make sure that we have a plan in place for what happens in the future when there are security vulnerabilities. So, you know, traditional commercial software, there's a security response team, who's kind of standing by 24/7, ready to respond, and then there's a defined protocol of what's going to happen, in terms of what's called responsible disclosure, telling the right folks in the right sequence, that there is a vulnerability causing there to be a patch version of the software available, communicating that through, you know, traditional commercial software vendors for, you know, years have been doing that internally, that doesn't exist by default for volunteer, you know, part-time open source, independent open source maintainers. So we fill that gap and we pre-wire that with them to make sure that that first track security is can be buttoned up. >> So, you're paying them, are you and your co-founders wealthy philanthropists that are just doing this, or what's the business model here? Now you're pulling these people who were doing it for free, they're happy, but how does that translate into a business model for Tidelift. >> Perfect, so, the work that they're doing, you know, I talked a little bit about security, we also do similar things on those other attributes, like licensing, making sure that the licenses are completely accurate, and we kind of know who wrote the software, et cetera, and then maintenance, is it being proactively cared for going forward? Is somebody still on the case with these projects? Now, the result of all of that work, is we create a vetted catalog of known good open source releases that we've vetted with the experts, often the individuals and teams that wrote the code in the first place, usually, we vet that it meets these enterprise standards. That's a really useful tool for organizations that are building with that. So, the way that we convey that to organizations that are building software in a useful way is we have a SAS service software, that as a service platform, that's what Tidelift is, and basically, the teams that use this stuff, they plug us into their software development process, typically alongside other tools that they might have, like CI/CD tools that are running tests on their application logic, they'll plug in Tidelift into their release process to ensure that those, the 70 or 80% of the software that they ship, that comes from GitHub, comes from the Python package index, or NPM, or the Maven Central Repository for Java, we're vetting that that meets their enterprise standards and ensuring that the ingredients, the building blocks that go into their applications are known good and vetted to these concrete standards. And they are, you know, this is an unsolved problem for almost every serious organization. There's a couple of, you know, over-performing organizations, like Google has done some amazing internal work on this, Amazon has an incredible dedicated team that does this internally for Amazon developers, very few other organizations, even some of the largest multinational companies have a dedicated internal function doing this comprehensively and systematically. Tidelift is that function that these organizations can use. They can work with us and our network, our unique network of hundreds of these independent open source maintainers, to ensure that there is a feed of known good vetted packages to go into their applications. >> So, were maintainers going in and auditing, and editing, and vetting software that was essentially created by others? That's one question, and then the other question that kind of goes along with that is, are you vetting a gold copy of something and saying, this software meets certain criteria, you should feel okay using it, that's one thing. Validating that the actual distribution, you know, the actual code that's being executed in their enterprise is secure and hasn't been tampered with is another thing. So where do you sit in that distribution channel or that supply chain? >> Sure, so, on the distribution front, you can think of us, we're sort of a GPS system that your application developers can use to know which versions of software are going to meet your enterprise standards. We don't create a separate world where we have our own, you know, side copy of the entire development ecosystem. It's not what these organizations want. They don't want to use some weird enterprise world set of open source packages, they want to just, you know, type NPM install have the, you know, software flow into their organization, but they also want it to not have no insecurity vulnerabilities in it, and they don't want to get bitten two weeks or two years later with a license violation, because there was kind of fuzzy, or incomplete data around the open source license. So what we do is, we help them consume the open source software, you know, knowing that it's been vetted to these standards. And then we also work with the open source community to cause the software to be changed to meet those standards, right? So back to the first part of your question, We work with a lot of projects with the prime maintainers, often the authors, as I said, and we've actually been extending our model over the years to work with these open source maintainers to cover not just their own project, but, some of those neighboring projects, right? Like the core projects that their project depends on, other projects that are co-used with them, they have a lot of expertise, and also, you know, relationships with the surrounding open source community there. So, they're working with us as curators, if you will, our ambassadors that help us get on the community and cover as much of the landscape as possible. >> And, so, what's the relationship with AWS? This is, you know, we're talking here as part of the AWS startup showcase season two, episode one, which is, that's actually pretty cool. So we need to, you know, the challenge here is, season one was awesome, much like Ted Lasso, season two, we have big shoes to fill here, Donald. So, what's the-- >> We got to up our game. >> (laughs) What's the relationship with AWS? And, I mean, why would they call you out as someone interesting for us to talk to? >> Yeah, so, we've had a great relationship that we've been investing in, and working on together with AWS. So, every one of AWS's customers faces this challenge around the software workloads that they're deploying on AWS. You know, it's just, you can't argue against the fact that the vast majority of the application software in the modern world is comprised majority of this third-party open source software. And so, it's really important whether it's running on a device, you know, an Edge device, or whether it's running in a Cloud data center, that those applications meet these standards, especially on the security front. So, AWS recognizes this need and opportunity for their customers, and so we've been working really well jointly with them. We're glad to say that we're an ISV, and AWS ISV accelerate partner now, which gives us the ability to co-engage with AWS and work together to solve mutual customers challenges, and we've had a great time working with the AWS team to help scale up our efforts to get the word word out around this important area, and then more importantly, give organizations the tools to address it and make sure that they have a comprehensive strategy for managing their open source in place. >> Fantastic, Donald, we're up against time, but I do have a 10 second answer I'd like from you. Tidelift, is that a reference to a rising tide lifting all boats, or is it an admonishment not to build a house on the beach in Malibu? >> It's the former, you know, think about this network of independent open source maintainers, working together, a rising tide lifts all boats. >> Eight seconds, that was like four seconds. Perfect. Donald Fischer, from Tidelift, thank you so much. For me, Dave Nicholson here at the CUBE. This has been a CUBE Conversation, as part of AWS's startup showcase, season two, episode one. Come to the CUBE for the best in tech coverage. (soft music)

(upbeat music) >> Narrator: From theCUBE studios in Palo Alto and Boston, connecting with thought leaders all around the world, this is a Cube conversation. >> Hi, I'm Stu Miniman, and I'm talking to you out of our Boston area studio, and we have been doing a CXO leadership series, talking with leaders across the IT industry about how they're managing during this global pandemic. I'm really happy to welcome back to the program, he's a Cube alumni. He was a Racker, and he is now with Stacker. We'll get into the company in a bit, but Prashanth Chandrasekar, the CEO of Stack Overflows, thanks so much for joining. >> Thank you for having me again Stu. Really a pleasure, and always a fan of the Cube, so great to be here. >> Alright, and we note that you sporting the quarantine, you know beard, you know, grown since the last time we had you on the program. Prasthanth, you were named CEO of Stack Overflow at the end of 2019. Obviously, certain plans that you have you're a Harvard Business School alum, you've worked in, you know, the enterprise and cloud communities for a while. Take us back to, you know, what your team has been doing, really to react and lead in this global pandemic. >> Ya, no happy to, Stu, and obviously this is a very, you know, trying time for, you know, just the world in general right. So, companies small and large are having to kind of grapple with the reality, but I would say in general, I started October 1st, 2019 at, you know, at this amazing company, and it's just, been a real joy to see us really adapt very quickly based on just you know, just kind of challenging environment that we're in, and primarily if you think about Stack Overflow, you know, we were blessed that our, you know, our company has an ethos, an ethos perspective. We've been you know, highly remote in nature for years, for over a decade so you know, 80% of our team, product engineering team has been remote. 60% of our marketing team was remote, and then 40% of our company was remote all around the world. So, moving from that 40% to 100%, which we did very proactively in March, early March of 2020, has been a huge boon for our company in just our Stackers as you pointed out, they've just been very, I would say grateful that we've done that very, very quickly. Secondly, I would say the just the notion of, you know, being able to think about our business, and you know, our community, and how do we help each other. We've done a lot, you know, we meet with you know, we come together as a team, you know, three times a week, and we've already had sort of this Covid stand up as a leadership team, as a newly formed leadership team mind you, which I've just helped form over the past six months, and we've all really gone, you know, really to the extremes to make sure that our Stackers are their health and safety are taken care of. How do we serve our community in this environment? How do we make sure our customers are being, you know, really are getting the maximum value of our products, which are all focused on collaboration, so very relevant in this remote world. So, it's really been, I would say, all around, people have really rallied we had sort of a, I would say, somewhat of an advantage just having you know, adopting remote work at this point. >> But Prasthanth, maybe it makes sense if actually step back for a second. I'm sure most people are familiar with Stack Overflow, but give us, the kind of, the high level view of, you know, what the company is, and what drew you into the leadership role there. >> Yeah, no absolutely. You know I think Stack Overflow extremely well known obviously, with every developer and technologist in the world. So, in a nutshell, you know, we are the world's most trusted and largest community for developers and technologists. We have something like 120 million unique visitors that come to our websites every month, and talking 180,000 sign ups on a monthly basis. So, just say we do say a dramatic amount of impact to help ultimately, these folks solve their most complex problems on a variety of topics, whether that is cloud related topics, security related topics, full stack engineering related topics like Python or Rust, or you name it. All those, you know, those areas are covered in very much and very a lot of detail for our community we effectively share. Solutions to common questions, and code, and really be able to accelerate the development of software around the world. So, ultimately, it comes down to our mission, which our mission what we like to say is we help write the script of the future by serving developers and technologists, and so, that's our company in a nutshell. On top of that, ecosystem of communities that we've built. We have a great set of products, SaaS products that we've also built to help with real time collaboration within companies in a very, very similar format to our public community format. So, that's been very compelling. So, the two reasons why I joined the company beyond obviously the mission, number 1 is just the global impact, you know, there are only a few companies that have the level of impact that this company has around the world and helping everybody sort of accelerate their software development. Whatever apps you're building, and obviously we know, that we're sort of in this beautiful, Goldilocks zone of digital transformation, where everything is accelerating, even given the current environment. That's the first reason, just given the vast reach of this company, and then secondly, you know, is the fact that we are really trying to transform the company and accelerate the transformation into a SaaS company. So, our Stack Overflow for teams product, which is again the knowledge sharing SaaS squad that we have internally, is really a phenomenal way to share evergreen knowledge, and non-ephemeral type information within companies so that your most important questions are answered. They're answered once, and your not, you know, constantly having to, you know, tap people on the shoulder to answer a common question. So, those are the two primary reasons. One is the impact to the community, and secondly acceleration of our SaaS business. >> Excellent, Prasthanth. So wonder if you could help us drill in, and understand the business little bit. There's private repository, there's teams there. You know, it's interesting, if you look on the outside you say wait, is this kind of like a Reddit? Or when I hear you describe it, sure reminds me a little bit of say GitHub, who obviously got taken off the table for a rather large number so, I'll let you bring us inside a little bit of you know, how does the company you know, make money, and what are the plans that both, you know, support, you know, those broad communities and diverse things, but also, you know built that business. >> Ya, no absolutely, you know I think for us you know, we really believe it's a common, our mission statement like I mentioned is really our core driver for us, and so the ecosystem of communities that we've built for developers, as well as technologists, again just a very, very vast number, and we create developers right, on a daily basis through our community. So, it's very powerful in that people are learning about new technologies, or frameworks, or you know, cloud technologies through our websites, and so they are you know, that's a bit of a huge accelerant to this creation of jobs, and you know, people's capabilities. On the foundation of that, which is obviously, you know, accessible to everybody, and you know, it's free in fact, we had this ecosystem of products, and the first one in the primary Saas product is Stack Overflow for teams, which is this knowledge sharing and collaboration product that allows companies within, or teams within companies to use the same format that they absolutely love in the public community that they use to, you know, learn up on those subjects that I mentioned, but now share internal priority information to accelerate their development internally. To breakdown walls between teams, like product, and engineering, and developers, and operations, and also go to market teams, like product marketing teams, and sales teams, and so we have you know, a tremendous number of enterprises that have joined our program, over the past several quarters including Microsoft, who is a very happy customer that uses, you know, they have something like 70,000 developers and technologists, and go to market folks within Microsoft that are using our product platform to breakdown walls, and to be able to move very quickly with launching their products, and staying collaborative internally. In addition to that, we have what we call our Reach and Relevance business which is all around helping, just based on the fact that we have such massive reach in 120 million people from around the world showing up on our websites. Being able to help companies you know, showcase their capabilities and products in our platform, and also engage with the community, and for obviously the community to then learn about many of the latest and greatest of what's being launched by these phenomenal companies that are innovating very rapidly. >> Ya, so Prasthanth, we started off the conversation, you talked a little bit about the impact of the global pandemic. I'm curious, are you seeing any, you know, changes in trends? Are there new things that are trending on your site? Are there things that are either on the website, or they're coming to your team to learn more about? >> Ya, no definitely I think there are two places that I can point to. One would be on the community side we've definitely seen a spike in traffic in places like our meta-academia website, you know, as an example. Online learning became a huge topic of interest when people went remote, and obviously, you have families around the world that are trying to figure out not only how to school their kids but we have teachers all around in schools trying to figure out what are the best set of resources. So, we have, you know, all sorts of, like I said, about 40 million questions and answers across all sorts of topics, including you know, next generation E-learning sort of capabilities in our communities, and so, we've seen a spike in traffic in places like that. We've seen a spike in our medical communities, and our biology communities obviously, because of you know, people's curiosity, and these are, you know fairly advanced, you know academics, and people who are in the scientific community that spend a lot of time thinking about, you know the what's really behind Covid-19. What are the details of, you know, if you think about all sorts of topics around genetics, and obviously, the pharmaceutical implications so, we've seen a tremendous uptake in those sites, and in addition of course, overall to our overall websites, because people are spending time, you know, just at home. In addition, we've seen a very material uptake in our Stack Overflow for teams product where we know we just closed, you know our company's like largest deal in our company's history this past week for about 30,000 seats, you know, at a very large financial services institution, a global services financial institution. There's more and more companies that are thinking about business continuity. They're thinking about how do they stay, how do they collaborate across their distributed teams, their remote teams, and we have, obviously a very significant solution in that space. >> Excellent, well congratulations on that deal. It brings up, I guess, what are some of the key KPI's that you're tracking for to really assure the growth and the health of your business. >> Ya, I think both in terms of, you know , if you think about two sides of the coin right. From the community standpoint, obviously we care about our active users, and our engaged users, and the number of sign-ups, and on that front, that first part of that, you know, we've seen just a dramatic increase, you know, in all those stats, including, you know this year, just as a result of Covid, on average last year, in 2019, you know, the number of sign-ups per month was something like, 150,000 sign-ups per month, unique sign ups from around the world. People signing up for Stack Overflow accounts. This year, on average, it's gone up, and March was our highest sign-up month ever with 180,000 sign-ups for the month. So, we're seeing so that's important. In addition to sign-ups of course, when they come on to our websites we want them to get the answers to their most pressing questions, to be able to engage them with content that is useful to them. So, engagement, you know in terms of monthly engaged users very important, monthly active users is very important for us, and obviously our sign-up numbers. So, those are kind of the community oriented stats that we'd, and KPI's that we'd really track, and those look, you know look very promising, and then, finally on the business side, which is the other side of the coin, in our teams business primarily, and our Reach and Relevance business. Our teams business is all about our customers getting value from the collaboration SaaS platform that we have that they've signed up for right. So, are they using the various features? We've integrated that teams product with all the other popular tools that people use for things like real time collaborations. We integrate with Slack. We integrate with Microsoft Teams. We've integrated with, you know Okta. We've integrated with, you know Okta. We've integrated even with Enterprise, because really the idea is to be a part of that developer and technologist workflow so, folks can really look to Stackflow for Teams as the place where they get common answers, get great answers to their common questions that are constantly being asked within companies, but it's not very effective to ask the same questions again and again. So, the idea is to integrate with these tools to make sure that you are able to have an evergreen place where you can keep that knowledge. So, that's, you know we track usage of those integrations. We talk about how many of those questions and answers are being, you know, being exchanged within companies, and how much ultimately the outcome of saving time and money for our clients so that they are being very effective in their product development cycles, and people are not being tapped on the shoulder for every single item that might comes across for an individual company. So, that's really, there's an economic study that we performed with Forrester that captures a lot of this. So that's, you know, that's and then region relevance is all around engagement on our websites. Some people already looking and seeing, finding value in the content that our companies are posting, and force companies to be effectively translating their knowledge to the audience. >> Awesome. Well, Prasthanth congratulations on the progress, and definitely look forward to cracking the how the Stack Overflow Team is doing going forward. >> Thanks so much Stu, really appreciate the chat, and great to see you again as usual. >> Alright, make sure to check out theCUBE.net for all the coverage. I'm Stu Miniman. Thank you for watching. (gentle music) (gentle music) (gentle music) (gentle music) (gentle music) (gentle music)

>> Announcer: Live from Orlando, Florida it's theCUBE! Covering Microsoft Ignite, brought to you by Cohesity. >> Hello, and happy taco Tuesday CUBE viewers! You are watching theCUBE's live coverage of Microsoft's Ignite here in Orlando, Florida. I'm your host Rebecca Knight, along with Stu Miniman. We're joined by Scott Hanselman, he is the partner program manager at Microsoft. Thank you so much for coming on theCUBE! >> Absolutely, my pleasure! >> Rebecca: And happy taco Tuesday to you! Will code for tacos. >> Will code for tacos. >> I'm digging it, I'm digging it >> I'm a very inexpensive coder. >> So you are the partner program manager, but you're really the people's programmer at Microsoft. Satya Nadella up on the main stage yesterday, talking about programming for everyone, empowering ordinary citizen developers, and you yourself were on the main stage this morning, "App Development for All", why is this such a priority for Microsoft at this point in time? >> Well there's the priority for Microsoft, and then I'll also speak selfishly as a priority for me, because when we talk about inclusion, what does that really mean? Well it is the opposite of exclusion. So when we mean inclusion, we need to mean everyone, we need to include everyone. So what can we do to make technology, to make programming possible, to make everyone enabled, whether that be something like drag and drop, and PowerApps, and the Power platform, all the way down to doing things like we did in the keynote this morning with C# on a tiny micro-controller, and the entire spectrum in between, whether it be citizen programmers in Excel using Power BI to go and do machine learning, or the silly things that we did in the keynote with rock, paper scissors that we might be able to talk about. All of that means including everyone and if the site isn't accessible, if Visual Studio as a tool isn't accessible, if you're training your AI in a non-ethical way, you are consciously excluding people. So back to what Satya thinks is why can't everyone do this? SatyaSacha thinks is why can't everyone do this? Why are we as programmers having any gate keeping, or you know, "You can't do that you're not a programmer, "you know, I'm a programmer, you can't have that." >> So what does the future look like, >> Rebecca: So what does the future look like, if everyone knows how to do it? I mean, do some imagining, visioning right now about if everyone does know how to do this, or at least can learn the building blocks for it, what does technology look like? >> Well hopefully it will be ethical, and it'll be democratized so that everyone can do it. I think that the things that are interesting, or innovative today will become commoditized tomorrow, like, something as simple as a webcam detecting your face, and putting a square around it and then you move around, and the square, we were like, "Oh my God, that was amazing!" And now it's just a library that you can download. What is amazing and interesting today, like AR and VR, where it's like, "Oh wow, I've never seen augmented reality work like that!" My eight-year-old will be able to do it in five years, and they'll be older than eight. >> So Scott, one of the big takeaways I had from the app dev keynote that you did this morning was in the past it was trying to get everybody on the same page, let's move them to our stack, let's move them to our cloud, let's move them on this programming language, and you really talked about how the example of Chipotle is different parts of the organization will write in a different language, and there needs to be, it's almost, you know, that service bus that you have between all of these environments, because we've spent, a lot of us, I know in my career I've spent decades trying to help break down those silos, and get everybody to work together, but we're never going to have everybody doing the same jobs, so we need to meet them where they are, they need to allow them to use the tools, the languages, the platforms that they want, but they need to all be able to work together, and this is not the Microsoft that I grew up with that is now an enabler of that environment. The word we keep coming back to is trust at the keynote. I know there's some awesome, cool new stuff about .net which is a piece of it, but it's all of the things together. >> Right, you know I was teaching a class at Mesa Community College down in San Diego a couple of days ago and they were trying, they were all people who wanted jobs, just community college people, I went to community college and it's like, I just want to know how to get a job, what is the thing that I can do? What language should I learn? And that's a tough question. They wonder, do I learn Java, do I learn C#? And someone had a really funny analogy, and I'll share it with you. They said, well you know English is the language, right? Why don't the other languages just give up? They said, you know, Finland, they're not going to win, right? Their language didn't win, so they should just give up, and they should all speak English, and I said, What an awful thing! They like their language! I'm not going to go to people who do Haskell, or Rust, or Scala, or F#, and say, you should give up! You're not going to win because C won, or Java won, or C# won. So instead, why don't we focus on standards where we can inter-operate, where we can accept that the reality is a hybrid cloud things like Azure Arc that allows us to connect multiple clouds, multi-vendor clouds together. That is all encompassing the concept of inclusion, including everyone means including every language, and as many standards as you can. So it might sound a little bit like a Tower of Babel, but we do have standards and the standards are HTTP, REST, JSON, JavaScript. It may not be the web we deserve, but it's the web that we have, so we'll use those building block technologies, and then let people do their own thing. >> So speaking of the keynote this morning, one of the cool things you were doing was talking about the rock, paper, scissors game, and how it's expanding. Tell our viewers a little bit more about the new elements to rock, paper, scissors. >> So folks named Sam Kass, a gentleman named Sam Kass many, many years ago on the internet, when the internet was much simpler web pages, created a game called Rock, Paper, Scissors, Lizard, Spock, and a lot of people will know that from a popular TV show on CBS, and they'll give credit to that show, in fact it was Sam Kass and Karen Bryla who created that, and we sent them a note and said, "Hey can I write a game about this?" And we basically built a Rock, Paper, Scissors, Lizard, Spock game in the cloud containerized at scale with multiple languages, and then we also put it on a tiny device, and what's fun about the game from a complexity perspective is that rock, paper, scissors is easy. There's only three rules, right? Paper covers rock, which makes no sense, but when you have five, it's hard! Spock shoots the Rock with his phaser, and then the lizard poisons Spock, and the paper disproves, and it gets really hard and complicated, but it's also super fun and nerdy. So we went and created a containerized app where we had all different bots, we had node, Python, Java, C#, and PHP, and then you can say, I'm going to pick Spock and .net, or node and paper, and have them fight, and then we added in some AI, and some machine learning, and some custom vision such that if you sign in with Twitter in this game, it will learn your patterns, and try to defeat you using your patterns and then, clicking on your choices and fun, snd then, clicking on your choices and fun, because we all want to go, "Rock, Paper, Scissors shoot!" So we made a custom vision model that would go, and detect your hand or whatever that is saying, this is Spock and then it would select it and play the game. So it was just great fun, and it was a lot more fun than a lot of the corporate demos that you see these days. >> All right Scott, you're doing a lot of different things at the show here. We said there's just a barrage of different announcements that were made. Love if you could share some of the things that might have flown under the radar. You know, Arc, everyone's talking about, but some cool things or things that you're geeking out on that you'd want to share with others? >> Two of the things that I'm most excited, one is an announcement that's specific to Ignite, and one's a community thing, the announcement is that .net Core 3.1 is coming. .net Core 3 has been a long time coming as we have began to mature, and create a cross platform open source .net runtime, but .net Core 3.1 LTS Long Term Support means that that's a version of .net core that you can put on a system for three years and be supported. Because a lot of people are saying, "All this open source is moving so fast! "I just upgraded to this, "and I don't want to upgrade to that". LTS releases are going to happen every November in the odd numbered years. So that means 2019, 2021, 2023, there's going to be a version of .net you can count on for three years, and then if you want to follow that train, the safe train, you can do that. In the even numbered years we're going to come out with a version of .net that will push the envelope, maybe introduce a new version of C#, it'll do something interesting and new, then we tighten the screws and then the following year that becomes a long term support version of .net. >> A question for you on that. One of the challenges I hear from customers is, when you talk about hybrid cloud, they're starting to get pulled apart a little bit, because in the public cloud, if I'm running Azure, I'm always on the latest version, but in my data center, often as you said, I want longer term support, I'm not ready to be able to take that CICD push all of the time, so it feels like I live, maybe call it bimodal if you want, but I'm being pulled with the am I always on the latest, getting the latest security, and it's all tested by them? Or am I on my own there? How do you help customers with that, when Microsoft's developing things, how do you live in both of those worlds or pull them together? >> Well, we're really just working on this idea of side-by-side, whether it be different versions of Visual Studio that are side-by-side, the stable one that your company is paying for, and then the preview version that you can go have side-by-side, or whether you could have .net Core 3, 3.1, or the next version, a preview version, and a safe version side-by-side. We want to enable people to experiment without fear of us messing up their machine, which is really, really important. >> One of the other things you were talking about is a cool community announcement. Can you tell us a little bit more about that? >> So this is a really cool product from a very, very small company out of Oregon, from a company called Wilderness Labs, and Wilderness Labs makes a micro-controller, not a micro-processor, not a raspberry pie, it doesn't run Linux, what it runs is .net, so we're actually playing Rock, Paper, Scissors, Lizard, Spock on this device. We've wired it all up, this is a screen from our friends at Adafruit, and I can write .net, so somehow if someone is working at, I don't know, the IT department at Little Debbie Snack Cakes, and they're making WinForms applications, they're suddenly now an IOT developer, 'cause they can go and write C# code, and control a device like this. And when you have a micro-controller, this will run for weeks on a battery, not hours. You go and 3D print a case, make this really tiny, it could become a sensor, it could become an IOT device, or one of thousands of devices that could check crops, check humidity, moisture wetness, whatever you want, and we're going to enable all kinds of things. This is just a commodity device here, this screen, it's not special. The actual device, this is the development version, size of my finger, it could be even smaller if we wanted to make it that way, and these are our friends at Wilderness Labs. and they had a successful Kickstarter, and I just wanted to give them a shout out, and I just wanted to give them a shoutout, I don't have any relationship with them, I just think they're great. >> Very cool, very cool. So you are a busy guy, and as Stu said, you're in a lot of different things within Microsoft, and yet you still have time to teach at community college. I'm interested in your perspective of why you do that? Why do you think it's so important to democratize learning about how to do this stuff? >> I am very fortunate and I think that we people, who have achieved some amount of success in our space, need to recognize that luck played a factor in that. That privilege played a factor in that. But, why can't we be the luck for somebody else, the luck can be as simple as a warm introduction. I believe very strongly in what I call the transitive value of friendship, so if we're friends, and you're friends, then the hypotenuse can be friends as well. A warm intro, a LinkedIn, a note that like, "Hey, I met this person, you should talk to them!" Non-transactional networking is really important. So I can go to a community college, and talk to a person that maybe wanted to quit, and give a speech and give them, I don't know, a week, three months, six months, more whatever, chutzpah, moxie, something that will keep them to finish their degree and then succeed, then I'm going to put good karma out into the world. >> Paying it forward. >> Exactly. >> So Scott, you mentioned that when people ask for advice, it's not about what language they do, is to, you know, is to,q you know, we talk in general about intellectual curiosity of course is good, being part of a community is a great way to participate, and Microsoft has a phenomenal one, any other tips you'd give for our listeners out there today? >> The fundamentals will never go out of style, and rather than thinking about learning how to code, why not think about learning how to think, and learning about systems thinking. One of my friends, Kishau Rogers, talked about systems thinking, I've hade her on my podcast a number of times, and we were giving a presentation at Black Girls Code, and I was talking to a fifteen-year-old young woman, and we were giving a presentation. It was clear that her mom wanted her to be there, and she's like, "Why are we here?" And I said, "All right, let's talk about programming "everybody, we're talking about programming. "My toaster is broken and the toast is not working. "What do you think is wrong?" Big, long, awkward pause and someone says, "Well is the power on?" I was like, "Well, I plugged a light in, "and nothing came on" and they were like, "Well is the fuse blown?" and then one little girl said "Well did the neighbors have power?", And I said, "You're debugging, we are debugging right?" This is the thing, you're a systems thinker, I don't know what's going on with the computer when my dad calls, I'm just figuring it out like, "Oh, I'm so happy, you work for Microsoft, "you're able to figure it out." >> Rebecca: He has his own IT guy now in you! >> Yeah, I don't know, I unplug the router, right? But that ability to think about things in the context of a larger system. I want toast, power is out in the neighborhood, drawing that line, that makes you a programmer, the language is secondary. >> Finally, the YouTube videos. Tell our viewers a little bit about those. you can go to D-O-T.net, so dot.net, the word dot, you can go to d-o-t.net, so dot.net, the word dot, slash videos and we went, and we made a 100 YouTube videos on everything from C# 101, .net, all the way up to database access, and putting things in the cloud. A very gentle, "Mr. Rodgers' Neighborhood" on-ramp. A lot of things, if you've ever seen that cartoon that says, "Want to draw an owl? "Well draw two circles, "and then draw the rest of the fricking owl." A lot of tutorials feel like that, and we don't want to do that, you know. We've got to have an on-ramp before we get on the freeway. So we've made those at dot.net/videos. >> Excellent, well that's a great plug! Thank you so much for coming on the show, Scott. >> Absolutely my pleasure! >> I'm Rebecca Knight, for Stu Miniman., stay tuned for more of theCUBE's live coverage of Microsoft Ignite. (upbeat music)

(upbeat music) >> From New York City, it's theCUBE. Covering "Welcome To The New Edge." Brought to you by Pensando Systems. >> Hey, welcome back here ready. Jeff Frick here with theCUBE. We are high atop Goldman Sachs in downtown Manhattan, I think it's 43 floors, for a really special event. It's the Pensando launch. It's really called welcome to the new edge and we talked about technology. We had some of the founders on but, these type of opportunities are really special to talk to some really senior leaders and we're excited to have John Chambers back on, who as you know, historic CEO of Cisco for many, many years. Has left that, is doing his own ventures he's writing books, he's investing and he's, happens to be chairman of the board of Pensando. So John, thanks for taking a few minutes with us. >> Well, more than a few minutes, I think what we talked about today is a major industry change and so to focus on that and focus about the implications will be a lot of fun. >> So let's jump into it. So, one of the things you led with earlier today was kind of these 10 year cycles and they're not exactly 10 years, but you outlined a series of them from mainframe, mini client server everybody knows kind of the sequence. What do you think it is about the 10 year kind of cycle besides the fact that it's easy and convenient for us to remember, that, kind of paces these big disruptions? >> Well, I think it has to do with once a company takes off they tend to, dominate that segment of the industry for so long that even if a creative idea came up they were just overpowering. And then toward the end of a 10 year cycle they quit reinventing themselves. And we talked earlier about the innovator's dilemma and the implications for it. Or an architecture that was designed that suddenly can't go to the next level. So I think it's probably a combination of three or four different factors, including the original incumbent who broke the glass, disrupted others, not disrupting themselves. >> Right, but you also talked about a story where you had to shift focus based on some customer feedback and you ran Cisco for a lot longer than 10 years. So how do you as a leader kind of keep your ears open to something that's a disruptive change that's not your regular best customer and your regular best salesman asking for a little bit faster, a little bit cheaper, a little bit of more the same versus the significant disruptive transformational shift? >> Well this goes back to one of my most basic views in life is I think we learn more from our setbacks or setbacks we were part of, or even the missteps or mistakes than you ever do your successes. Everybody loves to talk about successes and I'm no different there. But when you watched a great state like West Virginia that was the chemical center of the world and the coal mining center of the world, the 125,000 coal mines, six miners very well paid, 6,000 of the top engineers in the world, it was the Silicon Valley of the chemical industry and those just disappear. And because our state did not reinvent itself, because the education system didn't change, because we didn't distract attract a new set of businesses in we just kept doing the right thing too long, we got left behind. Then I went to Boston, it was the Silicon Valley of the world. And Route 128 around Boston was symbolic with the Silicon Valley and I-101 and 280 around it. And we had the top university at that time. Much like Stanford today, but MIT generating new companies. We had great companies, DEC, Wang, Data General. Probably a million jobs in the area and because we got stuck in a segment of the market, quit listening to our customers and missed the transitions, not only did we lose probably 1.2 million jobs on it, 100,000 out of DEC, 32,000 out of Wang, etc, we did not catch the next generation of technology changes. So I understand the implications if you don't disrupt yourself. But I also learned, that if you're not regularly reinventing yourself, you get left behind as a leader. And one of my toughest competitors came up to me and said, "John, I love the way you're reinventing Cisco "and how you've done that multiple times." And then I turned and I said "That's why a CEO has got to be in the job "for more than four or five years" and he said, "Now we disagree again." Which we usually did and he said, "Most people can't reinvent themselves." And he said "I'm an example." "I'm a pretty good CEO" he's actually a very good CEO, but he said, "After I've been there three or four years "I've made the changes, that I know "I've got to go somewhere else." And he could see I didn't buy-in and then he said, "How many of your top 100 people "you've been happy with once they've been "in the job for more than five years?" I hesitated and I said "Only one." And he's right, you've got to move people around, you've got to get people comfortable with disruption on it and, the hardest one to disrupt are the companies or the leaders who've been most successful and yet, that's when you got to think about disruption. >> Right, so to pivot on that a little bit in terms of kind of the government's role in jobs, specifically. >> Yes. >> We're in this really strange period of time. We have record low unemployment, right, tiny, tiny unemployment, and yet, we see automation coming in aggressively with autonomous vehicles and this and that and just to pick truck drivers as a category, everyone can clearly see that autonomous vehicles are going to knock them out in the not too distant future. That said, there's more demand for truck drivers today than there's every been and they can't fill the positions So, with this weird thing where we're going to have a bunch of new jobs that are created by technology, we're going to have a bunch of old jobs that get displaced by technology, but those people aren't necessarily the same people that can leave the one and go to the other. So as you look at that challenge, and I know you work with a lot of government leaders, how should they be thinking about taking on this challenge? >> Well, I think you've got to take it on very squarely and let's use the U.S. as an example and then I'll parallel what France is doing and what India is doing that is actually much more creative that what we are, from countries you wouldn't have anticipated. In the U.S. we know that 50% of the Fortune 500 will probably not exist in 10 years, 12 at the most. We know that the large companies will not incrementally hire people over this next decade and they've often been one of the best sources of hiring because of AI and automation will change that. So, it's not just a question of being schooled in one area and move to another, those jobs will disappear within the companies. If we don't have new jobs in startups and if we don't have the startups running at about three to four times the current volumes, we've got a real problem looking out five to 10 years. And the startups where everyone thinks we're doing a good job, the app user, third to a half of what they were two decades ago. And so if you need 25 million jobs over this next decade and your startups are at a level more like they were in the 90s, that's going to be a challenge. And so I think we've got to think from the government perspective of how we become a startup nation again, how we think about long-term job creation, how we think about job creation not taking money out of one pocket and give it to another. People want a real job, they want to have a meaningful job. We got to change our K through 12 education system which is broken, we've got to change our university system to generate the jobs for where people are going and then we've got to retrain people. That is very doable, if you got at it with a total plan and approach it from a scale perspective. That was lacking. And one of the disappointing things in the debate last night, and while I'm a republican I really want who's going to really lead us well both at the presidential level, but also within the senate, the house. Is, there was a complete lack of any vision on what the country should look like 10 years from now, and how we're going to create 25 million jobs and how we're going to create 10 million more that are going to be displaced and how we're going to re-educate people for it. It was a lot of finger pointing and transactional, but no overall plan. Modi did the reverse in India, and actually Macron, in all places, in France. Where they looked at GDP growth, job creation, startups, education changes, etc, and they executed to an overall approach. So, I'm looking for our government really to change the approach and to really say how are we going to generate jobs and how are we going to deal with the issues that are coming at us. It's a combination of all the the above. >> Yep. Let's shift gears a little bit about the education system and you're very involved and you talked about MIT. Obviously, I think Stanford and Cal are such big drivers of innovation in the Bay area because smart people go there and they don't leave. And then there's a lot of good buzz now happening in Atlanta as an investment really piggy-backing on Georgia Tech, which also creates a lot of great engineers. As you look at education, I don't want to go through K through 12, but more higher education, how do you see that evolving in today's world? It's super expensive, there's tremendous debt for the kids coming out, it doesn't necessarily train them for the new jobs. >> Where the jobs are. >> How do you see, kind of the role of higher education and that evolving into kind of this new world in which we're headed? >> Well, the good news and bad news about when I look at successful startups around the world, they're always centered around a innovative university and it isn't just about the raw horse power of the kids, It starts with the CEO of the university, the president of the university, their curriculum, their entrepreneurial approach, do they knock down the barriers across the various groups from engineering to business to law, etc? And are they thinking out of box? And if you watch, there is a huge missing piece between, Georgia Tech more of an exception, but still not running at the level they need to. And the Northeast around Boston and New York and Silicon Valley, The rest of the country's being left behind. So I'm looking for universities to completely redo their curriculum. I'm looking for it really breaking down the silos within the groups and focus on the outcomes. And much like Steve Case has done a very good job on focusing, about the Rust Belt and how do you do startups? I'm going to learn from what I saw in France at Polytechnique and the ITs in India, and what occurred in Stanford and MIT used to occur is, you've got to get the universities to be the core and that's where they kids want to stay close to, and we've got to generate a whole different curriculum, if you will, in the universities, including, continuous learning for their graduates, to be able to come back virtually and say how do I learn about re-skilling myself? >> Yeah. >> The current model is just not >> the right model >> It's broken. >> For the, for going forward. >> K through 12 is >> hopelessly broken >> Yeah. >> and the universities, while were still better than anywhere else in the world, we're still teaching, and some of the teachers and some of the books are what I could have used in college. >> Right, right >> So, we got to rethink the whole curriculum >> darn papers on the inside >> disrupt, disrupt >> So, shifting gears a little bit, you, played with lots of companies in your CEO role you guys did a ton of M&A, you're very famous for the successful M&A that you did over a number of years, but in an investor role, J2 now, you're looking at a more early stage. And you said you made a number of investments which is exciting. So, as you evaluate opportunities A. In teams that come to pitch to you >> Yeah. >> B. What are the key things you look for? >> In the sequence you've raised them, first in my prior world, I was really happy to do 180 acquisitions, in my current world, I'm reversed, I want them to go IPO. Because you add 76% of your headcount after an IPO, or after you've become a unicorn. When companies are bought, including what I bought in my prior role, their headcount growth is pretty well done. We'd add engineers after that, but would blow them through our sales channel, services, finance, etc. So, I want to see many more of these companies go public, and this goes back to national agenda about getting IPO's, not back to where they were during the 90's when it was almost two to three times, what you've seen over the last decade. But probably double, even that number the 90's, to generate the jobs we want. So, I'm very interested now about companies going public in direction. To the second part of your question, on what do I look for in startups and why, if I can bridge it, to am I so faired up about Pensando? If I look for my startups and, it's like I do acquisitions, I develop a playbook, I run that playbook faster and faster, it's how I do digitization of countries, etc. And so for a area I'm going to invest in and bet on, first thing I look at, is their market, technology transition, and business model transition occurring at the same time. That was Amazon of 15 years ago as an example. The second thing I look at, is the CEO and ideally, the whole founding team but it's usually just the CEO. The third thing I look for, is what are the customers really say about them? There's only one Steve Jobs, and it took him seven years. So, I go to the customers and say "What do you really think of this company?" Fourth thing I look for, is how close to an inflection point are they. The fifth thing I look for, is what they have in their ecosystem. Are they partnering? Things of that type. So, if I were to look at Pensando, Which is really the topic about can they bring to the market the new edge in a way that will be a market leading force for a whole decade? Through a ecosystem of partners that will change business dramatically and perhaps become the next major tech icon. It's how well you do that. Their vision in terms of market transitions, and business transitions 100% right. We've talked about it, 5G, IOT, internet of things, going from 15 billion devices to 500 billion devices in probably seven years. And, with the movement to the edge the business models will also change. And this is where, democratization, the cloud, and people able to share that power, where every technology company becomes a business becomes a, every business company becomes a technology company. >> Right. >> The other thing I look at is, the team. This is a team of six people, myself being a part of it, that thinks like one. That is so unusual, If you're lucky, you get a CEO and maybe a founder, a co-founder. This team, you've got six people who've worked together for over 20 years who think alike. The customers, you heard the discussions today. >> Right. >> And we've not talked to a single cloud player, a single enterprise company, a single insurance provider, or major technology company who doesn't say "This is very unique, let's talk about "how we work together on it." The inflection point, it's now you saw that today. >> Nobody told them it's young mans game obviously, they got the twenty-something mixed up >> No, actually were redefining (laughs) twenty-something, (laughs) but it does say, age is more perspective on how you think. >> Right, right. >> And Shimone Peres, who, passed away unfortunately, two years ago, was a very good friend. He basically said "You've got all your life "to think like a teenager, "and to really think and dream out of box." And he did it remarkably well. So, I think leaders, whether their twenty-something, or twenty-some years of experience working you've got to think that way. >> Right. So I'm curious, your take on how this has evolved, because, there was data and there was compute. And networking brought those two thing together, and you were at the heart of that. >> Mm-hmm. Now, it's getting so much more complex with edge, to get your take on edge. But, also more importantly exponential growth. You've talked about going from, how ever many millions the devices that were connected, to the billions of devices that are connected now. How do you stay? How do you help yourself think along exponential curves? Because that is not easy, and it's not human. But you have to, if you're going to try to get ahead of that next wave. >> Completely agree. And this is not just for me, how do I do it? I'm sharing it more that other people can learn and think about it perhaps the same way. The first thing is, it's always good to think of the positive, You can change the world here, the positive things, But I've also seen the negatives we talked about earlier. If you don't think that way, if you don't think that way as a leader of your company, leader of your country, or the leader of a venture group you're going to get left behind. The implications for it are really bad. The second is, you've got to say how do you catch and get a replicable playbook? The neat thing about what were talking about, whether it's by country in France, or India or the U.S., we've got replicable playbooks we know what to run. The third element is, you've got to have the courage to get outside your comfort zone. And I love change when it happens to you, I don't like it when it happens to me And I know that, So, I've got to get people around me who push me outside my comfort zone on that. And then, you've got to be able to dream and think like that teenager we talked about before. But that's what we were just with a group of customers, who were at this event. And they were asking "How do we get "this innovation into our company?" "How do we get the ability to innovate, through not just strategic partnerships with other large companies or partnerships with startups?" But "How do we build that internally?" It's comes down to the leader has to create that image and that approach. Modi's done it for 1.3 billion people in India. A vision, of the future on GDP growth. A digital country, startups, etc. If they can do it for 1.3 billion, tell me why the U.S. can not do it? (laughs) And why even small states here, can't do it. >> Yeah. Shifting gears a little bit, >> All right. >> A lot of black eyes in Silicon Valley right now, a lot of negativity going on, a lot of problems with privacy and trading data for currency and, it's been a rough road. You're way into tech for good and as you said, you can use technology for good you can use technology for bad. What are some things you're doing on the tech for good side? Because I don't think it gets the spotlight that it probably should, because it doesn't sell papers. >> Well, actually the press has been pretty good we just need to do it more on scale. Going back to Cisco days, we never had any major issues with governments. Even though there was a Snowden issue, there were a lot of implications about the power of the internet. Because we work with governments and citizens to say "What are the legitimate needs so that everybody benefits from this?" And where the things that we might have considered doing that, governments felt strongly about or the citizens wouldn't prosper from we just didn't do it. And we work with democrats and republicans alike and 90% of our nation believed tech was for good. But we worked hard on that. And today, I think you got to have more companies doing this and then, what, were doing uniquely in JC2, is were literally partnering with France on tech is for good and I'm Macron's, global tech ambassador and we focus about job creation and inclusion. Not just in Paris, or around Station F but throughout all the various regions in the country. Same thing within India, across 26 different states with Modi on how do you drive it through? And then if we can do it in France or India why can't we do it in each state in the U.S.? Partnering with West Virginia, with a very creative, president of the university there West Virginia University. With the democrats and republicans in their national senate, but also within the governor and speaker of the house and the president and senate within West Virginia, and really saying were going to change it together. And getting a model that you can then cookie cut across the U.S. if you change the curriculum, to your earlier comments. If you begin to focus on outcomes, not being an expert in one area, which is liable not to have a job >> Right. >> Ten years later. So, I'm a dreamer within that, but I think you owe an obligation to giving back, and I think they're all within our grasps >> Right >> And I think you can do, the both together. I think at JC2 we can create a billion dollar company with less than 10 people. I think you can change the world and also make a very good profit. And I think technology companies have to get back to that, you got to create more jobs than you destroy. And you can't be destroying jobs, then telling other people how to live their lives and what their politics should be. >> Yeah. >> That just doesn't work in terms of the environment. >> Well John, again, thanks for your time. Give you the last word on >> Sure >> Account of what happened here today, I mean you're here, and Tony O'Neary was here or at the headquarters of Goldman. A flagship launch customer, for the people that weren't here today why should they be paying attention? >> Well, if we've got this market transition right, the technology and business model, the next transition will be everything goes to the edge. And as every company or every government, or every person has to be both good in their "Area of expertise." or their vertical their in, they've got to also be good in technology. What happened today was a leveling of the playing field as it relates to cloud. In terms of everyone should have choice, democratization there, but also in architecture that allows people to really change their business models, as everything moves to the edge where 75% of all transactions, all data will be had and it might even be higher than that. Secondly, you saw a historic first never has anybody ever emerged from stealth after only two and a half years of existing as a company, with this type of powerhouse behind them. And you saw the players where you have a customer, Goldman Sachs, in one of the most leading edge areas, of industry change which is obviously finance leading as the customer who's driven our direction from the very beginning. And a company like NetApp, that understood the implication on storage, from two and a half years ago and drove our direction from the very beginning. A company like HP Enterprise's, who understood this could go across their whole company in terms of the implications, and the unique opportunity to really change and focus on, how do they evolve their company to provide their customer experience in a very unique way? How do you really begin to think about Equinix in terms of how they changed entirely from a source matter prospective, what they have to do in terms of the direction and capabilities? And then Lightspeed, one of the most creative intra capital that really understands this transition saying "I want to be a part of this." Including being on the board and changing the world one more time. So, what happened today? If we're right, I think this was the beginning of a major inflection point as everything moves to the edge. And how ecosystem players, with Pensando at the heart of that ecosystem, can take on the giants but also really use this technology to give everybody choice, and how they really make a difference in the future. As well as, perhaps give back to society. >> Love it. Thank you John >> My pleasure, that was fun. >> Appreciate it. You're John, I'm Jeff you're watching theCUBE. Thanks for watching, we'll see you next time. (upbeat music)

[Music] [Applause] [Music] welcome to the special cube conversations here in Palo Alto cube studios I'm John Ferrier the founder of Silicon angle in the cube we're here with divest cargoes the founder and CEO of arcus Inc our curse com ar-are see us calm and Steve Herod General Partner at at General Catalyst VCU's funded him congratulations on your launch these guys launched on Monday a hot new product software OS for networking powering white boxes in a whole new generation of potentially cloud computing welcome to this cube conversation congratulations on your >> launch thank you John >> so today I should talk about this this >> startup when do you guys were founded let's get to the specifics date you were founded some of the people on the team and the funding and we were formally incorporated in February of 2016 we really got going in earnest in August of 2016 and have you know chosen to stay in stealth the the founding team consists of myself a gentleman by the name of Kop tell he's our CTO we also have a gentleman by the name of Derek Young he's our chief architect and our backgrounds are a combination of the semiconductor industry I spent a lot of time in the semiconductor industry most recently I was president of easy chip and we sold that company to Mellanox and Kher and Derek our networking protocol experts spent 20 plus years at places like Cisco and arguably some of the best protocol guys in the world so the three of us got together and basically saw an opportunity to to bring some of the insights and and architectural innovation you know we had in mind to the Mobius a pedigree in there some some top talent absolutely some of the things that they've done in the past from some notable yeah I mean you know some if you if you'd like some just high-level numbers we have 600 plus years of experience of deep networking expertise within the company our collective team has shipped over 400 products to production we have over 200 IETF RFC papers that have been filed by the team as well as 150 plus patents so we really can do something on the pedigree for sure yeah we absolutely focused on getting the best talent in the world because we felt that it would be a significant differentiation to be able to start from a clean sheet of paper and so really having people who have that expertise allowed us to kind of take a step back and you know reimagine what could be possible with an operating system and gave us the benefit of being able to you know choose >> best-in-class approaches so what's the >> cap the point that this all came >> together what was the guiding vision was it network os's are going to be cloud-based was it going to be more I owe t what was the some of the founding principles that really got this going because clearly we see a trend where you know Intel's been dominating we see what NVIDIA is doing competitively certainly on the GPU side you're seeing the white box has become a trend Google makes their own stuff apples big making their own silicon seeking the that's kind of a whole big scale world out there that has got a lot of hardware experience what was the catalyst for you guys when you found this kinda was the guiding principle yeah I would say there were three John and you hit you hit on a couple of them in your reference to Intel and NVIDIA with some of the innovation but if I start at the top level the market the networking market is a large market and it's also very strategic and foundational in a hyper-connected world that market is also dominated by a few people and there's essentially three vertically integrated OEM so that dominate that market and when you have that type of dominance it leads to ultimately high prices and muted innovations so we felt number one the market was going through tremendous change but at the same time it had been tightly controlled by a few people the other part of it was that there was a tremendous amount of innovation that was happening at the silicon component level coming from the semiconductor industry I was early at Broadcom very you know involved in some of the networking things that happened in the early stages of the company we saw tremendous amounts of innovation feature velocity that was happening at the silicon component level that in turn led to a lot of system hardware people coming into the market and producing systems based on this wide variety of choices for you know for the silicon but the missing link was really an operating system that would unleash all that innovation so Silicon Valley is back Steve you you know you're a VC now but you were the CTO at VMware one of the companies that actually changed how data centers operate certainly as it certainly as a pretext and cloud computing was seeing with micro services and the growth of cloud silicon's hot IT operations is certainly being decimated as we old knew it in the past everything's being automated away you need more function now there's a demand this is this penny how you see I mean you always see things are a little early as of technologist now VC what got you excited about these guys what's the what's the bottom line yeah maybe two points on that which so one silicon is is definitely become interesting again if you will in the in the Silicon Valley area and I think that's partly because cloud scale and web scale allows these environments where you can afford to put in new hardware and really take advantage of it I was a semiconductor I first austerity too so it's exciting for me to see that but um you know is the fish that it's kind of a straightforward story you know especially in a world of whether it's cloud or IOT or everything networking is you know like literally the core to all of us working going forward and the opportunity to rethink it in a new design and in software first mentality felt kind of perfect right now I think I I think device even sell the team a little short even is with all the numbers that are there kr for instance this co-founder was sort of everyone you talk to will call him mister BGP which is one of the main routing protocols in the internet so just a ridiculously deep team trying to take this on and there been a few companies trying to do something kind of like this and I think what do they say that the second Mouse gets the cheese and I think I think we've seen some things that didn't work the first time around and we can really I think improve on them and have a >> chance to make a major impact on the networking market you know just to kind of go on a tangent here for a second >> because you know as you're talking kind of my brain is kind of firing away because you know one of things I've been talking about on the cube a lot is ageism and if you look at the movement of the cloud that's brought us systems mindset back you look at all the best successes out there right now it's almost a old guys and gals but it's really systems people people who understand networking and systems because the cloud is an operating system you have an operating system for networking so you're seeing that trend certainly happened that's awesome the question I have for you device is what is the difference what's the impact of this new network OS because I'm almost envisioning if I think through my mind's eye you got servers and server list certainly big train seeing and cloud it's one resource pools one operating system and that needs to have cohesiveness and connectedness through services so is this how you guys are thinking about how are you guys think about the network os what's different about what you guys are doing with ARC OS versus what's out there today now that's a great question John so in terms of in terms of what we've done the the third piece you know of the puzzle so to speak when we were talking about our team I talked a little bit about the market opportunity I talked a little bit about the innovation that was happening at the semiconductor and systems level and said the missing link was on the OS and so as I said at the onset we had the benefit of hiring some of the best people in the world and what that gave us the opportunity was to look at the twenty plus years of development that had happened on the operating system side for networking and basically identify those things that really made sense so we had the benefit of being able to adopt what worked and then augment that with those things that were needed for a modern day networking infrastructure environment and so we set about producing a product we call it our Co s and the the characteristics of it that are unique are that its first of all its best-in-class protocols we have minimal dependency on open source protocols and the reason for that is that no serious network operator is going to put an open source networking protocol in the core of their network they're just not going to risk their business and the efficacy and performance of their network for something like that so we start with best-in-class protocols and then we captured them in a very open modular Services microservices based architecture and that allows us the flexibility and the extensibility to be able to compose it in a manner that's consistent with what the end-use case is going to be so it's designed from the onset to be very scalable and very versatile in terms of where it can be deployed we can deploy it you know in a physical environment we can deploy it visa via a container or we could deploy it in the cloud so we're agnostic to all of those use case scenarios and then in addition to that we knew that we had to make it usable it makes no sense to have the best-in-class protocols if our end customers can't use them so what we've done is we've adopted open config yang based models and we have programmable api's so in any environment people can leverage their existing tools their existing applications and they can relatively easily and efficiently integrate our Co s into their networking environment and then similarly we did the same thing on the hardware side we have something that we call D pal it's a data plane adaptation layer it's an intelligent how and what that allows us to do is be Hardware agnostic so we're indifferent to what the underlying hardware is and what we want to do is be able to take advantage of the advancements in the silicon component level as well as at the system level and be able to deploy our go S anywhere it's let's take a step back so you guys so the protocols that's awesome what's the value proposition for our Co S and who's the target audience you mentioned data centers in the past is a data center operators is it developers is it service providers who was your target customer yeah so so the the piece of the puzzle that wraps everything together is we wanted to do it at massive scale and so we have the ability to support internet scale with deep routing capabilities within our Co s and as a byproduct of that and all the other things that we've done architectural II were the world's first operating system that's been ported to the high-end Broadcom strata DNX family that product is called jericho plus in the marketplace and as a byproduct of that we can ingest a full internet routing table and as a byproduct of that we can be used in the highest end applications for network operators so performance is a key value public performance as measured by internet scale as measured by convergence times as measured by the amount of control visibility and access that we provide and by virtue of being able to solve that high-end problem it's very easy for us to come down so in terms of your specific question about what are the use cases we have active discussions in data center centric applications for the leaf and spine we have active discussions for edge applications we have active discussions going on for cloud centric applications arcus can be used anywhere who's the buyer those network operator so since we can go look a variety of personas network operator large telco that's right inner person running a killer app that's you know high mission-critical high scale is that Mike right yeah you're getting you're absolutely getting it right basically anybody that has a network and has a networking infrastructure that is consuming networking equipment is a potential customer for ours now the product has the extensibility to be used anywhere in the data center at the edge or in the cloud we're very focused on some of the use cases that are in the CDN peering and IP you know route reflector IP peering use cases great Steve I want to get your thoughts because I say I know how you invest you guys a great great firm over there you're pretty finicky on investments certainly team check pedigrees they're on the team so that's a good inside market tamp big markets what's the market here for you but how do you see this market what's the bet for you guys on the market side yeah it's pretty pretty straightforward as you look at the size of the networking market with you know three major players around here and you know a longer tail owning a small piece of Haitian giant market is a great way to get started and if you believe in the and the secular trends that are going on with innovation and hardware and the ability to take advantage of them I think we have identified a few really interesting starting use cases and web-scale companies that have a lot of cost and needs in the networking side but what I would love about the software architecture it reminds me a lot of things do have kind of just even the early virtualization pieces if you if you can take advantage of movement in advantages and hardware as they improve and really bring them into a company more quickly than before then those companies are gonna be able to have you know better economics on their networking early on so get a great layer in solve a particular use case but then the trends of being able to take advantage of new hardware and to be able to provide the data and the API is to programmatic and to manage it who one would that it's creative limp limitless opportunity because with custom silicon that has you know purpose-built protocols it's easy to put a box together and in a large data center or even boxes yeah you can imagine the vendors of the advances and the chips really love that there's a good company that can take advantage of them more quickly than others can so cloud cloud service refined certainly as a target audience here large the large clouds would love it there's an app coming in Broadcom as a customer they a partner of you guys in two parts first comes a partner so we we've ported arc OS onto multiple members of the Broadcom switching family so we have five or six of their components their networking system on chip components that we've ported to including the two highest end which is the jericho plus and you got a letter in the Broadcom buying CA and that's gonna open up IT operations to you guys and volge instead of applications and me to talk about what you just said extensibility of taking what you just said about boxes and tying applique and application performance you know what's going to see that vertically integrated and i think i think eloping yeah from from a semiconductor perspective since i spent a lot of time in the industry you know one of the challenges i had founded a high court count multi processor company and one of the challenges we always had was the software and at easy chip we had the world's highest and network processor challenge with software and i think if you take all the innovation in the silicon industry and couple it with the right software the combination of those two things opens up a vast number of opportunities and we feel that with our Co s we provide you know that software piece that's going to help people take advantage of all the great innovation that's happening you mentioned earlier open source people don't want to bring open source at the core the network yet the open source communities are growing really at an exponential rate you starting to see open source be the lingua franca for all developers especially the modern software developers wine not open sourcing the core the amino acids gotta be bulletproof you need security obviously answers there but that seems difficult to the trend on open source what's the what's the answer there on why not open source in the core yeah so we we take advantage of open source where it makes sense so we take advantage of open and onl open network Linux and we have developed our protocols that run on that environment the reason we feel that the protocols being developed in-house as opposed to leveraging things from the open source community are the internet scale multi-threading of bgp integrating things like open config yang based models into that environment right well it's not only proven but our the the the capabilities that we're able to innovate on and bring unique differentiation weren't really going back to a clean sheet of paper and so we designed it ground-up to really be optimized for the needs of today Steve your old boss Palmer rich used to talk about the harden top mmm-hmm similar here right you know one really no one's really gonna care if it works great it's under the under the harden top where you use open source as a connection point for services and opportunities to grow that similar concept yes I mean at the end of the day open source is great for certain things and for community and extensibility and for visibility and then on the flip side they look to a company that's accountable and for making sure it performs and as high quality and so I think I think that modern way for especially for the mission critical infrastructure is to have a mix of both and to give back to community where it makes sense to be responsible for hardening things are building them when they don't expense so how'd you how'd you how'd you land these guys you get him early and don't sit don't talk to any other VCS how did it all come together between you guys we've actually been friends for a while which has been great in it at one point we actually decided to ask hey what do you actually do I found that I was a venture investor and he is a network engineer but now I actually have actually really liked the networking space as a whole as much as people talk about the cloud or open source or storage being tough networking is literally everywhere and will be everywhere and whatever our world looks like so I always been looking for the most interesting companies in that space and we always joke like the investment world kind of San Francisco's applications mid here's sort of operating systems and the lower you get the more technical it gets and so well there's a vaccine I mean we're a media company I think we're doing things different we're team before we came on camera but I think media is undervalued I wrote just wrote a tweet on that got some traction on that but it's shifting back to silicon you're seeing systems if you look at some of the hottest areas IT operations is being automated away AI ops you know Auto machine learning starting to see some of these high-end like home systems like that's exactly where I was gonna go it's like the vid I I especially just love very deep intellectual property that is hard to replicate and that you can you know ultimately you can charge a premium for something that is that hard to do and so that's that's really something I get drugs in the deal with in you guys you have any other syndicates in the video about soda sure you know so our initial seed investor was clear ventures gentleman by the name of Chris rust is on our board and then Steve came in and led our most recent round of funding and he also was on the board what we've done beyond that institutional money is we have a group of very strategic individual investors two people I would maybe highlight amongst the vast number of advisers we have our gentleman by the name of Pankaj Patel punka JH was the chief development officer at Cisco he was basically number two at Cisco for a number of years deep operating experience across all facets of what we would need and then there's another gentleman by the name of Amarjeet Gill I've been friends with armored teeth for 30 years he's probably one of the single most successful entrepreneurs in the he's incubated companies that have been purchased by Broadcom by Apple by Google by Facebook by Intel by EMC so we were fortunate enough to get him involved and keep him busy great pedigree great investors with that kind of electoral property and those smart mines they're a lot of pressure on you as the CEO not to screw it up right I mean come on now get all those smart man come on okay you got it look at really good you know I I welcome it actually I enjoy it you know we look when you have a great team and you have as many capable people surrounding you it really comes together and so I don't think it's about me I actually think number one it's about I was just kidding by the way I think it's about the team and I'm merely a spokesperson to represent all the great work that our team has done so I'm really proud of the guys we have and frankly it makes my job easier you've got a lot of people to tap for for advice certainly the shared experiences electively in the different areas make a lot of sense in the investors certainly yeah up to you absolutely absolutely and it's not it's not just at the at the board it's just not at the investor level it's at the adviser level and also at you know at our individual team members when we have a team that executes as well as we have you know everything falls into place well we think the software worlds change we think the economics are changing certainly when you look at cloud whether it's cloud computing or token economics with blockchain and new emerging tech around AI we think the world is certainly going to change so you guys got a great team to kind of figure it out I mean you got a-you know execute in real time you got a real technology play with IP question is what's the next step what is your priorities now that you're out there congratulations on your launch thank you in stealth mode you got some customers you've got Broadcom relationships and looking out in the landscape what's your what's your plan for the next year what's your goals really to take every facet of what you said and just scale the business you know we're actively hiring we have a lot of customer activity this week happens to be the most recent IETF conference that happened in Montreal given our company launch on Monday there's been a tremendous amount of interest in everything that we're doing so that coupled with the existing customer discussions we have is only going to expand and then we have a very robust roadmap to continue to augment and add capabilities to the baseline capabilities that we brought to the market so I I really view the next year as scaling the business in all aspects and increasingly my time is going to be focused on commercially centric activities right well congratulations got a great team we receive great investment cube conversation here I'm John furry here the hot startup here launching this week here in California in Silicon Valley where silicon is back and software is back it's the cube bringing you all the action I'm John Fourier thanks for watching [Music]

>> Announcer: From Copenhagen, Denmark, it's theCUBE. Covering KubeCon and CloudNativeCon Europe 2018. Brought to you by the Cloud Native Computing Foundation and its ecosystem partners. >> Hello and welcome back to the live CUBE coverage here in Copenhagen, Denmark, for KubeCon 2018, Kubernetes European conference. This is theCUBE, I'm John Furrier, my co-host Lauren Cooney here with Adrian Cockcroft who is the Vice President of Cloud Architecture and Strategy for Amazon Web Services, AWS. CUBE alumni, great to see you, a legend in the industry, great to have you on board today. Thanks for coming on. >> Thanks very much. >> Quick update, Amazon, we were at AWS Summit recently, I was at re:Invent last year, it gets bigger and bigger just continue to grow. Congratulations on successful great earnings. You guys posted last week, just continuing to show the scale and leverage that the cloud has. So, again, nothing really new here, cloud is winning and the model of choice. So you guys are doing a great job, so congratulations. Open source, you're handling a lot of that now. This community here, is all about driving cloud standards. >> Adrian: Yeah. >> Your guys position on that is? Standards are great, you do what customers want, as Andy Jassy always says, what's the update? I mean, what's new since Austin last year? >> Yeah, well, it's been great to be back on had a great video of us talking at Austin, it's been very helpful to get the message out of what we're doing in containers and what the open source team that I lead has been up to. It's been very nice. Since then we've done quite a lot. We were talking about doing things then, which we've now actually done and delivered on. We're getting closer to getting our Kubernetes service out, EKS. We hired Bob Wise, he started with us in January, he's the general manager of EKS. Some of you may know Bob has been working with Kubernetes since the early days. He was on the CNCF board before he joined us. He's working very hard, they have a team cranking away on all the things we need to do to get the EKS service out. So that's been major focus, just get it out. We have a lot of people signed up for the preview. Huge interest, we're onboarding a lot of people every week, and we're getting good feedback from people. We have demos of it in the booth here this week. >> So you guys are very customer-centric, following you guys closely as you know. What's the feedback that you're hearing and what are you guys ingesting from an intelligence standpoint from the field. Obviously, a new constituent, not new, but a major constituent is open source communities, as well as paying enterprise customers? What's the feedback? What are you hearing? I would say beyond tire kicking, there's general interest in what Kubernetes has enabled. What's Amazon's view of that? >> Yeah, well, open source in general is always getting a larger slice of what people want to do. Generally, people are trying to get off of their enterprise solutions and evolving into an open source space and then you kind of evolve from that into buying it as a service. So that's kind of the evolution from one trend, custom or enterprise software, to open source to as a service. And we're standing up all of these tools as a service to make them easier to consume for people. Just, everybody's happy to do that. What I'm hearing from customers is that that's what they're looking for. They want it to be easy to use, they want it to scale, they want it to be reliable and work, and that's what we're good at doing. And then they want to track the latest moves in the industry and run with the latest technologies and that's what Kubernetes and the CNCF is doing, gathering together a lot of technologies. Building the community around it, just able to move faster than we'd move on our own. We're leveraging all of those things into what we're doing. >> And the status of EKS right now is in preview? And the estimated timetable for GA? >> In the next few months. >> Next few months. >> You know, get it out then right now it's running in Oregon, in our Oregon data center, so the previews are all happening there. That gets us our initial thing and then everyone go okay, we want to in our other regions, so we have to do that. So another service we have is Fargate, which is basically say just here's a container, I want to run it, you don't have to declare a node or an instance to run it first. We launched that at re:Invent, that's already in production obviously, we just rolled that out to four regions. That's in Virginia, Oregon, Dublin and Ohio right now. A huge interest in Fargate, it lets you simplify your deployments a little bit. We just posted a new blog post that we have an open source blog, you can find if you want to keep up with what's going on with the open source team at AWS. Just another post this morning and it's a first pass at getting Fargate to work with Kubernetes using Virtual Kubelet which is a project that was kicked off by, it's an experimental project, not part of the core Kubernetes system. But it's running on the side. It's something that Microsoft came up with a little while ago. So we now have, we're working with them. We did a pull request, they accepted it, so that team and AWS and a few other customers and other people in the community, working together to provide you a way to start up Fargate as the underlying layer for provisioning containers underneath Kubernetes as the API for doing you know the management of that. >> So who do you work with mostly when you're working in open source? Who do you partner with? What communities are you engaging with in particular? >> It's all over. >> All over? >> Wherever the communities are we're engaging with them. >> Lauren: Okay, any particular ones that stand out? >> Other than CNCF, we have a lot of engagement with Apache Hadoop ecosystem. A lot of work in data science, there's many, many projects in that space. In AI and machine learning, we've sponsored, we've spend a lot of time working with Apache MXNet, we were also working off with TensorFlow by Torch and Caffe and there's a lot, those are all open source frameworks so there's lots of contributions there. In the serverless arena, we have our own SAM service application model. We've been open sourcing more of that recently ourselves and we're working with various other people. Across these different groups there's different conferences you go to, there's different things we do. We just sponsored Rails Conference. My team sponsors and manages most of the open source conference events we go to now. We just did RAILCON, we're doing a Rust conference, soon I think, there's Python conferences. I forget when all these are. There's a massive calendar of conferences that we're supporting. >> Make sure you email us that that list, we're interested actually in looking at what the news and action is. >> So the language ones, AltCon's our flagship one, we'll be top-level sponsor there. When we get to the U.S., CubeCon in Seattle, it's right there, it's two weeks after re:Invent. It's going to be much easier to manage. When we go to re:Invent it's like everyone just wants to take that week off, right. We got a week for everyone to recover and then it's in the hometown. >> You still have that look in your eyes when we interviewed you in Austin you came down, we both were pretty exhausted after re:Invent. >> Yeah, so we announced a bunch of things on Wednesday and Thursday and I had to turn it into a keynote by Tuesday and get everyone to agree. That's what was going on, that was very compressed. We have more time and all of the engineering teams that really want to be at an event like this, were right in the hometown for a lot. >> What's it like workin' at Amazon, I got to ask you it since you brought it up. I mean and you guys run hard at Amazon, you're releasing stuff with a pace that's unbelievable. I mean, I get blown away every year. Almost seems like, inhuman that that you guys can run at that pace. And earnings, obviously, the business results speak for themselves, what's it like there? I mean, you put your running shoes on, you run a marathon every day. >> It's lots of small teams working relatively independently and that scales and that's something other engineering organizations have trouble with. They build hierarchies that slow down. We have a really good engineering culture where every time you start a new team, it runs at its own speed. We've shown that as we add more and more resources, more teams, they are just executing. In fact, their accelerated, they're building on top of other things. We get to build higher and higher level abstractions to layer into. Just getting easier and easier to build things. We're accelerating our pace of innovation there's no slowing down. >> I was telling Jassy they're going to write a Harvard Business School case study on a lot of the management practices, but certainly the impact on the business side with the model that you guys do. But I got to ask you, on the momentum side, super impressed with SageMaker. I predicted on theCUBE at AWS Summit that that will be the fastest growing service. It will overtake Aurora, I think that is currently on stage, presented as the fastest growing service. SageMaker is really popular. Updates there, its role in the community. Obviously, Kubernete's a good fit for orchestrating things. We heard about CubeFlow, is an interesting model. What's going on with SageMaker how is it interplaying with Kubernetes? >> People that want to run, if you're running on-premise, cluster of GPU enabled machines then CubeFlow is a great way of doing that. You're on TensorFlow, that manages your cluster, you run CubeFlow on top. SageMaker is running at very low scale and like a lot of things we do at AWS, what you need to run an individual cluster for any one customer is different from running a multi-tenant service. SageMaker sits on top of ECS and it's now one of the largest generators of traffic to ECS which is Amazon's horizontally scaled, multi-tenant, cluster management system, which is now doing hundreds of millions of container launches a week. That is continuing to grow. We see Kubernetes as it's a more portable abstraction. It has some more, different layers of API's and a big community around it. But for the heavy lifting of running tens of thousands of containers in for a single application, we're still at the level where ECS does that every day and Kubernetes that's kind of the extreme case, where a few people are pushing it. It'll gradually grow scale. >> It's evolution. >> There's an evolution here. But the interesting things are, we're starting to get some convergence on some of the interfaces. Like the interfacing at CNA, CNA is the way you do networking on containers and there is one way of doing that, that is shared by everybody through CNA. EKS uses it, BCS uses it and Kubernetes uses it. >> And the impact of customers is what for that? What's the impact? >> It means the networking structures you want to set up will be the same. And the capabilities and the interfaces. But what happens on AWS is because it has a direct plug-in, you can hook it up to our accelerated networking infrastructure. So, AWS's instances right now, we've offloaded most of the network traffic processing. You're running 25 gigabits of traffic, that's quite a lot of work even for a big CPU, but it's handled by the the Nitro plug-in architecture we have, this in our latest instance type. So if you talked a bit about that at re:Invent but what you're getting is enormous, complete hypervisor offload at the core machine level. You get to use that accelerated networking. You're plugging into that interface. But that, if you want to have a huge number of containers on a machine and you're not really trying to drive very high throughput, then you can use Calico and we support that as well. So, multiple different ways but all through the same thing, the same plug-ins on both. >> System portability. You mentioned some stats, what's the numbers you mentioned? How many containers you're launching a week, hundreds of thousands? On ECS, our container platform that's been out for a few years, so hundreds of millions a week. It's really growing very fast. The containers are taking off everywhere. >> Microservices growth is, again that's the architecture. As architecture is a big part of the conversation what's your dialogue with customers? Because the modern software architecture in cloud, looks a lot different than what it was in the three layered approach that used to be the web stack. >> Yeah, and I think to add to that, you know we were just talking to folks about how in large enterprise organizations, you're still finding groups that do waterfall development. How are you working to kind of bring these customers and these developers into the future, per se? >> Yeah, that's actually, I spend about half my time managing the open source team and recruiting. The other half is talking to customers about this topic. I spend my time traveling around the world, talking at summits and events like this and meeting with customers. There's lots of different problems slowing people down. I think you see three phases of adoption of cloud, in general. One is just speed. I want to get something done quickly, I have a business need, I want to do it. I want machines in minutes instead of months, right, and that speeds everything up so you get something done quickly. The second phase is where you're starting to do stuff at scale and that's where you need cloud native. You really need to have elastic services, you can scale down as well as up, otherwise, you just end up with a lot of idle machines that cost you too much and it's not giving you the flexibility. The third phase we're getting into is complete data center shutdown. If you look at investing in a new data center or data center refresh or just opening an AWS account, it really doesn't make sense nowadays. We're seeing lots of large enterprises either considering it or well into it. Some are a long way into this. When you shut down the data center all of the backend core infrastructure starts coming out. So we're starting to see sort of mainframe replacement and the really critical business systems being replaced. Those are the interesting conversations, that's one of the areas that I'm particularly interested in right now and it's leading into this other buzzword, if you like, called chaos engineering. Which is sort of the, think of it as the availability model for cloud native and microservices. We're just starting a working group at CNCF around chaos engineering, is being started this week. So you can get a bit involved in how we can build some standards. >> That's going to be at Stanford? >> It's here, I mean it's a working group. >> Okay, online. >> The CNCF working group, they are wherever the people are, right. >> So, what is that conversation when you talk about that mainframe kind of conversation or shut down data centers to the cloud. What is the key thing that you promote, up front, that needs to get done by the by the customer? I mean, obviously you have the pillars, the key pillars, but you think about microservices it's a global platform, it's not a lift and shift situation, kind of is, it shut down, but I mean not at that scale. But, security, identity, authentication, there's no perimeter so you know microservices, potentially going to scale. What are the things that you promote upfront, that they have to do up front. What are the up front, table stake decisions? >> For management level, the real problem is people problems. And it's a technology problem somewhere down in the weeds. Really, if you don't get the people structures right then you'll spend forever going through these migrations. So if you sort of bite the bullet and do the reorganization that's needed first and get the right people in the right place, then you move much faster through it. I say a lot of the time, we're way upstream of picking a technology, it's much more about understanding the sort of DevOps, Agile and the organizational structures for these more cellular based organizations, you know, AWS is a great example of that. Netflix are another good example of that. Capital One is becoming a good example of that too. In banking, they're going much faster because they've already gone through that. >> So they're taking the Amazon model, small teams. Is that your general recommendation? What's your general recommendation? >> Well, this is the whole point of microservices, is that they're built by these small teams. It's called Conway's law, which says that the code will end up looking like the team, the org structure that built it. So, if you set up a lots of small teams, you will end up with microservices. That's just the way it works, right. If you try to take your existing siloed architecture with your long waterfall things, it's very hard not to build a monolith. Getting the org structure done first is right. Then we get into kind of the landing zone thing. You could spend years just debating what your architecture should be and some people have and then every year they come back, and it's changing faster than they can decide what to do. That's another kind of like analysis paralysis mode you see some larger enterprises in. I always think just do it. What's the standard best practice, layout my accounts like this, my networks like this, my structures we call it landing zone. We get somebody up to speed incredibly quickly and it's the beaten path. We're starting to build automation around these on boarding things, we're just getting stuff going. >> That's great. >> Yeah, and then going back to the sort of chaos engineering kind of idea, one of the first things I should think you should put into this infrastructure is the disaster recovery automation. Because if that gets there before the apps do, then the apps learn to live with the chaos monkeys and things like that. Really, one of the first apps we installed at Netflix was Chaos Monkey. It wasn't added later, it was there when you arrived. Your app had to survive the chaos that was in the system. So, think of that as, it used to be disaster recovery was incredibly expensive, hard to build, custom and very difficult to test. People very rarely run through their disaster recovery testing data center fail over, but if you build it in on day one, you can build it automated. I think Kubernetes is particularly interesting because the API's to do that automation are there. So we're looking at automating injecting failure at the Kubernetes level and also injecting into the underlying machines that are running Good Maze, like attacking the control plane to make sure that the control plane recovery works. I think there's a lot we can do there to automate it and make it into a low-cost, productized, safe, reliable thing, that you do a lot. Rather than being something that everyone's scared of doing that. >> Or they bolted on after they make decisions and the retrofit, pre-existing conditions into a disaster recovery. Which is chaotic in and of itself. >> So, get the org chart right and then actually get the disaster recovery patterns. If you need something highly available, do that first, before the apps turn up. >> Adrian, thanks for coming on, chaos engineering, congratulations and again, we know you know a little about Netflix, you know that environment, and been big Amazon customer. Congratulations on your success, looking forward to keeping in touch. Thanks for coming on and sharing the AWS perspective on theCUBE. I'm John Furrier, Lauren Cooney live in Denmark for KubeCon 2018 part of the CNC at the Cloud Native Compute Foundation. We'll back with more live coverage, stay with us. We'll be right back. (upbeat music)