(bright music) >> Hey everyone, welcome to this CUBE conversation featuring Ahana, I'm your host Lisa Martin. I've got two guests here with me today. Steven Mih joins us, the Presto Foundation governing board member, co-founder and CEO of Ahana, and Girish Baliga Presto Foundation governing board chair and senior engineering manager at Uber. Guys thanks for joining us. >> Thanks for having us. >> Thanks for having us. >> So Steven we're going to dig into and unpack Presto in the next few minutes or so, but Steven let's go ahead and start with you. Talk to us about some of the challenges with the open data lake house market. What are some of those key challenges that organizations are facing? >> Yeah, just pulling up the slide you know, what we see is that many organizations are dealing with a lot more data and very different data types and putting that all into, traditionally as the data warehouse, which has been the workhorse for BI and analytics traditionally, it becomes very, very expensive, and there's a lot of lock in associated with that. And so what's happening is that people are putting the data semistructured and unstructured data for example, in cloud data lakes or other data lakes, and they find that they can query directly with a SQL query engine like Presto. And that lets you have a much more approach to dealing with getting insights out of your data. And that's what this is all about, and that's why companies are moving to a modern architecture. Girish maybe you can share some of your thoughts on how Uber uses Presto for this. >> Yeah, at Uber we use Presto in our internal deployments. So at Uber we have our own data centers, we store data locally in our data centers, but we have made the conscious choice to go with an open data stack. Our entire data stack is built around open source technologies like Hadoop, Hive, Spark and Presto. And so Presto is an invaluable engine that is able to connect to all these different storage and data formats and allow us to have a single entry point for our users, to run their SQL engines and get insights rather quickly compared to some of the other engines that we have at Uber. >> So let's talk a little bit about Presto so that the audience gets a good overview of that. Steven starting with you, you talked about the challenges of the traditional data warehouse application. Talk to us about why Presto was founded the open, the project, give us that background information if you will. >> Absolutely, so Presto was originally developed out of the biggest hyperscaler out there which is Facebook now known as Meta. And they donated that project to the, and open sourced it and donated it to the Linux Foundation. And so Presto is a SQL query engine, it's a storage SQL query engine, that runs directly on open data lakes, so you can put your data into open formats like 4K or C, and get insights directly from that at a very good price performance ratio. The Presto Foundation of which Girish and I are part of, we're all working together as a consortium of companies that all want to see Presto continue to get bigger and bigger. Kind of like Kubernetes has a, has an organization called CNCF, Presto has Presto Foundation all under the umbrella of the Linux Foundation. And so there's a lot of exciting things that are coming on the roadmap that make Presto very unique. You know, RaptorX is a multilevel caching system that it's been fantastic, Aria optimizations are another area, we Ahana have developed some security features with donating the integrations with Apache Ranger and that's the type of things that we do to help the community. But maybe Girish can talk about some of the exciting items on the roadmap that you're looking forward to. >> Absolutely, I think from Uber's point of view just a sheer scale of data and our volume of query traffic. So we run about half a million Presto queries a day, right? And we have thousands of machines in our Presto deployments. So at that scale in addition to functionality you really want a system that can handle traffic reliably, that can scale, and that is backed by a strong community which guarantees that if you pull in the new version of Presto, you won't break anything, right? So all of those things are very important to us. So I think that's where we are relying on our partners particularly folks like Facebook and Twitter and Ahana to build and maintain this ecosystem that gives us those guarantees. So that is on the reliability front, but on the roadmap side we are also excited to see where Presto is extending. So in addition to the projects that Steven talked about, we are also looking at things like Presto and Spark, right? So take the Presto SQL and run it as a Spark job for instance, or running Presto on real-time analytics applications something that we built and contributed from Uber side. So we are all taking it in very different directions, we all have different use cases to support, and that's the exciting thing about the foundation. That it allows us all to work together to get Presto to a bigger and better and more flexible engine. >> You guys mentioned Facebook and I saw on the slide I think Twitter as well. Talk to me about some of the organizations that are leveraging the Presto engine and some of the business benefits. I think Steve you talked about insights, Steven obviously being able to get insights from data is critical for every business these days. >> Yeah, a major, major use case is finding the ad hoc and interactive queries, and being able to drive insights from doing so. And so, as I mentioned there's so much data that's being generated and stored, and to be able to query that data in place, at a, with very, very high performance, meaning that you can get answers back in seconds of time. That lets you have the interactive ability to drill into data and innovate your business. And so this is fantastic because it's been developed at hyperscalers like Uber that allow you to have open source technology, pick that up, and just download it right from prestodb.io, and then start to run with this and join the community. I think from an open source perspective this project under the governance of Linux Foundation gives you the confidence that it's fully transparent and you'll never see any licensing changes by the Linux Foundation charter. And therefore that means the technology remains free forever without later on limitations occurring, which then would perhaps favor commercialization of any one vendor. That's not the case. So maybe Girish your thoughts on how we've been able to attract industry giants to collaborate, to innovate further, and your thoughts on that. >> Yeah, so of the interesting I've seen in the space is that there is a bifurcation of companies in this ecosystem. So there are these large internet scale companies like Facebook, and Uber, and Twitter, which basically want to use something like Presto for their internal use cases. And then there is the second set of companies, enterprise companies like Ahana which basically wanted to take Presto and provide it as a service for other companies to use as an alternative to things like Snowflake and other systems right? So, and the foundation is a great place for both sets of companies to come together and work. The internet scale companies bring in the scale, the reliability, the different kind of ways in which you can challenge the system, optimize it, and so forth, and then companies like Ahana bring in the flexibility and the extensibility. So you can work with different clouds, different storage formats, different engines, and I think it's a great partnership that we can see happening primarily through the foundational spaces. Which you would be hard pressed to find in a single vendor or a, you know, a single-source system that is there on the market today. >> How long ago was the Presto Foundation initiated? >> It's been over three years now and it's been going strong, we're over a dozen members and it's open to everyone. And it's all governed like the Linux Foundation so we use best practices from that and you can just check it out at prestodb.io where you can get the software, or you can hear about how to join the foundation. So it includes members like Intel, and HPE as well, and we're really excited for new members to come, and contribute in and participate. >> Sounds like you've got good momentum there in the foundation. Steven talk a little bit about the last two years. Have you seen the acceleration in use cases in the number of users as we've been in such an interesting environment where the need for real-time insights is essential for every business initially a few couple of years ago to survive but now to be, to really thrive, is it, have you seen the acceleration in Presto in that timeframe? >> Absolutely, we see there's acceleration of being more data-driven and especially moving to cloud and having more data in the cloud, we think that innovation is happening, digital innovation is happening very fast and Presto is a major enabler of that, again, being able to get, drive insights from the data this is not just your typical business data, it's now getting into really clickstream data, knowing about how customers are operating today, Uber is a great example of all the different types of innovations they can drive, whether it be, you know, knowing in real time what's happening with rides, or offering you a subscription for special deals to use the service more. So, you know, Ahana we really love Presto, and we provide a SaaS manage service of the open source and provide free trials, and help people get up to speed that may not have the same type of skills as Uber or Facebook does. And we work with all companies in that way. >> Think about the consumers these days, we're very demanding, right? When I think one of the things that was in short supply during the last two years was patience. And if I think of Uber as a great example, I want to know if I'm asking for a ride I want to know exactly in real time what's coming for me? Where is it now? How many more minutes is it going to take? I mean, that need to fulfill real-time insights is critical across every industry but have you seen anything in the last couple years that's been more leading edge, like e-commerce or retail for example? I'm just curious. >> Girish you want to take that one or? >> Yeah, sure. So I can speak from the Uber point of view. So real-time insights has really exploded as an area, particularly as you mentioned with this just-in-time economy, right? Just to talk about it a little bit from Uber side, so some of the insights that you mentioned about when is your ride coming, and things of that nature, right? Look at it from the driver's point of view who are, now we have Uber Eats, so look at it from the restaurant manager's point of view, right? They also want to know how is their business coming? How many customer orders are coming for instance? what is the conversion rate? And so forth, right? And today these are all insights that are powered by a system which has a Presto as an front-end interface at Uber. And these queries run like, you have like tens of thousands of queries every single second, and the queries run in like a second and so forth. So you are really talking about production systems running on top of Presto, production serving systems. So coming to other use cases like eCommerce, we definitely have seen some of that uptake happen as well, so in the broader community for instance, we have companies like Stripe, and other folks who are also using this hashtag which is very similar to us based on another open source technology called Pino, using Presto as an interface. And so we are seeing this whole open data lakehouse more from just being, you know, about interactive analytics to driving all different kinds of analytics. Having anything to do with data and insights in this space. >> Yeah, sounds like the evolution has been kind of on a rocket ship the last couple years. Steven, one more time we're out of time, but can you mention that URL where folks can go to learn more? >> Yeah, prestodb.io and that's the Presto Foundation. And you know, just want to say that we'll be sharing the use case at the Startup Showcase coming up with theCUBE. We're excited about that and really welcome everyone to join the community, it's a real vibrant, expanding community and look forward to seeing you online. >> Sounds great guys. Thank you so much for sharing with us what Presto Foundation is doing, all of the things that it is catalyzing, great stuff, we look forward to hearing that customer use case, thanks for your time. >> Thank you. >> Thanks Lisa, thank you. >> Thanks everyone. >> For Steven and Girish, I'm Lisa Martin, you're watching theCUBE the leader in live tech coverage. (bright music)

>> Sue: Hi, everybody. Thank you for joining us today, for the Virtual Vertica BDC 2020. This breakout session is entitled "Vertica @ Uber Scale" My name is Sue LeClaire, Director of Marketing at Vertica. And I'll be your host for this webinar. Joining me is Girish Baliga, Director I'm sorry, user, Uber Engineering Manager of Big Data at Uber. Before we begin, I encourage you to submit questions or comments during the virtual session. You don't have to wait, just type your question or comment in the question box below the slides and click Submit. There will be a Q and A session, at the end of the presentation. We'll answer as many questions as we're able to during that time. Any questions that we don't address, we'll do our best to answer offline. Alternately, you can also Vertica forums to post your questions there after the session. Our engineering team is planning to join the forums to keep the conversation going. And as a reminder, you can maximize your screen by clicking the double arrow button, in the lower right corner of the slides. And yet, this virtual session is being recorded, and you'll be able to view on demand this week. We'll send you a notification as soon as it's ready. So let's get started. Girish over to you. >> Girish: Thanks a lot Sue. Good afternoon, everyone. Thanks a lot for joining this session. My name is Girish Baliga. And as Sue mentioned, I manage interactive and real time analytics teams at Uber. Vertica is one of the main platforms that we support, and Vertica powers a lot of core business use cases. In today's talk, I wanted to cover two main things. First, how Vertica is powering critical business use cases, across a variety of orgs in the company. And second, how we are able to do this at scale and with reliability, using some of the additional functionalities and systems that we have built into the Vertica ecosystem at Uber. And towards the end, I also have a little extra bonus for all of you. I will be sharing an easy way for you to take advantage of, many of the ideas and solutions that I'm going to present today, that you can apply to your own Vertica deployments in your companies. So stick around and put on your seat belts, and let's go start on the ride. At Uber, our mission is to ignite opportunity by setting the world in motion. So we are focused on solving mobility problems, and enabling people all over the world to solve their local problems, their local needs, their local issues, in a manner that's efficient, fast and reliable. As our CEO Dara has said, we want to become the mobile operating system of local cities and communities throughout the world. As of today, Uber is operational in over 10,000 cities around the world. So, across our various business lines, we have over 110 million monthly users, who use our rides, services, or eat services, and a whole bunch of other services that we provide to Uber. And just to give you a scale of our daily operations, we in the ride business, have over 20 million trips per day. And that each business is also catching up, particularly during the recent times that we've been having. And so, I hope these numbers give you a scale of the amount of data, that we process each and every day. And support our users in their analytical and business reporting needs. So who are these users at Uber? Let's take a quick look. So, Uber to describe it very briefly, is a lot like Amazon. We are largely an operation and logistics company. And employee work based reflects that. So over 70% of our employees work in teams, which come under the umbrella of Community Operations and Centers of Excellence. So these are all folks working in various cities and towns that we operate around the world, and run the Uber businesses, as somewhat local businesses responding to local needs, local market conditions, local regulation and so forth. And Vertica is one of the most important tools, that these folks use in their day to day business activities. So they use Vertica to get insights into how their businesses are going, to deeply into any issues that they want to triage , to generate reports, to plan for the future, a whole lot of use cases. The second big class of users, are in our marketplace team. So marketplace is the engineering team, that backs our ride shared business. And as part of this, running this business, a key problem that they have to solve, is how to determine what prices to set, for particular rides, so that we have a good match between supply and demand. So obviously the real time pricing decisions they're made by serving systems, with very detailed and well crafted machine learning models. However, the training data that goes into this models, the historical trends, the insights that go into building these models, a lot of these things are powered by the data that we store, and serve out of Vertica. Similarly, in each business, we have use cases spanning all the way from engineering and back-end systems, to support operations, incentives, growth, and a whole bunch of other domains. So the big class of applications that we support across a lot of these business lines, is dashboards and reporting. So we have a lot of dashboards, which are built by core data analysts teams and shared with a whole bunch of our operations and other teams. So these are dashboards and reports that run, periodically say once a week or once a day even, depending on the frequency of data that they need. And many of these are powered by the data, and the analytics support that we provide on our Vertica platform. Another big category of use cases is for growth marketing. So this is to understand historical trends, figure out what are various business lines, various customer segments, various geographical areas, doing in terms of growth, where it is necessary for us to reinvest or provide some additional incentives, or marketing support, and so forth. So the analysis that backs a lot of these decisions, is powered by queries running on Vertica. And finally, the heart and soul of Uber is data science. So data science is, how we provide best in class algorithms, pricing, and matching. And a lot of the analysis that goes into, figuring out how to build these systems, how to build the models, how to build the various coefficients and parameters that go into making real time decisions, are based on analysis that data scientists run on Vertica systems. So as you can see, Vertica usage spans a whole bunch of organizations and users, all across the different Uber teams and ecosystems. Just to give you some quick numbers, we have over 5000 weekly active, people who run queries at least once a week, to do some critical business role or problem to solve, that they have in their day to day operations. So next, let's see how Vertica fits into the Uber data ecosystem. So when users open up their apps, and request for a ride or order food delivery on each platform, the apps are talking to our serving systems. And the serving systems use online storage systems, to store the data as the trips and eat orders are getting processed in real time. So for this, we primarily use an in house built, key value storage system called Schemaless, and an open source system called Cassandra. We also have other systems like MySQL and Redis, which we use for storing various bits of data to support serving systems. So all of this operations generates a lot of data, that we then want to process and analyze, and use for our operational improvements. So, we have ingestion systems that periodically pull in data from our serving systems and land them in our data lake. So at Uber a data lake is powered by Hadoop, with files stored on HDFS clusters. So once the raw data lines on the data lake, we then have ETL jobs that process these raw datasets, and generate, modeled and customize datasets which we then use for further analysis. So once these model datasets are available, we load them into our data warehouse, which is entirely powered by Vertica. So then we have a business intelligence layer. So with internal tools, like QueryBuilder, which is a UI interface to write queries, and look at results. And it read over the front-end sites, and Dashbuilder, which is a dash, board building tool, and report management tool. So these are all various tools that we have built within Uber. And these can talk to Vertica and run SQL queries to power, whatever, dashboards and reports that they are supporting. So this is what the data ecosystem looks like at Uber. So why Vertica and what does it really do for us? So it powers insights, that we show on dashboards as folks use, and it also powers reports that we run periodically. But more importantly, we have some core, properties and core feature sets that Vertica provides, which allows us to support many of these use cases, very well and at scale. So let me take a brief tour of what these are. So as I mentioned, Vertica powers Uber's data warehouse. So what this means is that we load our core fact and dimension tables onto Vertica. The core fact tables are all the trips, all the each orders and all these other line items for various businesses from Uber, stored as partitioned tables. So think of having one partition per day, as well as dimension tables like cities, users, riders, career partners and so forth. So we have both these two kinds of datasets, which will load into Vertica. And we have full historical data, all the way since we launched these businesses to today. So that folks can do deeper longitudinal analysis, so they can look at patterns, like how the business has grown from month to month, year to year, the same month, over a year, over multiple years, and so forth. And, the really powerful thing about Vertica, is that most of these queries, you run the deep longitudinal queries, run very, very fast. And that's really why we love Vertica. Because we see query latency P90s. That is 90 percentile of all queries that we run on our platform, typically finish in under a minute. So that's very important for us because Vertica is used, primarily for interactive analytics use cases. And providing SQL query execution times under a minute, is critical for our users and business owners to get the most out of analytics and Big Data platforms. Vertica also provides a few advanced features that we use very heavily. So as you might imagine, at Uber, one of the most important set of use cases we have is around geospatial analytics. In particular, we have some critical internal dashboards, that rely very heavily on being able to restrict datasets by geographic areas, cities, source destination pairs, heat maps, and so forth. And Vertica has a rich array of functions that we use very heavily. We also have, support for custom projections in Vertica. And this really helps us, have very good performance for critical datasets. So for instance, in some of our core fact tables, we have done a lot of query and analysis to figure out, how users run their queries, what kind of columns they use, what combination of columns they use, and what joints they do for typical queries. And then we have laid out our custom projections to maximize performance on these particular dimensions. And the ability to do that through Vertica, is very valuable for us. So we've also had some very successful collaborations, with the Vertica engineering team. About a year and a half back, we had open-sourced a Python Client, that we had built in house to talk to Vertica. We were using this Python Client in our business intelligence layer that I'd shown on the previous slide. And we had open-sourced it after working closely with Eng team. And now Vertica formally supports the Python Client as an open-source project, which you can download to and integrate into your systems. Another more recent example of collaboration is the Vertica Eon mode on GCP. So as most of or at least some of you know, Vertica Eon mode is formally supported on AWS. And at Uber, we were also looking to see if we could run our data infrastructure on GCP. So Vertica team hustled on this, and provided us early preview version, which we've been testing out to see how performance, is impacted by running on the Cloud, and on GCP. And so far, I think things are going pretty well, but we should have some numbers about this very soon. So here I have a visualization of an internal dashboard, that is powered solely by data and queries running on Vertica. So this GIF has sequence have different visualizations supported by this tool. So for instance, here you see a heat map, downgrading heat map of source of traffic demand for ride shares. And then you will see a bunch of arrows here about source destination pairs and the trip lines. And then you can see how demand moves around. So, as the cycles through the various animations, you can basically see all the different kinds of insights, and query shapes that we send to Vertica, which powers this critical business dashboard for our operations teams. All right, so now how do we do all of this at scale? So, we started off with a single Vertica cluster, a few years back. So we had our data lake, the data would land into Vertica. So these are the core fact and dimension tables that I just spoke about. And then Vertica powers queries at our business intelligence layer, right? So this is a very simple, and effective architecture for most use cases. But at Uber scale, we ran into a few problems. So the first issue that we have is that, Uber is a pretty big company at this point, with a lot of users sending almost millions of queries every week. And at that scale, what we began to see was that a single cluster was not able to handle all the query traffic. So for those of you who have done an introductory course, on queueing theory, you will realize that basically, even though you could have all the query is processed through a single serving system. You will tend to see larger and larger queue wait times, as the number of queries pile up. And what this means in practice for end users, is that they are basically just seeing longer and longer query latencies. But even though the actual query execution time on Vertica itself, is probably less than a minute, their query sitting in the queue for a bunch of minutes, and that's the end user perceived latency. So this was a huge problem for us. The second problem we had was that the cluster becomes a single point of failure. Now Vertica can handle single node failures very gracefully, and it can probably also handle like two or three node failures depending on your cluster size and your application. But very soon, you will see that, when you basically have beyond a certain number of failures or nodes in maintenance, then your cluster will probably need to be restarted or you will start seeing some down times due to other issues. So another example of why you would have to have a downtime, is when you're upgrading software in your clusters. So, essentially we're a global company, and we have users all around the world, we really cannot afford to have downtime, even for one hour slot. So that turned out to be a big problem for us. And as I mentioned, we could have hardware issues. So we we might need to upgrade our machines, or we might need to replace storage or memory due to issues with the hardware in there, due to normal wear and tear, or due to abnormal issues. And so because of all of these things, having a single point of failure, having a single cluster was not really practical for us. So the next thing we did, was we set up multiple clusters, right? So we had a bunch of identities clusters, all of which have the same datasets. So then we would basically load data using ingestion pipelines from our data lake, onto each of these clusters. And then the business intelligence layer would be able to query any of these clusters. So this actually solved most of the issues that I pointed out in the previous slide. So we no longer had a single point of failure. Anytime we had to do version upgrades, we would just take off one cluster offline, upgrade the software on it. If we had node failures, we would probably just take out one cluster, if we had to, or we would just have some spare nodes, which would rotate into our production clusters and so forth. However, having multiple clusters, led to a new set of issues. So the first problem was that since we have multiple clusters, you would end up with inconsistent schema. So one of the things to understand about our platform, is that we are an infrastructure team. So we don't actually own or manage any of the data that is served on Vertica clusters. So we have dataset owners and publishers, who manage their own datasets. Now exposing multiple clusters to these dataset owners. Turns out, it's not a great idea, right? Because they are not really aware of, the importance of having consistency of schemas and datasets across different clusters. So over time, what we saw was that the schema for the same tables would basically get out of order, because they were all the updates are not consistently applied on all clusters. Or maybe they were just experimenting some new columns or some new tables in one cluster, but they forgot to delete it, whatever the case might be. We basically ended up in a situation where, we saw a lot of inconsistent schemas, even across some of our core tables in our different clusters. A second issue was, since we had ingestion pipelines that were ingesting data independently into all these clusters, these pipelines could fail independently as well. So what this meant is that if, for instance, the ingestion pipeline into cluster B failed, then the data there would be older than clusters A and C. So, when a query comes in from the BI layer, and if it happens to hit B, you would probably see different results, than you would if you went to a or C. And this was obviously not an ideal situation for our end users, because they would end up seeing slightly inconsistent, slightly different counts. But then that would lead to a bad situation for them where they would not able to fully trust the data that was, and the results and insights that were being returned by the SQL queries and Vertica systems. And then the third problem was, we had a lot of extra replication. So the 20/80 Rule, or maybe even the 90/10 Rule, applies to datasets on our clusters as well. So less than 10% of our datasets, for instance, in 90% of the queries, right? And so it doesn't really make sense for us to replicate all of our data on all the clusters. And so having this set up where we had to do that, was obviously very suboptimal for us. So then what we did, was we basically built some additional systems to solve these problems. So this brings us to our Vertica ecosystem that we have in production today. So on the ingestion side, we built a system called Vertica Data Manager, which basically manages all the ingestion into various clusters. So at this point, people who are managing datasets or dataset owners and publishers, they no longer have to be aware of individual clusters. They just set up their ingestion pipelines with an endpoint in Vertica Data Manager. And the Vertica Data Manager ensures that, all the schemas and data is consistent across all our clusters. And on the query side, we built a proxy layer. So what this ensures is that, when queries come in from the BI layer, the query was forwarded, smartly and with knowledge and data about which cluster up, which clusters are down, which clusters are available, which clusters are loaded, and so forth. So with these two layers of abstraction between our ingestion and our query, we were able to have a very consistent, almost single system view of our entire Vertica deployment. And the third bit, we had put in place, was the data manifest, which were the communication mechanism between ingestion and proxy. So the data manifest basically is a listing of, which tables are available on which clusters, which clusters are up to date, and so forth. So with this ecosystem in place, we were also able to solve the extra replication problem. So now we basically have some big clusters, where all the core tables, and all the tables, in fact, are served. So any query that hits 90%, less so tables, goes to the big clusters. And most of the queries which hit 10% heavily queried important tables, can also be served by many other small clusters, so much more efficient use of resources. So this basically is the view that we have today, of Vertica within Uber, so external to our team, folks, just have an endpoint, where they basically set up their ingestion jobs, and another endpoint where they can forward their Vertica SQL queries. And they are so to a proxy layer. So let's get a little more into details, about each of these layers. So, on the data management side, as I mentioned, we have two kinds of tables. So we have dimension tables. So these tables are updated every cycle, so the list of cities list of drivers, the list of users and so forth. So these change not so frequently, maybe once a day or so. And so we are able to, and since these datasets are not very big, we basically swap them out on every single cycle. Whereas the fact tables, so these are tables which have information about our trips or each orders and so forth. So these are partition. So we have one partition roughly per day, for the last couple of years, and then we have more of a hierarchical partitions set up for older data. So what we do is we load the partitions for the last three days on every cycle. The reason we do that, is because not all our data comes in at the same time. So we have updates for trips, going over the past two or three days, for instance, where people add ratings to their trips, or provide feedback for drivers and so forth. So we want to capture them all in the row corresponding to that particular trip. And so we upload partitions for the last few days to make sure we capture all those updates. And we also update older partitions, if for instance, records were deleted for retention purposes, or GDPR purposes, for instance, or other regulatory reasons. So we do this less frequently, but these are also updated if necessary. So there are endpoints which allow dataset owners to specify what partitions they want to update. And as I mentioned, data is typically managed using a hierarchical partitioning scheme. So in this way, we are able to make sure that, we take advantage of the data being clustered by day, so that we don't have to update all the data at once. So when we are recovering from an cluster event, like a version upgrade or software upgrade, or hardware fix or failure handling, or even when we are adding a new cluster to the system, the data manager takes care of updating the tables, and copying all the new partitions, making sure the schemas are all right. And then we update the data and schema consistency and make sure everything is up to date before we, add this cluster to our serving pool, and the proxy starts sending traffic to it. The second thing that the data manager provides is consistency. So the main thing we do here, is we do atomic updates of our tables and partitions for fact tables using a two-phase commit scheme. So what we do is we load all the new data in temp tables, in all the clusters in phase one. And then when all the clusters give us access signals, then we basically promote them to primary and set them as the main serving tables for incoming queries. We also optimize the load, using Vertica Data Copy. So what this means is earlier, in a parallel pipelines scheme, we had to ingest data individually from HDFS clusters into each of the Vertica clusters. That took a lot of HDFS bandwidth. But using this nice feature that Vertica provides called Vertica Data Copy, we just load it data into one cluster and then much more efficiently copy it, to the other clusters. So this has significantly reduced our ingestion overheads, and speed it up our load process. And as I mentioned as the second phase of the commit, all data is promoted at the same time. Finally, we make sure that all the data is up to date, by doing some checks around the number of rows and various other key signals for freshness and correctness, which we compare with the data in the data lake. So in terms of schema changes, VDM automatically applies these consistently across all the clusters. So first, what we do is we stage these changes to make sure that these are correct. So this catches errors that are trying to do, an incompatible update, like changing a column type or something like that. So we make sure that schema changes are validated. And then we apply them to all clusters atomically again for consistency. And provide a overall consistent view of our data to all our users. So on the proxy side, we have transparent support for, replicated clusters to all our users. So the way we handle that is, as I mentioned, the cluster to table mapping is maintained in the manifest database. And when we have an incoming query, the proxy is able to see which cluster has all the tables in that query, and route the query to the appropriate cluster based on the manifest information. Also the proxy is aware of the health of individual clusters. So if for some reason a cluster is down for maintenance or upgrades, the proxy is aware of this information. And it does the monitoring based on query response and execution times as well. And it uses this information to route queries to healthy clusters, and do some load balancing to ensure that we award hotspots on various clusters. So the key takeaways that I have from the stock, are primarily these. So we started off with single cluster mode on Vertica, and we ran into a bunch of issues around scaling and availability due to cluster downtime. We had then set up a bunch of replicated clusters to handle the scaling and availability issues. Then we run into issues around schema consistency, data staleness, and data replication. So we built an entire ecosystem around Vertica, with abstraction layers around data management and ingestion, and proxy. And with this setup, we were able to enforce consistency and improve storage utilization. So, hopefully this gives you all a brief idea of how we have been able to scale Vertica usage at Uber, and power some of our most business critical and important use cases. So as I mentioned at the beginning, I have a interesting and simple extra update for you. So an easy way in which you all can take advantage of many of the features that we have built into our ecosystem, is to use the Vertica Eon mode. So the Vertica Eon mode, allows you to set up multiple clusters with consistent data updates, and set them up at various different sizes to handle different query loads. And it automatically handles many of these issues that I mentioned in our ecosystem. So do check it out. We've also been, trying it out on DCP, and initial results look very, very promising. So thank you all for joining me on this talk today. I hope you guys learned something new. And hopefully you took away something that you can also apply to your systems. We have a few more time for some questions. So I'll pause for now and take any questions.

>> Announcer: Live from San Francisco, it's theCUBE covering Flink Forward, brought to you by data Artisans. (upbeat music) >> This is George Gilbert. We are at Flink Forward, the user conference for the Apache Flink community, sponsored by data Artisans, the company behind Flink. And we are here with Shuyi Chen from Uber, and Shuyi works on a very important project which is the Calcite Query Optimizer, SQL Query Optimizer, that's used in Apache Flink as well as several other projects. Why don't we start with, Shuyi tell us where Calcite's used and its role. >> Calcite is basically used in the Flink Table and SQL API, as the SQL POSSTR and query optimizer in planner for Flink. >> OK. >> Yeah. >> So now let's go to Uber and talk about the pipeline or pipelines you guys have been building and then how you've been using Flink and Calcite to enable the SQL API and the Table API. What workloads are you putting on that platform, or on that pipeline? >> Yeah, so basically I'm the technical lead of the streaming platform, processing platform in Uber, and so we use Apache Flink as the stream processing engine for Uber. Basically we build two different platforms one is the, called AthenaX, which use Flink SQL. So basically enable user to use SQL to compose the stream processing logic. And we have a UI, and with one click, they can just deploy the stream processing job in production. >> When you say UI, did you build a custom UI to take essentially, turn it a business intelligence tool so you have a visual way of constructing your queries? Is that what you're describing, or? >> Yeah, so it's similar to how you compose your, write a SQL query to query database. We have a UI for you to write your SQL query, with all the syntax highlight and all the hint. To write a SQL query so that, even the data scientists and also non engineers in general can actually use that UI to compose stream processing lock jobs. >> Okay, give us an example of some applications 'cause this sounds like it's a high-level API so it makes it more accessible to a wider audience. So what are some of the things they build? >> So for example, in our Uber Eats team, they use the SQL API to, as the stream processing tool to build their Restaurant Manager Dashboard. Restaurant Manager Dashboard. >> Okay. >> So basically, the data log lives in Kafka, get real-time stream into the Flink job, which it's composed using the SQL API and then that got stored in our lab database, P notes, then when the restaurant owners opens the Restaurant Manager, they will see the dashboard of their real-time earnings and everything. And with the SQL API, they no longer need to write the Flink job, they don't need to use Java or skala code, or do any testing or debugging, It's all SQL, so they, yeah. >> And then what's the SQL coverage, the SQL semantics that are implemented in the current Calcite engine? >> So it's about basic transformation, projection, and window hopping and tumbling window and also drawing, and group eye, and having, and also not to mention about the event time and real time, processing time support. >> And you can shuffle from anywhere, you don't have to have two partitions with the same join key on one node. You can have arbitrary, the data placement can be arbitrary for the partitions? >> Well the SQL is the collective, right? And so once the user compose the logic the underlying panel will actually take care of how the key by and group by, everything. >> Okay, 'cause the reason I ask is many of the early Hadoop based MPP sequel engines had the limitation where you had to co-locate the partitions that you were going to join. >> That's the same thing for Flink. >> Oh. >> But it just the SQL part is just take care of that. >> Okay. >> So you do describe what you do, but underlying get translated into a Flink program that actually will do all the co-location. >> Oh it redoes it for you, okay >> Yeah, yeah. So now they don't even need to learn Flink, they just need to learn the SQL, yeah. >> Now you said there a second platform that Uber is building on top of Flink. >> Yeah, the second platform is the, we call it the Flink as a service platform. So the motivation is, we found that SQL actually cannot satisfy all the advanced need in Uber to build stream processing, due to the reason, like for example, they will need to call up RPC services within their stream processing application or even training the RCP call, so which is hard to express in SQL and also when they are having a complicated DAG, like a workflow, it's very difficult to debug individual stages, so they want the control to actually to use delative Flink data stream APL dataset API to build their stream of batch job. >> Is the dataset API the lowest level one? >> No it's on the same level with the data stream, so it's one for streaming, one for batch. >> Okay, data stream and then the other was table? >> Dataset. >> Oh dataset, data stream, data set. >> Yeah. >> And there's one lower than that right? >> Yeah, there's one lower API but it's usually, most people don't use that API. >> So that's system programmers? >> Yeah, yeah. >> So then tell me, who is using, like what type of programmer uses the data stream or the data set API, and what do they build at Uber? >> So for example, in one of the talk later, there's a marketplace team, marketplace dynamics team, it's actually using the platform to do online model update, machinery model update, using Flink, and so basically they need to take in the model that is trained offline and do a few group by, time and location and then apply the model, and then incrementally update the model. >> And so are they taking a window of updates and then updating the model and then somehow promoting it as the candidate or, >> Yeah, yeah, yeah. Something similar, yeah. >> Okay, that's interesting. And what type of, so are these the data scientists who are using this API? >> Well data scientists are not really, it's not designed for data scientists. >> Oh so they're just going the models off, they're preparing the models offline and then they're being updated in line on the stream processing platform. >> Yes. >> And so it's maybe, data engineers who are essentially updating the features that get fed in and are continually training, or updating the models. >> Basically it's a online model update. So as Kafka event comes in, continue to refine the model. >> Okay, and so as Uber looks out couple years, what sorts of things do you see adding to one of these, either of these pipelines, and do you see a shift away from the batch and request response type workloads towards more continuous processing. >> Yes actually there we do see that trend, actually, before becoming entirely of stream processing platform team in Uber, I was in marketplace as well and at that point we always see there's a shift, like people would love to use stream processing technology to actually replace some of the normal backhand service applications. >> Tell me some examples. >> Yeah, for example... So in our dispatch platform, we have the need to actually shard the workload by, for example, writers, to different hosts to process. For example, compute say ETA or compute some of the time average, and this is before done in back hand services and say use our internal distribution system things to do the sharding. But actually with Flink, this can be just done very easily, right. And so actually there's a shift, those people will also want to adopt stream processing technology and, so long as this is not a request response style application. >> So the key thing, just to make sure I understand it's that Flink can take care of the distributed joins, whereas when it was a data base based workload, DBA had to set up the sharding and now it's sort of more transparent like it's more automated? >> I think, it's... More of the support, so if before people writing backhand services they have to write everything: the state management, the sharding, and everything, they need to-- >> George: Oh it's not even data base based-- >> Yeah, it's not data base, it's real time. >> So they have to do the physical data management, and Flink takes care of that now? >> Yeah, yeah. >> Oh got it, got it. >> For some of the application it's real time so we don't really need to store the data all the time in the database, So it's usually keep in memory and somehow gets snapshot, But we have, for normal backhand service writer they have to do everything. But with Flink it has already built in support for state management and all the sharding, partitioning and the time window, aggregation primitive, and it's all built in and they don't need to worry about re-implement the logic and we architect the system again and again. >> So it's a new platform for real time it gives you a whole lot of services, higher abstraction for real time applications. >> Yeah, yeah. >> Okay. Alright with that, Shuyi we're going to have to call it a day. This was Shuyi Chen from Uber talking about how they're building more and more of their real time platforms on Apache Flink and using a whole bunch of services to complement it. We are at Flink Forward, the user conference of data Artisans for the Apache Flink community, we're in San Francisco, this is the second Flink Forward conference and we'll be back in a couple minutes, thanks. (upbeat music)

>> Announcer: Live from Stanford University in Palo Alto, California, it's theCUBE! Covering Women In Data Science Conference 2018. Brought to you by-- >> Coverage of Women in Data Science 2018. I am Lisa Martin. We're at Stanford University. This is where the big in-person event is, but there are more than 177 regional WiDS events going on around the globe today. They are in 53 countries, and they're actually expecting to have about 100,000 people engaged with WiDS 2018. Pretty awesome. I'm joined by one of the speakers for WiDS 2018, Dawn Woodard, the senior data science manager of maps at Uber. Welcome to theCUBE! >> Thank you so much, Lisa. >> It's exciting to have you here. This is your first WiDS, and you are already a speaker. Tell us a little bit about what attracted you to WiDS. What was it that kind of spoke to you as a female leader in data science? >> Well, I tried to do a fair amount of reach-out to women in data science. I really feel like I've been blessed throughout my career with inspiring female mentors, including my mother, for example. Not every woman comes into her career with that kind of mentorship, so I really wanted to reach out and help provide that to some of the younger folks in our community. >> That's fantastic. One of the things that's remarkable about WiDS, one, is the growth and scale that they've achieved reaching such big, broad audiences in such a short time period. But it's also from a thematic perspective, aiming to inspire and to educate data scientists worldwide, and of course, to support females in that. What are some of the, tell us a little bit about your talk is Dynamic Pricing and Matching in Ride Sharing. What are some of the takeaways that the audience watching the livestream and here in person are going to hear from your talk? >> There are two technical takeaways, and then there's one non-technical takeaway. The first technical takeaway is that the matching algorithms that we use are really designed to reduce the amount of time that riders and drivers have to spend waiting in the app. For drivers, that means that we're working to increase the amount of time that they spend on-trip and getting paid. For riders, that means that we're reducing the amount of time that they have to wait to be picked up by a car. That's the first takeaway. The second takeaway is around dynamic pricing, and why it's important in ride-hailing services in particular. It turns out that it's really important in creating a seamless and reliable experience, both for riders and for drivers, so I talk through the technical reasons for that. Interestingly, these technical arguments are based not just on machine learning and statistics, but also on economic analyses and some optimization concepts. The third takeaway is really that data science is this incredibly interdisciplinary environment in which we have economics, statistics, optimization, machine learning, and more. >> It's really, data sciences has the opportunity, or really is, very horizontal. Every sector, every area of our lives is impacted by it. I mean, we think of all of us that use Uber and ride-sharing apps. I think that's one of the neat things that we're hearing from the event and from the speakers like yourself is these demarcated lines of career paths are blurring, or some of 'em are evaporating. And so, I think having the opportunity to talk to the younger generation, showing them how much impact they can make in this field has got to sort of be maybe, I would even guess, invigorating for you, as someone who's been in the tech in both industry and academia for a while. >> Absolutely. I think about data science as being the way that we learn about the world, statistics and data science. So, how do we use data to learn about the world, and how do we use data to improve, to make great products, to make great apps, for example. >> Exactly. Tell me a little bit about your career path. You have your PhD in statistics from Duke University. Tell me about how you got there, and then how you also got into industry. Were you always a STEM fan as a kid, or was it something that you had a passion for early on, or developed over time? >> I was always passionate about math and science. When I was an undergraduate, I did an internship with a defense contractor. That's how I got interested in machine learning in particular. That's where it took off. I decided to get a PhD in statistics from there. Statistics and machine learning are really closely related. And then, continued down that path throughout my academic career, and now my career in tech. >> What are some of the things that you think that prepared you for a being a female leader? Was it those mentors that you mentioned before? Was it the fact that you just had a passion for it and thought, "If I'm one of the only females in the room, I don't care. "This is something that's interesting to me." What were some of those foundational elements that really guided you? >> One is the inspiration of some women in my life, and if we have to be completely honest, I'm a person who, when, the very rare times in my career when somebody has acted like I couldn't hack it or couldn't make it, it always really got me angry. The way that I channeled that was really to turn it around and to say, "No problem. "I'm going to show you that I can go well beyond "anything that you had conceived of." >> You know, I love that you said that, 'cause Margot Gerritsen, one of the founders of WiDS actually said a couple hours ago, a few years ago, when they had this idea, from concept to first conference was six months, and she said she almost thought of it like a revenge conference. Like, "We can do this!" I think it's kind of, when they had this idea in 2015, the fact that even in 2015, there's still not only demand for, but the demand is growing. As we're seeing, the statistics that show a low percentage of women that have degrees in engineering, I want to say 20%, but only 11% of them are actually working in their field. We still have a lot of work to do to ignite the fire in this next generation of prospective leaders in technology. There's still a lot of groundwork to make up there. I think we're hearing that a lot at WiDS. Are you hearing that in your peer groups as well? >> Absolutely. I think one of the things that I've really focused on is mentoring women as leaders and managers within my organization, and I really find that that's an amazing way to reach out, is not just to reach out myself, but also to do that through female leaders in my own organization. For example, I've mentored and managed two women through the transition from individual contributor to manager. Just watching their trajectory afterwards is incredibly inspiring. But then, of course, those female managers bring in additional female contributors, and it grows from there. >> Right. And you have a pretty good, pretty diverse team at Uber. Tell us a little bit about your rise at Uber. One of the things that I saw on your LinkedIn profile, that you achieved pretty quickly in the first three years, or probably less, was that you led the marketplace data science team through a period of transformative growth. You started that team with 10 data scientists, and by the time you transitioned into your next role, there were 49 data scientists, including seven managers. How were you able to come in and make such a big impact so quickly? >> Well, the whole team chipped in in terms of hiring and reaching out. But at the time when I joined Uber, data science was still relatively small. Those 10 people were being asked to do all of the pricing and matching algorithms, all of the data science for Uber Pool, all of the data science for Uber Eats. We just had one person in each of these areas, and those people very quickly stepped up to the plate and said, "Okay, I need help." We worked together to help grow their teams. It's really a collaborative effort involving the whole team. >> The current team that you're managing, what does that look like from a male/female ratio standpoint? >> The current team is more than 50% female at this point, which is something that I'm really proud of. It's definitely not only my achievement. There was a manager who was leading the team just before I switched to leading maps, and that person also helped increase the presence of women in data science for Uber's mapping organization. The first data scientist on maps at Uber was a woman, actually. >> That's fantastic. And you were saying before we went live that there's a good-sized contingent of women data scientists at Uber today that are participating in WiDS up in San Francisco? >> That's right, yes. We're live-streaming it. There's a Women in Data Science organization at Uber, and that organization is sponsoring the internal events for the live stream, not just for my talk, but really, the whole conference. >> That's one of the things that Margot Gerritsen was also saying, that from a timing perspective, they really knew they were on to something pretty quickly, and being able to take advantage of technology, live streaming, they're also doing it on Facebook, gives them that opportunity to reach a bigger audience. It also is, for you and your peers as speakers, gives you an even bigger platform to be able to reach that audience. But one of the things I find interesting about WiDS is it's not just the younger audience. Like Maria Klawe had said in her opening remarks this morning and before, that the optimal time that she's found of reaching women to get them interested in STEM subjects is first year college, first semester of college. I actually had the same exact experience many years ago, and I didn't realize that was a timing that was actually proven to be the most successful. But it's not just young women at that stage of their university career. It's also those who've been in tech, academia, and industry for a while who, we're hearing, are feeling invigorated by events like WiDS. Do you feel the same? Is this something that just sort of turns up that bunsen burner maybe a little bit higher? >> Oh, it's incredibly empowering to be in a room full of such technically powerful women. It's a wonderful opportunity. >> It really is, and I think that reinvigoration is key. Some of the things like, as we look at what you've already achieved at Uber so far, and we're in 2018, what are some of the things that you're looking forward to your team helping to impact for Uber in 2018? >> In 2018, we're looking to magnify the impact of data science within Uber's mapping organization, which is my main focus right now. Maps at Uber does several things. Think of Uber as being a physical logistics platform. We move people and things from point A to point B. Maps, as our physical world, really impacts every aspect of the user experience, both for riders and for drivers. And then, whenever we're making a dispatch decision or a pricing decision, we need to know something about how long it would take this driver to get to this rider, for example, which is really a mapping prediction. We are looking at increasing the presence of data science within the mapping organization, really bringing that perspective to the table, both at the individual contributor level, but really also growing leadership of data science within the mapping organization so that we can help drive the direction of maps at Uber through data-driven insights. >> Data-driven insights, I'm glad that you brought that up. That's something that, as we talk about data science. Data science is helping to make decisions on policy, healthcare, so many different things, you name it. It really seems like these blurred lines of job categories, as businesses use data science, and even Uber, to extend, grow the business, open new business models, so can the next generation leverage data science to just open up this infinite box, if you will, of careers that they can go into and industries they can impact by having this foundation of data science. >> Absolutely. Well, any time we have to make a decision about what direction we go in, right, as a business, for example, as an organization, then doing that starting from data, understanding what is the world really like, what are the opportunities, what are the places in which we as a company are not doing very well, for example, and can make a simple change and get an incredible impact? Those are incredibly powerful insights. What do you think, last question-ish, 'cause we're getting low on time. We talk a lot about, there's the hard skills/soft skills. Soft is kind of a weird word these days to describe that. You know, statistical analysis, data mining. But there's also this, the softer skills, empathy, things like that. How do you find those two sides, maybe it's right brain/left brain, as being essential for people to become well-rounded data scientists? >> The couple of soft skills that I really look for heavily when I'm hiring a data scientist, one is being really focused on impact, as opposed to focused on building a new shiny thing. That's quite a different approach to the world, and if we stay focused on the product that we're creating, that means that we're willing to chip in, even if the work that's being done is not as glamorous, or is not going to get as much attention, or is not as fancy of a model. We can really stay focused on what are some simple approaches that we can use that can really drive the product forward. That kind of impact focus, and also, that great attitude about being willing to chip in on something, even if it's not that fancy or if I'm not going to get in the limelight for doing this. Those are the kinds of soft skills that really are so critical for us. >> Attitude and impact. I've heard impact a number of times today. Dawn, thank you so much for carving out some time to chat with us on theCUBE. We congratulate you on being a speaker at this year's event, and look forward to talking to you next year. >> Thank you, Lisa. >> We want to thank you for watching theCUBE. We are live at Stanford for the third annual Women in Data Science Conference, hashtag #WiDS2018. Get involved in the conversation. It is happening in over 53 countries. After this short break, I will be right back with my next guest. (fast electronic music)

>> Narrator: From San Francisco, it's theCUBE. Covering Samsung Developer Conference 2017. Brought to you by Samsung. (cheerful music) >> Welcome back everyone, live here from San Fransisco, Moscone West, at the Samsung Developer Conference, 2017 #SDC2017. I'm John Furrier, cohost of theCUBE, co-founder of Silicon Angle Media. My next guest, Hilary Karls, Senior Software Engineer at Uber. Great to have you on, welcome to theCUBE. >> Thank you for having me. >> Great to have a female engineer for Uber, pun intended. In all seriousness, thanks for coming out, appreciate it. >> Oh, my pleasure. >> Love talking tech here; Samsung obviously is huge with Android, that's what you do. What do you do with Uber? Let's get that out on the table. What's your role at Uber as an engineer? >> I'm an Android engineer, I work on UberEATS, which is our food delivery product. I've been at Uber for about three years now. So I've gotten to see UberEATS grow from just a tiny operations experiment, in LA, all the way up to being in over 120 cities and having its own app. >> Uber's got that great culture; trying new things. Love the iteration and our family uses the competitor, DoorDash, and I can't wait for UberEATS to come to Palo Alto. Is it in Palo Alto yet? >> Yeah, it is. >> I better try it out. Bring me a discount, can you send me a little discount code? >> Yeah, I can definitely send you a discount code. >> Okay good, got that out of the way. Got my free phone here from Samsung, as well. Thank you Samsung for the free phone. So let's talk about Samsung, obviously this is a developer's conference, you're on the front line, building the app on the Android side. >> Yeah >> Google was on stage yesterday so there it is, it's all out there, Samsung innovating with the handset. Great reviews on the new 8, great screen. This is an Apple-esque going on here. They're there, they got an iOT strategy, so they're bringing together lifestyle. >> Yeah. >> Okay, this is the new user experience. Okay, so has Android gone up? They got Bixby. >> They got Bixby, yeah. >> What's your experience with Android and developing in the environment? >> Well I've been developing on Android for about four or five years now, and it's wonderful. It's open, it enables you to actually go and build. Using more of the APIs and having more ability to connect into the phone itself. To be honest, I love it. It's one of those where you actually get to take your ideas and ship it out to a much broader range of people. >> Build things that are still very very beautiful and very usable to people. >> I was talking with Sam Ramji, who's heading up Cloud product management at Google, as well as, developer programs across Google. We were talking at VM World, and I know their conference is more Cloud oriented. >> Hmm Hmm >> Talking about the user experience and Google is hyper-focused on building open-sourced libraries. >> Hmm Hmm >> So Google is going to be contributing all this goodness, you got the tensor flow, you got all this innovations going on at Google, that's translating to the edge of the network, we're seeing that with Samsung. House devices are great, you got TVs that are known for that, but connecting the smart things is really their strategy. As a developer, how do you look at that landscape, because you're looking at now tons of open source goodness coming in, from Google and then the communities, Android leveling up on capability. >> Hmm hmm >> And with the open APIs, where do you try to navigate, how do you extract the signal from the noise in the community? >> Well basically, you look for whatever libraries are actually used very heavily by people that have latest commands. If it comes from something like from Google, or from other really strong reputed open source players, like Square, you'll end up seeing adoption of that a lot more. When you then go into internet of things, it becomes also like how you see community - the chicken and egg thing. Is this the best opportunity to go for right now? Is the community there, can I monetize this? How do you integrate it? >> Yeah, that's the big question, what's the integration? So let me ask you a question on the Android. In your opinion, Android, Apple IOS, leveled up, in terms of capability? >> Pretty much, yeah. >> What's the difference between IOS and the Android, if you could explain to someone that's knocking on the door of both developer communities. >> Well basically, I would go for who you were trying to go for first. Are you trying to target the U.S, and trying to target that higher end of consumer? Or are you trying to have a mass appeal? For a lot of small start-ups, I'd actually say go IOS first, it makes sense. But when you want to bring it to the global market that's when you really, really, really need Android. When you want to bring it to places around the world where there are more Androids than IOS, that's definitely something that just needs that core experience. >> Is there anything that Android has, because it seems to me that Android has a lot more unique opportunities to customize. >> Yes. >> I think that seems to appeal to developers. When I look at our data, we look at our audience, I'll see the general purpose, I call general purpose Apple because Apple is like fine jewelry. Everyone wants to have the high-end suits, and the high-end code, whatever. But, you don't have the flexibility because it's kind of a walled garden from Java's standpoint. You have the open garden with the Android, that community, their playing more in open source, you're seeing block chain, you're seeing all these kinds of cool communities. What are some of the things that Android offers in this open approach that you like? Is there things that jump out at you? >> Well a lot of it's the ability to get customized, to have better security controls, to have widgets on your home screen. From a developer's standpoint, to have more flexibility about when you ask for permissions vs. when you don't. Also, the richness of the notifications, we've had that for a while on Android. It's like you can do actions and have things animate and keep things like progress bars on there for quite some time and IOS is finally catching up to that. >> And you're happy with Android? >> Oh, very much so. >> Alright, so what's your take on this show at Samsung? What's your vibe on the show, feeling, they got some good stuff? When do they need to work on things? >> Ummm. >> Give me the positive review and then give me the critical what they can do better? >> Yeah, I think actually bringing everything together in a connected cloud makes a lot of sense. Developers don't really like fragmented APIs. If you have to actually go talk to five or six different services to get something done, that's a lot more effort and a lot more overhead of understanding of how to connect to each one. So bringing it all together in one place, especially if that can have the overhead of, handle the overhead, making it secure, and doing data storage, and all that stuff, makes a lot of sense. >> Awesome. Hilary, final question for ya, personal question, put you on the spot here. What's the coolest thing you've built this past couple of years? That you coded, could be anything. >> Ohhh. >> What's the coolest thing that you feel >> Personally? >> Yeah. >> Alright, so I guess it was a year and a half now, there was a team of four of us that build the new Android Eats app. Built it from scratch in about three and a half months It was a team that was magical and never happens in Silicon Valley. It was three women and a latino dude, and we shipped it. >> No way. >> Yeah. >> Three months? >> Yeah, three and a half months. Starting in September, shipping early. >> We're talking, on the drawing board, on the design, >> Yeah. Literally, start to end. First line of code in early September, end of code writing, like wrapping and shipping it up right before Christmas. >> That's the new format, three women and a latino dude. (laughing) Congratulations, anything you'd like to share with the audience about the development scene here, what you're thinking about, what you're passionate about? >> I really really loved the piece of using technology to build more human connection, better humanity. I thought that was amazing; that piece was really inspiring to me. >> This is the new lifestyle tech is coming. The consumerization, blending analog, and digital together. This is the new normal from fashion tech, to UberEats. Congratulations, thanks for coming on theCUBE, Hilary Karls, Senior Software Engineer at Uber. This is theCUBE's live coverage of Samsung Developer Conference 2017, in San Francisco. I'm John Furrier, we'll be back with more after this short break. (cheerful music)

>> Welcome to the latest edition of the MIT Sloan Expert Series. I'm your host, Rebecca Knight. Our topic today is racial bias in the sharing economy, how Uber and Lyft are failing black passengers, and what to do about it. Here to talk about that is Chris Knittel. He is a professor of Applied Economics here at MIT Sloan, and he's also the co-author of a study that shows how Uber and Lyft drivers discriminate based on a passenger's skin color. Thanks so much for joining us. >> Oh, it's great to be here. >> Before we begin, I want to remind our viewers that we will be taking your questions live on social media. Please use the hashtag MITSloanExpert to pose your questions on Twitter. Chris, let's get started. >> Chris: Sure. So there is a lot of research that shows how difficult it is to hail a cab, particularly for black people. Uber and Lyft were supposed to represent a more egalitarian travel option, but you didn't find that. >> That's right, so what we found in two experiments that we ran, and one in Seattle, and one in Boston, is that Uber and Lyft drivers were discriminating based on race. >> Rebecca: We've already seen, actually some evidence of racial discrimination in the sharing economy, not just with ride sharing apps. >> Sure, so there's evidence for Airbnb. And what's interesting about Airbnb actually, is that discrimination is two-sided. So not only do white renters of properties not want to rent to black rentees, but white renters do not stay at a home of a black home owner. >> Did your findings and the findings of that other research you just talked about, does it make you discouraged? >> Partly, I was an optimist. We went into this, at least I went into this hoping that we wouldn't find discrimination, but one thing that has helped, or at least shined a more positive light, is that there are ways that we can do better in this sector. >> You've talked about this study, which you undertook with colleagues from the University of Washington and Stanford, shows the power of the experiment. Can you talk a little bit about what you mean by that? >> Sure, what we did was actually run two randomized control trials. Just like you would test whether a blood pressure medication works, so you would have a control group that wouldn't get the medication, and a treatment group that would. We did the same thing where we sent out in Seattle both black and white RAs that hailed Uber and Lyft rides, and we randomized whether or not it was a black RA calling the ride or a white RA that particular time, and they all drove the same exact route at the same exact times of the day. >> So what did you find? Let's talk about first, what you found in Seattle. >> Sure, so in Seattle, we measured how long it took for a ride to be accepted, and also, how long it took, once it was accepted, for the driver to show up and pick up the passenger. And what we found is, if you're a black research assistant, that in hailing an Uber ride, it took 30 percent longer for a ride to be accepted, and also 30 percent longer for the driver to show up and pick you up. >> 30 percent seems substantial. >> Well, for the time it takes to accept the ride, we're talking seconds, but for the time it takes for a passenger to actually be picked up, it's over a minute longer. And I'll mention also for Lyft, we found a 30 percent increase in the amount of time it took to be accepted, but there was no statistically significant impact on how long it took for the driver to actually show up. >> So, the thing about the minute difference, that can be material, particularly if you're trying to catch a cab, an Uber or a Lyft for a job interview or to get to the airport. >> Yeah, this is introspection, but I always seem to be late, so even a minute can be very costly. >> I hear you, I hear you. So why do you think there was the difference between Lyft and Uber? >> What's interesting, and we learned this while we were doing the experiment, a Lyft driver sees the name of the passenger before they accept the ride, whereas an Uber driver only sees the name after they've accepted. So in order for an Uber driver to discriminate, they have to first accept the ride, and then see the name and then cancel, whereas a Lyft driver can just pass it up right away. So it turns out because of that, the Lyft platform is more easily capable of handling discrimination because it pushed it to another driver faster than the Uber platform. >> I want to come back to that, but I want to say also, that difference caused you to change the way you did the experiment in Boston. >> In Boston, a couple differences. One is that we sent out RAs with two cell phones actually. So each RA had an Uber and Lyft account under a stereotypically white sounding name, and then also an Uber and Lyft account under a stereotypically black sounding name. That was one difference, and then also, what we measured in Boston that we didn't measure in Seattle, is cancellations. So an Uber driver accepts the ride, and then cancels on the RA. >> Let's go back to the stereotypically black sounding name verses white sounding name. You're an economist, how did you determine what those names are? >> We relied on another published paper that actually looked at birth records from the 1970s in Boston, and the birth records tell you not only the name, but also the race of the baby. So they found names that actually 100 percent of the time were African American or 100 percent of the time were not African American. So we relied on those names. >> And the names were... >> So you could imagine Jamal for example, compared to Jerry. >> Alright, Ayisha and Alison. >> Chris: Sure. >> So what was your headline finding in Boston? >> In Boston, what we found is, if you were a black male calling an Uber ride, that you were canceled upon more than twice as often as if you were a white male. >> And what about Lyft? >> For Lyft, there is no cancellation effect, and that's not because there's no discrimination, it's just that they don't have to accept and then cancel the ride, they can just pass up the ride completely. It's actually a nice little experiment within the experiment, we shouldn't find an effect of names on cancellations for Lyft and in fact, we don't. >> And also, the driver network is much thicker in Boston than in Seattle. >> So in Boston, although we found this cancellation effect, we didn't find that it has a measurable impact on how long you wait. And this is somewhat speculation, but we speculate that that's because the driver network is so much more dense in Boston that, although you were canceled upon, there's so many only drivers nearby, that it doesn't lead to a longer wait time. >> How do you think what you found compares to hailing traditional cabs? We started our conversation talking about the vast body of research that shows how difficult it is for black people to hail cabs. >> Yeah, we are quick to point out that we are not at all saying that Uber and Lyft are worse than traditional, status quo system, and we want to definitely make that clear. In fact, in Seattle, we had our same research assistants stand at the busiest corners and hail cabs. What we found there is, if you were a black research assistant, the first cab passed you 80 percent of the time. But if you were a white research assistant, it only passed you 20 percent of the time. So just like the previous literature has found, we found discrimination with the status quo system as well. >> You've talked to the companies about you findings, what has the response been? >> That's been actually heartening. Both companies reached out to us very quickly, and we've had continued conversations with them, and we're actually trying to design followup studies to minimize the amount of discrimination that's occurring for both Uber and Lyft. >> But those are off the record and... >> Right, we're not talking specifics, but what I can say is that the companies understand this research and they definitely want to do better. >> In fact, the companies both have issued statements about this, the first one is from Lyft, "we are extremely proud of the positive impact..." Uber has also responded. So let's talk about solutions to this. What do you and your colleagues who undertook this research suggest? >> We've been brainstorming, we don't know for sure if we have the silver bullet, but a few things could change, for example, you could imagine Uber and Lyft getting rid of names completely. We realize that has a trade off in the sense that it's nice to know the name of the driver... >> Rebecca: Sure, you can strike up a conversation... >> It makes it more social, it makes it more personal, more peer to peer if you will. But it would eliminate the type of discrimination that we uncovered. Another potential change is to delay when you give the name to the driver, so that the driver has to commit more to the ride than he or she previously had to. And that may increase the costs of discrimination. >> So that would be changing the software? >> Right, so you could imagine now, like I said, with Lyft that you see the name right away. Maybe you wait until they're 30 seconds away from the passenger before you give them the name. >> What about the dawn of the age of autonomous vehicles? Might that have an impact? We already know that Uber is experimenting with driverless cars in Pittsburgh and Arizona. >> That would obviously solve it, so that would take the human element out of things, and it's important to point out that these are the drivers that are deciding to discriminate. So provided you didn't write the autonomous vehicle software to discriminate, you would know for sure that that car is not going to discriminate. >> What about a driver education campaign? Do you think that would make a difference? I'm reminded of an essay written by Doug Glanville, who is an ESPN commentator and former pro ball player. He writes, on talking about his experience being denied service by an Uber driver, "the driver had concluded I was a threat, "either because I was dangerous myself, "or because I would direct him to a bad neighborhood, "or give him a lower tip, either way, "given the circumstances, it was hard "to attribute his refusal to anything other than my race. "Shortly after we walked away, I saw the driver assisting "another passenger who was white." >> We all hope that information helps, and eliminates discrimination. It's certainly possible that Uber and Lyft could have a full information campaign, where they show the tip rates for different ethnicities, they show the bad ride probabilities for different ethnicities, and my hope is that once the drivers learn that there aren't differences across ethnicities, that the drivers would internalize that, and stop discriminating. >> Policy, Senator Al Franken has weighed in on this, urging Uber and Lyft to address your research. Do you think that there could be policies too? Does government have a role to play? >> Potentially, but what I'll say again is, that Uber and Lyft, I think have all the incentive in the world to fix this, and that they seem to be taking active steps to fixing this. So what could policy makers do? They can, obviously it's already outlawed. They could come down and potentially fine the companies if there's more evidence of discrimination. But I would at least allow the companies some time to internalize this research, and respond to it, and see how effective they can be. >> Many, many think tanks and government advocacy groups have weighed in too. The MIT Sloan Expert Series recently sat down with Eva Millona of the Massachusetts Immigrant and Refugee Coalition. She will talk about this research in the context of immigration, let's roll that clip. >> We're an advocacy organization, and we represent the interest of foreign born, and our mission is to promote and enhance immigrant and refugee integration. Anecdotally, yes, I would say that the research, and given the impressive sample of the research really leads to a sad belief that discrimination is still out there, and there is a lot that needs to be done across sectors to really address these issues. We are really privileged to live in such a fantastic commonwealth with the right leadership and all sectors together, really making our commonwealth a welcoming place. And I do want to highlight the fantastic role of our Attorney General for standing up for our values, but Massachusetts is one state, and it could be an example, but the concern is nation wide. Given a very divisive campaign, and also not just a campaign, but also, what is currently happening at the national level that the current administration is really rejecting this welcoming effort, and the values of our country as a country, who welcomes immigrants. All sectors need to be involved in an effort to really make our society a better one for everyone. And it's going to take political leadership to really set the right tone, send the right message, and really look into the integration, and the welcoming of the newcomers as an investment in our future of our nation. Uber and Lyft have an opportunity here to provide leadership and come up with promotion of policies that integrate the newcomers, or that are welcoming to the newcomers, provide education and training, and train their people. And as troubling as the result of this research are, we like to believe that this is the attitude of the drivers, but not really what the corporate represents, so we see an opportunity for the corporate to really step in and work and promote policies of integration, policies of improvement and betterment for the whole of society and provide an example. Let me say thank you to Professor Knittle for his leadership and MIT for always being a leader, and looking into these issues. But if we can go deeper into A, the size, B, the geography, but also looking into a wider range of all communities that are represented. Looking into the Latino community, looking into the Arab communities in other parts of the nation in a more rigorous, more deep and larger size of research will be very helpful in terms of promoting better policies and integration for everybody who chooses America to be their home. >> That was Eva Millona of the Massechusetts Immigrant and Refugee Advocacy Coalition. Chris, are you confident this problem can in fact be remedied? >> I think we can do better, for sure. And I would say we need more studies like what we just preformed to see how widespread it is. We only studied two cities, we also haven't looked at all at how the driver's race impacts the discrimination. >> Now we're going to turn to you, questions from our viewers. Questions have already been coming in this morning and overnight, lots of great ones. Please use the hashtag MITSloanExpert to pose your question. The first one comes from Justin Wang, who is an MIT Sloan MBA student. He asks, "what policies can sharing economy startups "implement to reduce racial bias?" >> Well, I would say the first thing is to be aware of this. I think Uber and Lyft and Airbnb potentially were caught off guard with the amount of discrimination that was taking place. So the research that we preformed, and the research on Airbnb gives new startups a head start on designing their platforms. >> Just knowing that this is an issue. >> Knowing it's an issue, and potentially designing their platforms to think of ways to limit the amount of discrimination. >> Another question, did you look at gender bias? Do you have any indication that drivers discriminate based on gender? >> We did look at gender bias. The experiments weren't set up to necessarily nail that, but one thing that we found, for example in Boston, is that there is some evidence that women drivers were taken on longer trips. Again, both the male and the female RAs are going from the same point A to the same point B. >> Rebecca: That was a controlled part of the setting. >> That was the controlled part of the experiment. And we found evidence that women passengers were taken on longer trips and in fact, one of our RAs commented that she remembers going through the same intersection three times before she finally said something to the driver. >> And you think... So you didn't necessarily study this as part of it, but do you have any speculation, conjecture about why this was happening? >> Well, there's two potential motives. One is a financial motive that, by taking the passenger on a longer drive. They potentially get a higher fare. But I've heard anecdotal evidence that a more social motive might also be at play. For example, I have a colleague here at Sloan, who's told me that she's been asked out on dates three times while taking Uber and Lyft rides. >> So drivers taking the opportunity to flirt a little bit. >> Chris: Sure. >> Another question, can you comment on the hashtag DeleteUber campaign? This of course, is about the backlash against Uber responding that it was intending to profit from President Trump's executive order, the banning immigrants and refugees from certain countries from entering the United States. Uber maintains that its intentions were misunderstood, but it didn't stop the hashtag DeleteUber campaign. >> Yeah, I haven't followed that super closely, but to me it seems like Uber's getting a bit of a bad rap. One potential reason why they allowed Uber drivers to continue working is that, maybe they wanted to bring protesters to the airports to protest. So from that perspective, actually having Uber and Lyft still in business would be beneficial. >> Another question, did your study take into account the race of the drivers themselves? >> We actually we not allowed to. So any time you do a randomized control trial in the field like this, you have to go through a campus committee that approves or disapproves the research, and they were worried that if we collected information on the driver, that potentially, Uber and Lyft could go back into their records and find the drivers that discriminate, and then have penalties assigned to those drivers. >> So it just wouldn't be allowed to... >> At least in this first phase, yeah. They didn't want us to collect those data. >> Last question, we have time for one more. Why aren't there more experiments in the field of applies economics like this one? That's a good question. >> That's a great question, and in fact, I think many of us are trying to push experiments as much as possible. My other line of research is actually in energy and climate change research, and we've been- >> Rebecca: You like the hot topic. (lauhging) >> We've been designing a bunch of experiments to look at how information impacts consumers' choices in terms of what cars to buy, how it impacts their use of electricity at home. And experiments, randomized control trials actually started in developmental economics, where MIT has actually pioneered their use. And again, it's the best way to actually test, the most rigorous way to test whether intervention actually has an effect because you have both the controlled group and the treatment group. >> So why aren't they done more often? >> Well, it's tough, often you need to find a third party, for example, we didn't need a third party in the sense that we could just send RAs out with Uber and Lyft. But if we wanted to do anything with the drivers, for example, an information campaign, or if we wanted to change the platform at all, we would've needed Uber and Lyft to partner with us, and that can sometimes be difficult to do. And also experiments, let's be honest, are pretty expensive. >> Expensive because, you obviously weren't partnered with Uber and Lyft for this one, but... >> Right, but we had research assistants take 1500 Uber and Lyft rides, so we had to pay for each of those rides, and we also had to give them an hourly rate for their time. >> Well, Chris Knittle, thank you so much. This has been great talking to you, and you've given us a lot to think about. >> It's been fun, thanks for having me. >> And thank you for joining us on this edition of the MIT Sloan Expert Series. We hope to see you again soon.

>> From theCUBE Studios in Palo Alto and Boston, bringing you data-driven insights from theCUBE and ETR. This is Breaking Analysis with Dave Vellante. >> Spark became a top level Apache project in 2014, and then shortly thereafter, burst onto the big data scene. Spark, along with the cloud, transformed and in many ways, disrupted the big data market. Databricks optimized its tech stack for Spark and took advantage of the cloud to really cleverly deliver a managed service that has become a leading AI and data platform among data scientists and data engineers. However, emerging customer data requirements are shifting into a direction that will cause modern data platform players generally and Databricks, specifically, we think, to make some key directional decisions and perhaps even reinvent themselves. Hello and welcome to this week's wikibon theCUBE Insights, powered by ETR. In this Breaking Analysis, we're going to do a deep dive into Databricks. We'll explore its current impressive market momentum. We're going to use some ETR survey data to show that, and then we'll lay out how customer data requirements are changing and what the ideal data platform will look like in the midterm future. We'll then evaluate core elements of the Databricks portfolio against that vision, and then we'll close with some strategic decisions that we think the company faces. And to do so, we welcome in our good friend, George Gilbert, former equities analyst, market analyst, and current Principal at TechAlpha Partners. George, good to see you. Thanks for coming on. >> Good to see you, Dave. >> All right, let me set this up. We're going to start by taking a look at where Databricks sits in the market in terms of how customers perceive the company and what it's momentum looks like. And this chart that we're showing here is data from ETS, the emerging technology survey of private companies. The N is 1,421. What we did is we cut the data on three sectors, analytics, database-data warehouse, and AI/ML. The vertical axis is a measure of customer sentiment, which evaluates an IT decision maker's awareness of the firm and the likelihood of engaging and/or purchase intent. The horizontal axis shows mindshare in the dataset, and we've highlighted Databricks, which has been a consistent high performer in this survey over the last several quarters. And as we, by the way, just as aside as we previously reported, OpenAI, which burst onto the scene this past quarter, leads all names, but Databricks is still prominent. You can see that the ETR shows some open source tools for reference, but as far as firms go, Databricks is very impressively positioned. Now, let's see how they stack up to some mainstream cohorts in the data space, against some bigger companies and sometimes public companies. This chart shows net score on the vertical axis, which is a measure of spending momentum and pervasiveness in the data set is on the horizontal axis. You can see that chart insert in the upper right, that informs how the dots are plotted, and net score against shared N. And that red dotted line at 40% indicates a highly elevated net score, anything above that we think is really, really impressive. And here we're just comparing Databricks with Snowflake, Cloudera, and Oracle. And that squiggly line leading to Databricks shows their path since 2021 by quarter. And you can see it's performing extremely well, maintaining an elevated net score and net range. Now it's comparable in the vertical axis to Snowflake, and it consistently is moving to the right and gaining share. Now, why did we choose to show Cloudera and Oracle? The reason is that Cloudera got the whole big data era started and was disrupted by Spark. And of course the cloud, Spark and Databricks and Oracle in many ways, was the target of early big data players like Cloudera. Take a listen to Cloudera CEO at the time, Mike Olson. This is back in 2010, first year of theCUBE, play the clip. >> Look, back in the day, if you had a data problem, if you needed to run business analytics, you wrote the biggest check you could to Sun Microsystems, and you bought a great big, single box, central server, and any money that was left over, you handed to Oracle for a database licenses and you installed that database on that box, and that was where you went for data. That was your temple of information. >> Okay? So Mike Olson implied that monolithic model was too expensive and inflexible, and Cloudera set out to fix that. But the best laid plans, as they say, George, what do you make of the data that we just shared? >> So where Databricks has really come up out of sort of Cloudera's tailpipe was they took big data processing, made it coherent, made it a managed service so it could run in the cloud. So it relieved customers of the operational burden. Where they're really strong and where their traditional meat and potatoes or bread and butter is the predictive and prescriptive analytics that building and training and serving machine learning models. They've tried to move into traditional business intelligence, the more traditional descriptive and diagnostic analytics, but they're less mature there. So what that means is, the reason you see Databricks and Snowflake kind of side by side is there are many, many accounts that have both Snowflake for business intelligence, Databricks for AI machine learning, where Snowflake, I'm sorry, where Databricks also did really well was in core data engineering, refining the data, the old ETL process, which kind of turned into ELT, where you loaded into the analytic repository in raw form and refine it. And so people have really used both, and each is trying to get into the other. >> Yeah, absolutely. We've reported on this quite a bit. Snowflake, kind of moving into the domain of Databricks and vice versa. And the last bit of ETR evidence that we want to share in terms of the company's momentum comes from ETR's Round Tables. They're run by Erik Bradley, and now former Gartner analyst and George, your colleague back at Gartner, Daren Brabham. And what we're going to show here is some direct quotes of IT pros in those Round Tables. There's a data science head and a CIO as well. Just make a few call outs here, we won't spend too much time on it, but starting at the top, like all of us, we can't talk about Databricks without mentioning Snowflake. Those two get us excited. Second comment zeros in on the flexibility and the robustness of Databricks from a data warehouse perspective. And then the last point is, despite competition from cloud players, Databricks has reinvented itself a couple of times over the year. And George, we're going to lay out today a scenario that perhaps calls for Databricks to do that once again. >> Their big opportunity and their big challenge for every tech company, it's managing a technology transition. The transition that we're talking about is something that's been bubbling up, but it's really epical. First time in 60 years, we're moving from an application-centric view of the world to a data-centric view, because decisions are becoming more important than automating processes. So let me let you sort of develop. >> Yeah, so let's talk about that here. We going to put up some bullets on precisely that point and the changing sort of customer environment. So you got IT stacks are shifting is George just said, from application centric silos to data centric stacks where the priority is shifting from automating processes to automating decision. You know how look at RPA and there's still a lot of automation going on, but from the focus of that application centricity and the data locked into those apps, that's changing. Data has historically been on the outskirts in silos, but organizations, you think of Amazon, think Uber, Airbnb, they're putting data at the core, and logic is increasingly being embedded in the data instead of the reverse. In other words, today, the data's locked inside the app, which is why you need to extract that data is sticking it to a data warehouse. The point, George, is we're putting forth this new vision for how data is going to be used. And you've used this Uber example to underscore the future state. Please explain? >> Okay, so this is hopefully an example everyone can relate to. The idea is first, you're automating things that are happening in the real world and decisions that make those things happen autonomously without humans in the loop all the time. So to use the Uber example on your phone, you call a car, you call a driver. Automatically, the Uber app then looks at what drivers are in the vicinity, what drivers are free, matches one, calculates an ETA to you, calculates a price, calculates an ETA to your destination, and then directs the driver once they're there. The point of this is that that cannot happen in an application-centric world very easily because all these little apps, the drivers, the riders, the routes, the fares, those call on data locked up in many different apps, but they have to sit on a layer that makes it all coherent. >> But George, so if Uber's doing this, doesn't this tech already exist? Isn't there a tech platform that does this already? >> Yes, and the mission of the entire tech industry is to build services that make it possible to compose and operate similar platforms and tools, but with the skills of mainstream developers in mainstream corporations, not the rocket scientists at Uber and Amazon. >> Okay, so we're talking about horizontally scaling across the industry, and actually giving a lot more organizations access to this technology. So by way of review, let's summarize the trend that's going on today in terms of the modern data stack that is propelling the likes of Databricks and Snowflake, which we just showed you in the ETR data and is really is a tailwind form. So the trend is toward this common repository for analytic data, that could be multiple virtual data warehouses inside of Snowflake, but you're in that Snowflake environment or Lakehouses from Databricks or multiple data lakes. And we've talked about what JP Morgan Chase is doing with the data mesh and gluing data lakes together, you've got various public clouds playing in this game, and then the data is annotated to have a common meaning. In other words, there's a semantic layer that enables applications to talk to the data elements and know that they have common and coherent meaning. So George, the good news is this approach is more effective than the legacy monolithic models that Mike Olson was talking about, so what's the problem with this in your view? >> So today's data platforms added immense value 'cause they connected the data that was previously locked up in these monolithic apps or on all these different microservices, and that supported traditional BI and AI/ML use cases. But now if we want to build apps like Uber or Amazon.com, where they've got essentially an autonomously running supply chain and e-commerce app where humans only care and feed it. But the thing is figuring out what to buy, when to buy, where to deploy it, when to ship it. We needed a semantic layer on top of the data. So that, as you were saying, the data that's coming from all those apps, the different apps that's integrated, not just connected, but it means the same. And the issue is whenever you add a new layer to a stack to support new applications, there are implications for the already existing layers, like can they support the new layer and its use cases? So for instance, if you add a semantic layer that embeds app logic with the data rather than vice versa, which we been talking about and that's been the case for 60 years, then the new data layer faces challenges that the way you manage that data, the way you analyze that data, is not supported by today's tools. >> Okay, so actually Alex, bring me up that last slide if you would, I mean, you're basically saying at the bottom here, today's repositories don't really do joins at scale. The future is you're talking about hundreds or thousands or millions of data connections, and today's systems, we're talking about, I don't know, 6, 8, 10 joins and that is the fundamental problem you're saying, is a new data error coming and existing systems won't be able to handle it? >> Yeah, one way of thinking about it is that even though we call them relational databases, when we actually want to do lots of joins or when we want to analyze data from lots of different tables, we created a whole new industry for analytic databases where you sort of mung the data together into fewer tables. So you didn't have to do as many joins because the joins are difficult and slow. And when you're going to arbitrarily join thousands, hundreds of thousands or across millions of elements, you need a new type of database. We have them, they're called graph databases, but to query them, you go back to the prerelational era in terms of their usability. >> Okay, so we're going to come back to that and talk about how you get around that problem. But let's first lay out what the ideal data platform of the future we think looks like. And again, we're going to come back to use this Uber example. In this graphic that George put together, awesome. We got three layers. The application layer is where the data products reside. The example here is drivers, rides, maps, routes, ETA, et cetera. The digital version of what we were talking about in the previous slide, people, places and things. The next layer is the data layer, that breaks down the silos and connects the data elements through semantics and everything is coherent. And then the bottom layers, the legacy operational systems feed that data layer. George, explain what's different here, the graph database element, you talk about the relational query capabilities, and why can't I just throw memory at solving this problem? >> Some of the graph databases do throw memory at the problem and maybe without naming names, some of them live entirely in memory. And what you're dealing with is a prerelational in-memory database system where you navigate between elements, and the issue with that is we've had SQL for 50 years, so we don't have to navigate, we can say what we want without how to get it. That's the core of the problem. >> Okay. So if I may, I just want to drill into this a little bit. So you're talking about the expressiveness of a graph. Alex, if you'd bring that back out, the fourth bullet, expressiveness of a graph database with the relational ease of query. Can you explain what you mean by that? >> Yeah, so graphs are great because when you can describe anything with a graph, that's why they're becoming so popular. Expressive means you can represent anything easily. They're conducive to, you might say, in a world where we now want like the metaverse, like with a 3D world, and I don't mean the Facebook metaverse, I mean like the business metaverse when we want to capture data about everything, but we want it in context, we want to build a set of digital twins that represent everything going on in the world. And Uber is a tiny example of that. Uber built a graph to represent all the drivers and riders and maps and routes. But what you need out of a database isn't just a way to store stuff and update stuff. You need to be able to ask questions of it, you need to be able to query it. And if you go back to prerelational days, you had to know how to find your way to the data. It's sort of like when you give directions to someone and they didn't have a GPS system and a mapping system, you had to give them turn by turn directions. Whereas when you have a GPS and a mapping system, which is like the relational thing, you just say where you want to go, and it spits out the turn by turn directions, which let's say, the car might follow or whoever you're directing would follow. But the point is, it's much easier in a relational database to say, "I just want to get these results. You figure out how to get it." The graph database, they have not taken over the world because in some ways, it's taking a 50 year leap backwards. >> Alright, got it. Okay. Let's take a look at how the current Databricks offerings map to that ideal state that we just laid out. So to do that, we put together this chart that looks at the key elements of the Databricks portfolio, the core capability, the weakness, and the threat that may loom. Start with the Delta Lake, that's the storage layer, which is great for files and tables. It's got true separation of compute and storage, I want you to double click on that George, as independent elements, but it's weaker for the type of low latency ingest that we see coming in the future. And some of the threats highlighted here. AWS could add transactional tables to S3, Iceberg adoption is picking up and could accelerate, that could disrupt Databricks. George, add some color here please? >> Okay, so this is the sort of a classic competitive forces where you want to look at, so what are customers demanding? What's competitive pressure? What are substitutes? Even what your suppliers might be pushing. Here, Delta Lake is at its core, a set of transactional tables that sit on an object store. So think of it in a database system, this is the storage engine. So since S3 has been getting stronger for 15 years, you could see a scenario where they add transactional tables. We have an open source alternative in Iceberg, which Snowflake and others support. But at the same time, Databricks has built an ecosystem out of tools, their own and others, that read and write to Delta tables, that's what makes the Delta Lake and ecosystem. So they have a catalog, the whole machine learning tool chain talks directly to the data here. That was their great advantage because in the past with Snowflake, you had to pull all the data out of the database before the machine learning tools could work with it, that was a major shortcoming. They fixed that. But the point here is that even before we get to the semantic layer, the core foundation is under threat. >> Yep. Got it. Okay. We got a lot of ground to cover. So we're going to take a look at the Spark Execution Engine next. Think of that as the refinery that runs really efficient batch processing. That's kind of what disrupted the DOOp in a large way, but it's not Python friendly and that's an issue because the data science and the data engineering crowd are moving in that direction, and/or they're using DBT. George, we had Tristan Handy on at Supercloud, really interesting discussion that you and I did. Explain why this is an issue for Databricks? >> So once the data lake was in place, what people did was they refined their data batch, and Spark has always had streaming support and it's gotten better. The underlying storage as we've talked about is an issue. But basically they took raw data, then they refined it into tables that were like customers and products and partners. And then they refined that again into what was like gold artifacts, which might be business intelligence metrics or dashboards, which were collections of metrics. But they were running it on the Spark Execution Engine, which it's a Java-based engine or it's running on a Java-based virtual machine, which means all the data scientists and the data engineers who want to work with Python are really working in sort of oil and water. Like if you get an error in Python, you can't tell whether the problems in Python or where it's in Spark. There's just an impedance mismatch between the two. And then at the same time, the whole world is now gravitating towards DBT because it's a very nice and simple way to compose these data processing pipelines, and people are using either SQL in DBT or Python in DBT, and that kind of is a substitute for doing it all in Spark. So it's under threat even before we get to that semantic layer, it so happens that DBT itself is becoming the authoring environment for the semantic layer with business intelligent metrics. But that's again, this is the second element that's under direct substitution and competitive threat. >> Okay, let's now move down to the third element, which is the Photon. Photon is Databricks' BI Lakehouse, which has integration with the Databricks tooling, which is very rich, it's newer. And it's also not well suited for high concurrency and low latency use cases, which we think are going to increasingly become the norm over time. George, the call out threat here is customers want to connect everything to a semantic layer. Explain your thinking here and why this is a potential threat to Databricks? >> Okay, so two issues here. What you were touching on, which is the high concurrency, low latency, when people are running like thousands of dashboards and data is streaming in, that's a problem because SQL data warehouse, the query engine, something like that matures over five to 10 years. It's one of these things, the joke that Andy Jassy makes just in general, he's really talking about Azure, but there's no compression algorithm for experience. The Snowflake guy started more than five years earlier, and for a bunch of reasons, that lead is not something that Databricks can shrink. They'll always be behind. So that's why Snowflake has transactional tables now and we can get into that in another show. But the key point is, so near term, it's struggling to keep up with the use cases that are core to business intelligence, which is highly concurrent, lots of users doing interactive query. But then when you get to a semantic layer, that's when you need to be able to query data that might have thousands or tens of thousands or hundreds of thousands of joins. And that's a SQL query engine, traditional SQL query engine is just not built for that. That's the core problem of traditional relational databases. >> Now this is a quick aside. We always talk about Snowflake and Databricks in sort of the same context. We're not necessarily saying that Snowflake is in a position to tackle all these problems. We'll deal with that separately. So we don't mean to imply that, but we're just sort of laying out some of the things that Snowflake or rather Databricks customers we think, need to be thinking about and having conversations with Databricks about and we hope to have them as well. We'll come back to that in terms of sort of strategic options. But finally, when come back to the table, we have Databricks' AI/ML Tool Chain, which has been an awesome capability for the data science crowd. It's comprehensive, it's a one-stop shop solution, but the kicker here is that it's optimized for supervised model building. And the concern is that foundational models like GPT could cannibalize the current Databricks tooling, but George, can't Databricks, like other software companies, integrate foundation model capabilities into its platform? >> Okay, so the sound bite answer to that is sure, IBM 3270 terminals could call out to a graphical user interface when they're running on the XT terminal, but they're not exactly good citizens in that world. The core issue is Databricks has this wonderful end-to-end tool chain for training, deploying, monitoring, running inference on supervised models. But the paradigm there is the customer builds and trains and deploys each model for each feature or application. In a world of foundation models which are pre-trained and unsupervised, the entire tool chain is different. So it's not like Databricks can junk everything they've done and start over with all their engineers. They have to keep maintaining what they've done in the old world, but they have to build something new that's optimized for the new world. It's a classic technology transition and their mentality appears to be, "Oh, we'll support the new stuff from our old stuff." Which is suboptimal, and as we'll talk about, their biggest patron and the company that put them on the map, Microsoft, really stopped working on their old stuff three years ago so that they could build a new tool chain optimized for this new world. >> Yeah, and so let's sort of close with what we think the options are and decisions that Databricks has for its future architecture. They're smart people. I mean we've had Ali Ghodsi on many times, super impressive. I think they've got to be keenly aware of the limitations, what's going on with foundation models. But at any rate, here in this chart, we lay out sort of three scenarios. One is re-architect the platform by incrementally adopting new technologies. And example might be to layer a graph query engine on top of its stack. They could license key technologies like graph database, they could get aggressive on M&A and buy-in, relational knowledge graphs, semantic technologies, vector database technologies. George, as David Floyer always says, "A lot of ways to skin a cat." We've seen companies like, even think about EMC maintained its relevance through M&A for many, many years. George, give us your thought on each of these strategic options? >> Okay, I find this question the most challenging 'cause remember, I used to be an equity research analyst. I worked for Frank Quattrone, we were one of the top tech shops in the banking industry, although this is 20 years ago. But the M&A team was the top team in the industry and everyone wanted them on their side. And I remember going to meetings with these CEOs, where Frank and the bankers would say, "You want us for your M&A work because we can do better." And they really could do better. But in software, it's not like with EMC in hardware because with hardware, it's easier to connect different boxes. With software, the whole point of a software company is to integrate and architect the components so they fit together and reinforce each other, and that makes M&A harder. You can do it, but it takes a long time to fit the pieces together. Let me give you examples. If they put a graph query engine, let's say something like TinkerPop, on top of, I don't even know if it's possible, but let's say they put it on top of Delta Lake, then you have this graph query engine talking to their storage layer, Delta Lake. But if you want to do analysis, you got to put the data in Photon, which is not really ideal for highly connected data. If you license a graph database, then most of your data is in the Delta Lake and how do you sync it with the graph database? If you do sync it, you've got data in two places, which kind of defeats the purpose of having a unified repository. I find this semantic layer option in number three actually more promising, because that's something that you can layer on top of the storage layer that you have already. You just have to figure out then how to have your query engines talk to that. What I'm trying to highlight is, it's easy as an analyst to say, "You can buy this company or license that technology." But the really hard work is making it all work together and that is where the challenge is. >> Yeah, and well look, I thank you for laying that out. We've seen it, certainly Microsoft and Oracle. I guess you might argue that well, Microsoft had a monopoly in its desktop software and was able to throw off cash for a decade plus while it's stock was going sideways. Oracle had won the database wars and had amazing margins and cash flow to be able to do that. Databricks isn't even gone public yet, but I want to close with some of the players to watch. Alex, if you'd bring that back up, number four here. AWS, we talked about some of their options with S3 and it's not just AWS, it's blob storage, object storage. Microsoft, as you sort of alluded to, was an early go-to market channel for Databricks. We didn't address that really. So maybe in the closing comments we can. Google obviously, Snowflake of course, we're going to dissect their options in future Breaking Analysis. Dbt labs, where do they fit? Bob Muglia's company, Relational.ai, why are these players to watch George, in your opinion? >> So everyone is trying to assemble and integrate the pieces that would make building data applications, data products easy. And the critical part isn't just assembling a bunch of pieces, which is traditionally what AWS did. It's a Unix ethos, which is we give you the tools, you put 'em together, 'cause you then have the maximum choice and maximum power. So what the hyperscalers are doing is they're taking their key value stores, in the case of ASW it's DynamoDB, in the case of Azure it's Cosmos DB, and each are putting a graph query engine on top of those. So they have a unified storage and graph database engine, like all the data would be collected in the key value store. Then you have a graph database, that's how they're going to be presenting a foundation for building these data apps. Dbt labs is putting a semantic layer on top of data lakes and data warehouses and as we'll talk about, I'm sure in the future, that makes it easier to swap out the underlying data platform or swap in new ones for specialized use cases. Snowflake, what they're doing, they're so strong in data management and with their transactional tables, what they're trying to do is take in the operational data that used to be in the province of many state stores like MongoDB and say, "If you manage that data with us, it'll be connected to your analytic data without having to send it through a pipeline." And that's hugely valuable. Relational.ai is the wildcard, 'cause what they're trying to do, it's almost like a holy grail where you're trying to take the expressiveness of connecting all your data in a graph but making it as easy to query as you've always had it in a SQL database or I should say, in a relational database. And if they do that, it's sort of like, it'll be as easy to program these data apps as a spreadsheet was compared to procedural languages, like BASIC or Pascal. That's the implications of Relational.ai. >> Yeah, and again, we talked before, why can't you just throw this all in memory? We're talking in that example of really getting down to differences in how you lay the data out on disk in really, new database architecture, correct? >> Yes. And that's why it's not clear that you could take a data lake or even a Snowflake and why you can't put a relational knowledge graph on those. You could potentially put a graph database, but it'll be compromised because to really do what Relational.ai has done, which is the ease of Relational on top of the power of graph, you actually need to change how you're storing your data on disk or even in memory. So you can't, in other words, it's not like, oh we can add graph support to Snowflake, 'cause if you did that, you'd have to change, or in your data lake, you'd have to change how the data is physically laid out. And then that would break all the tools that talk to that currently. >> What in your estimation, is the timeframe where this becomes critical for a Databricks and potentially Snowflake and others? I mentioned earlier midterm, are we talking three to five years here? Are we talking end of decade? What's your radar say? >> I think something surprising is going on that's going to sort of come up the tailpipe and take everyone by storm. All the hype around business intelligence metrics, which is what we used to put in our dashboards where bookings, billings, revenue, customer, those things, those were the key artifacts that used to live in definitions in your BI tools, and DBT has basically created a standard for defining those so they live in your data pipeline or they're defined in their data pipeline and executed in the data warehouse or data lake in a shared way, so that all tools can use them. This sounds like a digression, it's not. All this stuff about data mesh, data fabric, all that's going on is we need a semantic layer and the business intelligence metrics are defining common semantics for your data. And I think we're going to find by the end of this year, that metrics are how we annotate all our analytic data to start adding common semantics to it. And we're going to find this semantic layer, it's not three to five years off, it's going to be staring us in the face by the end of this year. >> Interesting. And of course SVB today was shut down. We're seeing serious tech headwinds, and oftentimes in these sort of downturns or flat turns, which feels like this could be going on for a while, we emerge with a lot of new players and a lot of new technology. George, we got to leave it there. Thank you to George Gilbert for excellent insights and input for today's episode. I want to thank Alex Myerson who's on production and manages the podcast, of course Ken Schiffman as well. Kristin Martin and Cheryl Knight help get the word out on social media and in our newsletters. And Rob Hof is our EIC over at Siliconangle.com, he does some great editing. Remember all these episodes, they're available as podcasts. Wherever you listen, all you got to do is search Breaking Analysis Podcast, we publish each week on wikibon.com and siliconangle.com, or you can email me at David.Vellante@siliconangle.com, or DM me @DVellante. Comment on our LinkedIn post, and please do check out ETR.ai, great survey data, enterprise tech focus, phenomenal. This is Dave Vellante for theCUBE Insights powered by ETR. Thanks for watching, and we'll see you next time on Breaking Analysis.

(upbeat music) >> Hello everyone. Welcome to theCube's presentation of the "AWS Startup Showcase." The topic this episode is AI and machine learning, top startups building foundational model infrastructure. This is season three, episode one of the ongoing series covering exciting startups from the AWS ecosystem. And this time we're talking about AI and machine learning. I'm your host, John Furrier. I'm excited I'm joined today by Robert Nishihara, who's the co-founder and CEO of a hot startup called Anyscale. He's here to talk about Ray, the open source project, Anyscale's infrastructure for foundation as well. Robert, thank you for joining us today. >> Yeah, thanks so much as well. >> I've been following your company since the founding pre pandemic and you guys really had a great vision scaled up and in a perfect position for this big wave that we all see with ChatGPT and OpenAI that's gone mainstream. Finally, AI has broken out through the ropes and now gone mainstream, so I think you guys are really well positioned. I'm looking forward to to talking with you today. But before we get into it, introduce the core mission for Anyscale. Why do you guys exist? What is the North Star for Anyscale? >> Yeah, like you mentioned, there's a tremendous amount of excitement about AI right now. You know, I think a lot of us believe that AI can transform just every different industry. So one of the things that was clear to us when we started this company was that the amount of compute needed to do AI was just exploding. Like to actually succeed with AI, companies like OpenAI or Google or you know, these companies getting a lot of value from AI, were not just running these machine learning models on their laptops or on a single machine. They were scaling these applications across hundreds or thousands or more machines and GPUs and other resources in the Cloud. And so to actually succeed with AI, and this has been one of the biggest trends in computing, maybe the biggest trend in computing in, you know, in recent history, the amount of compute has been exploding. And so to actually succeed with that AI, to actually build these scalable applications and scale the AI applications, there's a tremendous software engineering lift to build the infrastructure to actually run these scalable applications. And that's very hard to do. So one of the reasons many AI projects and initiatives fail is that, or don't make it to production, is the need for this scale, the infrastructure lift, to actually make it happen. So our goal here with Anyscale and Ray, is to make that easy, is to make scalable computing easy. So that as a developer or as a business, if you want to do AI, if you want to get value out of AI, all you need to know is how to program on your laptop. Like, all you need to know is how to program in Python. And if you can do that, then you're good to go. Then you can do what companies like OpenAI or Google do and get value out of machine learning. >> That programming example of how easy it is with Python reminds me of the early days of Cloud, when infrastructure as code was talked about was, it was just code the infrastructure programmable. That's super important. That's what AI people wanted, first program AI. That's the new trend. And I want to understand, if you don't mind explaining, the relationship that Anyscale has to these foundational models and particular the large language models, also called LLMs, was seen with like OpenAI and ChatGPT. Before you get into the relationship that you have with them, can you explain why the hype around foundational models? Why are people going crazy over foundational models? What is it and why is it so important? >> Yeah, so foundational models and foundation models are incredibly important because they enable businesses and developers to get value out of machine learning, to use machine learning off the shelf with these large models that have been trained on tons of data and that are useful out of the box. And then, of course, you know, as a business or as a developer, you can take those foundational models and repurpose them or fine tune them or adapt them to your specific use case and what you want to achieve. But it's much easier to do that than to train them from scratch. And I think there are three, for people to actually use foundation models, there are three main types of workloads or problems that need to be solved. One is training these foundation models in the first place, like actually creating them. The second is fine tuning them and adapting them to your use case. And the third is serving them and actually deploying them. Okay, so Ray and Anyscale are used for all of these three different workloads. Companies like OpenAI or Cohere that train large language models. Or open source versions like GPTJ are done on top of Ray. There are many startups and other businesses that fine tune, that, you know, don't want to train the large underlying foundation models, but that do want to fine tune them, do want to adapt them to their purposes, and build products around them and serve them, those are also using Ray and Anyscale for that fine tuning and that serving. And so the reason that Ray and Anyscale are important here is that, you know, building and using foundation models requires a huge scale. It requires a lot of data. It requires a lot of compute, GPUs, TPUs, other resources. And to actually take advantage of that and actually build these scalable applications, there's a lot of infrastructure that needs to happen under the hood. And so you can either use Ray and Anyscale to take care of that and manage the infrastructure and solve those infrastructure problems. Or you can build the infrastructure and manage the infrastructure yourself, which you can do, but it's going to slow your team down. It's going to, you know, many of the businesses we work with simply don't want to be in the business of managing infrastructure and building infrastructure. They want to focus on product development and move faster. >> I know you got a keynote presentation we're going to go to in a second, but I think you hit on something I think is the real tipping point, doing it yourself, hard to do. These are things where opportunities are and the Cloud did that with data centers. Turned a data center and made it an API. The heavy lifting went away and went to the Cloud so people could be more creative and build their product. In this case, build their creativity. Is that kind of what's the big deal? Is that kind of a big deal happening that you guys are taking the learnings and making that available so people don't have to do that? >> That's exactly right. So today, if you want to succeed with AI, if you want to use AI in your business, infrastructure work is on the critical path for doing that. To do AI, you have to build infrastructure. You have to figure out how to scale your applications. That's going to change. We're going to get to the point, and you know, with Ray and Anyscale, we're going to remove the infrastructure from the critical path so that as a developer or as a business, all you need to focus on is your application logic, what you want the the program to do, what you want your application to do, how you want the AI to actually interface with the rest of your product. Now the way that will happen is that Ray and Anyscale will still, the infrastructure work will still happen. It'll just be under the hood and taken care of by Ray in Anyscale. And so I think something like this is really necessary for AI to reach its potential, for AI to have the impact and the reach that we think it will, you have to make it easier to do. >> And just for clarification to point out, if you don't mind explaining the relationship of Ray and Anyscale real quick just before we get into the presentation. >> So Ray is an open source project. We created it. We were at Berkeley doing machine learning. We started Ray so that, in order to provide an easy, a simple open source tool for building and running scalable applications. And Anyscale is the managed version of Ray, basically we will run Ray for you in the Cloud, provide a lot of tools around the developer experience and managing the infrastructure and providing more performance and superior infrastructure. >> Awesome. I know you got a presentation on Ray and Anyscale and you guys are positioning as the infrastructure for foundational models. So I'll let you take it away and then when you're done presenting, we'll come back, I'll probably grill you with a few questions and then we'll close it out so take it away. >> Robert: Sounds great. So I'll say a little bit about how companies are using Ray and Anyscale for foundation models. The first thing I want to mention is just why we're doing this in the first place. And the underlying observation, the underlying trend here, and this is a plot from OpenAI, is that the amount of compute needed to do machine learning has been exploding. It's been growing at something like 35 times every 18 months. This is absolutely enormous. And other people have written papers measuring this trend and you get different numbers. But the point is, no matter how you slice and dice it, it' a astronomical rate. Now if you compare that to something we're all familiar with, like Moore's Law, which says that, you know, the processor performance doubles every roughly 18 months, you can see that there's just a tremendous gap between the needs, the compute needs of machine learning applications, and what you can do with a single chip, right. So even if Moore's Law were continuing strong and you know, doing what it used to be doing, even if that were the case, there would still be a tremendous gap between what you can do with the chip and what you need in order to do machine learning. And so given this graph, what we've seen, and what has been clear to us since we started this company, is that doing AI requires scaling. There's no way around it. It's not a nice to have, it's really a requirement. And so that led us to start Ray, which is the open source project that we started to make it easy to build these scalable Python applications and scalable machine learning applications. And since we started the project, it's been adopted by a tremendous number of companies. Companies like OpenAI, which use Ray to train their large models like ChatGPT, companies like Uber, which run all of their deep learning and classical machine learning on top of Ray, companies like Shopify or Spotify or Instacart or Lyft or Netflix, ByteDance, which use Ray for their machine learning infrastructure. Companies like Ant Group, which makes Alipay, you know, they use Ray across the board for fraud detection, for online learning, for detecting money laundering, you know, for graph processing, stream processing. Companies like Amazon, you know, run Ray at a tremendous scale and just petabytes of data every single day. And so the project has seen just enormous adoption since, over the past few years. And one of the most exciting use cases is really providing the infrastructure for building training, fine tuning, and serving foundation models. So I'll say a little bit about, you know, here are some examples of companies using Ray for foundation models. Cohere trains large language models. OpenAI also trains large language models. You can think about the workloads required there are things like supervised pre-training, also reinforcement learning from human feedback. So this is not only the regular supervised learning, but actually more complex reinforcement learning workloads that take human input about what response to a particular question, you know is better than a certain other response. And incorporating that into the learning. There's open source versions as well, like GPTJ also built on top of Ray as well as projects like Alpa coming out of UC Berkeley. So these are some of the examples of exciting projects in organizations, training and creating these large language models and serving them using Ray. Okay, so what actually is Ray? Well, there are two layers to Ray. At the lowest level, there's the core Ray system. This is essentially low level primitives for building scalable Python applications. Things like taking a Python function or a Python class and executing them in the cluster setting. So Ray core is extremely flexible and you can build arbitrary scalable applications on top of Ray. So on top of Ray, on top of the core system, what really gives Ray a lot of its power is this ecosystem of scalable libraries. So on top of the core system you have libraries, scalable libraries for ingesting and pre-processing data, for training your models, for fine tuning those models, for hyper parameter tuning, for doing batch processing and batch inference, for doing model serving and deployment, right. And a lot of the Ray users, the reason they like Ray is that they want to run multiple workloads. They want to train and serve their models, right. They want to load their data and feed that into training. And Ray provides common infrastructure for all of these different workloads. So this is a little overview of what Ray, the different components of Ray. So why do people choose to go with Ray? I think there are three main reasons. The first is the unified nature. The fact that it is common infrastructure for scaling arbitrary workloads, from data ingest to pre-processing to training to inference and serving, right. This also includes the fact that it's future proof. AI is incredibly fast moving. And so many people, many companies that have built their own machine learning infrastructure and standardized on particular workflows for doing machine learning have found that their workflows are too rigid to enable new capabilities. If they want to do reinforcement learning, if they want to use graph neural networks, they don't have a way of doing that with their standard tooling. And so Ray, being future proof and being flexible and general gives them that ability. Another reason people choose Ray in Anyscale is the scalability. This is really our bread and butter. This is the reason, the whole point of Ray, you know, making it easy to go from your laptop to running on thousands of GPUs, making it easy to scale your development workloads and run them in production, making it easy to scale, you know, training to scale data ingest, pre-processing and so on. So scalability and performance, you know, are critical for doing machine learning and that is something that Ray provides out of the box. And lastly, Ray is an open ecosystem. You can run it anywhere. You can run it on any Cloud provider. Google, you know, Google Cloud, AWS, Asure. You can run it on your Kubernetes cluster. You can run it on your laptop. It's extremely portable. And not only that, it's framework agnostic. You can use Ray to scale arbitrary Python workloads. You can use it to scale and it integrates with libraries like TensorFlow or PyTorch or JAX or XG Boost or Hugging Face or PyTorch Lightning, right, or Scikit-learn or just your own arbitrary Python code. It's open source. And in addition to integrating with the rest of the machine learning ecosystem and these machine learning frameworks, you can use Ray along with all of the other tooling in the machine learning ecosystem. That's things like weights and biases or ML flow, right. Or you know, different data platforms like Databricks, you know, Delta Lake or Snowflake or tools for model monitoring for feature stores, all of these integrate with Ray. And that's, you know, Ray provides that kind of flexibility so that you can integrate it into the rest of your workflow. And then Anyscale is the scalable compute platform that's built on top, you know, that provides Ray. So Anyscale is a managed Ray service that runs in the Cloud. And what Anyscale does is it offers the best way to run Ray. And if you think about what you get with Anyscale, there are fundamentally two things. One is about moving faster, accelerating the time to market. And you get that by having the managed service so that as a developer you don't have to worry about managing infrastructure, you don't have to worry about configuring infrastructure. You also, it provides, you know, optimized developer workflows. Things like easily moving from development to production, things like having the observability tooling, the debug ability to actually easily diagnose what's going wrong in a distributed application. So things like the dashboards and the other other kinds of tooling for collaboration, for monitoring and so on. And then on top of that, so that's the first bucket, developer productivity, moving faster, faster experimentation and iteration. The second reason that people choose Anyscale is superior infrastructure. So this is things like, you know, cost deficiency, being able to easily take advantage of spot instances, being able to get higher GPU utilization, things like faster cluster startup times and auto scaling. Things like just overall better performance and faster scheduling. And so these are the kinds of things that Anyscale provides on top of Ray. It's the managed infrastructure. It's fast, it's like the developer productivity and velocity as well as performance. So this is what I wanted to share about Ray in Anyscale. >> John: Awesome. >> Provide that context. But John, I'm curious what you think. >> I love it. I love the, so first of all, it's a platform because that's the platform architecture right there. So just to clarify, this is an Anyscale platform, not- >> That's right. >> Tools. So you got tools in the platform. Okay, that's key. Love that managed service. Just curious, you mentioned Python multiple times, is that because of PyTorch and TensorFlow or Python's the most friendly with machine learning or it's because it's very common amongst all developers? >> That's a great question. Python is the language that people are using to do machine learning. So it's the natural starting point. Now, of course, Ray is actually designed in a language agnostic way and there are companies out there that use Ray to build scalable Java applications. But for the most part right now we're focused on Python and being the best way to build these scalable Python and machine learning applications. But, of course, down the road there always is that potential. >> So if you're slinging Python code out there and you're watching that, you're watching this video, get on Anyscale bus quickly. Also, I just, while you were giving the presentation, I couldn't help, since you mentioned OpenAI, which by the way, congratulations 'cause they've had great scale, I've noticed in their rapid growth 'cause they were the fastest company to the number of users than anyone in the history of the computer industry, so major successor, OpenAI and ChatGPT, huge fan. I'm not a skeptic at all. I think it's just the beginning, so congratulations. But I actually typed into ChatGPT, what are the top three benefits of Anyscale and came up with scalability, flexibility, and ease of use. Obviously, scalability is what you guys are called. >> That's pretty good. >> So that's what they came up with. So they nailed it. Did you have an inside prompt training, buy it there? Only kidding. (Robert laughs) >> Yeah, we hard coded that one. >> But that's the kind of thing that came up really, really quickly if I asked it to write a sales document, it probably will, but this is the future interface. This is why people are getting excited about the foundational models and the large language models because it's allowing the interface with the user, the consumer, to be more human, more natural. And this is clearly will be in every application in the future. >> Absolutely. This is how people are going to interface with software, how they're going to interface with products in the future. It's not just something, you know, not just a chat bot that you talk to. This is going to be how you get things done, right. How you use your web browser or how you use, you know, how you use Photoshop or how you use other products. Like you're not going to spend hours learning all the APIs and how to use them. You're going to talk to it and tell it what you want it to do. And of course, you know, if it doesn't understand it, it's going to ask clarifying questions. You're going to have a conversation and then it'll figure it out. >> This is going to be one of those things, we're going to look back at this time Robert and saying, "Yeah, from that company, that was the beginning of that wave." And just like AWS and Cloud Computing, the folks who got in early really were in position when say the pandemic came. So getting in early is a good thing and that's what everyone's talking about is getting in early and playing around, maybe replatforming or even picking one or few apps to refactor with some staff and managed services. So people are definitely jumping in. So I have to ask you the ROI cost question. You mentioned some of those, Moore's Law versus what's going on in the industry. When you look at that kind of scale, the first thing that jumps out at people is, "Okay, I love it. Let's go play around." But what's it going to cost me? Am I going to be tied to certain GPUs? What's the landscape look like from an operational standpoint, from the customer? Are they locked in and the benefit was flexibility, are you flexible to handle any Cloud? What is the customers, what are they looking at? Basically, that's my question. What's the customer looking at? >> Cost is super important here and many of the companies, I mean, companies are spending a huge amount on their Cloud computing, on AWS, and on doing AI, right. And I think a lot of the advantage of Anyscale, what we can provide here is not only better performance, but cost efficiency. Because if we can run something faster and more efficiently, it can also use less resources and you can lower your Cloud spending, right. We've seen companies go from, you know, 20% GPU utilization with their current setup and the current tools they're using to running on Anyscale and getting more like 95, you know, 100% GPU utilization. That's something like a five x improvement right there. So depending on the kind of application you're running, you know, it's a significant cost savings. We've seen companies that have, you know, processing petabytes of data every single day with Ray going from, you know, getting order of magnitude cost savings by switching from what they were previously doing to running their application on Ray. And when you have applications that are spending, you know, potentially $100 million a year and getting a 10 X cost savings is just absolutely enormous. So these are some of the kinds of- >> Data infrastructure is super important. Again, if the customer, if you're a prospect to this and thinking about going in here, just like the Cloud, you got infrastructure, you got the platform, you got SaaS, same kind of thing's going to go on in AI. So I want to get into that, you know, ROI discussion and some of the impact with your customers that are leveraging the platform. But first I hear you got a demo. >> Robert: Yeah, so let me show you, let me give you a quick run through here. So what I have open here is the Anyscale UI. I've started a little Anyscale Workspace. So Workspaces are the Anyscale concept for interactive developments, right. So here, imagine I'm just, you want to have a familiar experience like you're developing on your laptop. And here I have a terminal. It's not on my laptop. It's actually in the cloud running on Anyscale. And I'm just going to kick this off. This is going to train a large language model, so OPT. And it's doing this on 32 GPUs. We've got a cluster here with a bunch of CPU cores, bunch of memory. And as that's running, and by the way, if I wanted to run this on instead of 32 GPUs, 64, 128, this is just a one line change when I launch the Workspace. And what I can do is I can pull up VS code, right. Remember this is the interactive development experience. I can look at the actual code. Here it's using Ray train to train the torch model. We've got the training loop and we're saying that each worker gets access to one GPU and four CPU cores. And, of course, as I make the model larger, this is using deep speed, as I make the model larger, I could increase the number of GPUs that each worker gets access to, right. And how that is distributed across the cluster. And if I wanted to run on CPUs instead of GPUs or a different, you know, accelerator type, again, this is just a one line change. And here we're using Ray train to train the models, just taking my vanilla PyTorch model using Hugging Face and then scaling that across a bunch of GPUs. And, of course, if I want to look at the dashboard, I can go to the Ray dashboard. There are a bunch of different visualizations I can look at. I can look at the GPU utilization. I can look at, you know, the CPU utilization here where I think we're currently loading the model and running that actual application to start the training. And some of the things that are really convenient here about Anyscale, both I can get that interactive development experience with VS code. You know, I can look at the dashboards. I can monitor what's going on. It feels, I have a terminal, it feels like my laptop, but it's actually running on a large cluster. And I can, with however many GPUs or other resources that I want. And so it's really trying to combine the best of having the familiar experience of programming on your laptop, but with the benefits, you know, being able to take advantage of all the resources in the Cloud to scale. And it's like when, you know, you're talking about cost efficiency. One of the biggest reasons that people waste money, one of the silly reasons for wasting money is just forgetting to turn off your GPUs. And what you can do here is, of course, things will auto terminate if they're idle. But imagine you go to sleep, I have this big cluster. You can turn it off, shut off the cluster, come back tomorrow, restart the Workspace, and you know, your big cluster is back up and all of your code changes are still there. All of your local file edits. It's like you just closed your laptop and came back and opened it up again. And so this is the kind of experience we want to provide for our users. So that's what I wanted to share with you. >> Well, I think that whole, couple of things, lines of code change, single line of code change, that's game changing. And then the cost thing, I mean human error is a big deal. People pass out at their computer. They've been coding all night or they just forget about it. I mean, and then it's just like leaving the lights on or your water running in your house. It's just, at the scale that it is, the numbers will add up. That's a huge deal. So I think, you know, compute back in the old days, there's no compute. Okay, it's just compute sitting there idle. But you know, data cranking the models is doing, that's a big point. >> Another thing I want to add there about cost efficiency is that we make it really easy to use, if you're running on Anyscale, to use spot instances and these preemptable instances that can just be significantly cheaper than the on-demand instances. And so when we see our customers go from what they're doing before to using Anyscale and they go from not using these spot instances 'cause they don't have the infrastructure around it, the fault tolerance to handle the preemption and things like that, to being able to just check a box and use spot instances and save a bunch of money. >> You know, this was my whole, my feature article at Reinvent last year when I met with Adam Selipsky, this next gen Cloud is here. I mean, it's not auto scale, it's infrastructure scale. It's agility. It's flexibility. I think this is where the world needs to go. Almost what DevOps did for Cloud and what you were showing me that demo had this whole SRE vibe. And remember Google had site reliability engines to manage all those servers. This is kind of like an SRE vibe for data at scale. I mean, a similar kind of order of magnitude. I mean, I might be a little bit off base there, but how would you explain it? >> It's a nice analogy. I mean, what we are trying to do here is get to the point where developers don't think about infrastructure. Where developers only think about their application logic. And where businesses can do AI, can succeed with AI, and build these scalable applications, but they don't have to build, you know, an infrastructure team. They don't have to develop that expertise. They don't have to invest years in building their internal machine learning infrastructure. They can just focus on the Python code, on their application logic, and run the stuff out of the box. >> Awesome. Well, I appreciate the time. Before we wrap up here, give a plug for the company. I know you got a couple websites. Again, go, Ray's got its own website. You got Anyscale. You got an event coming up. Give a plug for the company looking to hire. Put a plug in for the company. >> Yeah, absolutely. Thank you. So first of all, you know, we think AI is really going to transform every industry and the opportunity is there, right. We can be the infrastructure that enables all of that to happen, that makes it easy for companies to succeed with AI, and get value out of AI. Now we have, if you're interested in learning more about Ray, Ray has been emerging as the standard way to build scalable applications. Our adoption has been exploding. I mentioned companies like OpenAI using Ray to train their models. But really across the board companies like Netflix and Cruise and Instacart and Lyft and Uber, you know, just among tech companies. It's across every industry. You know, gaming companies, agriculture, you know, farming, robotics, drug discovery, you know, FinTech, we see it across the board. And all of these companies can get value out of AI, can really use AI to improve their businesses. So if you're interested in learning more about Ray and Anyscale, we have our Ray Summit coming up in September. This is going to highlight a lot of the most impressive use cases and stories across the industry. And if your business, if you want to use LLMs, you want to train these LLMs, these large language models, you want to fine tune them with your data, you want to deploy them, serve them, and build applications and products around them, give us a call, talk to us. You know, we can really take the infrastructure piece, you know, off the critical path and make that easy for you. So that's what I would say. And, you know, like you mentioned, we're hiring across the board, you know, engineering, product, go-to-market, and it's an exciting time. >> Robert Nishihara, co-founder and CEO of Anyscale, congratulations on a great company you've built and continuing to iterate on and you got growth ahead of you, you got a tailwind. I mean, the AI wave is here. I think OpenAI and ChatGPT, a customer of yours, have really opened up the mainstream visibility into this new generation of applications, user interface, roll of data, large scale, how to make that programmable so we're going to need that infrastructure. So thanks for coming on this season three, episode one of the ongoing series of the hot startups. In this case, this episode is the top startups building foundational model infrastructure for AI and ML. I'm John Furrier, your host. Thanks for watching. (upbeat music)

(upbeat music) >> Welcome back to The Cubes coverage of WIDS 2023 the eighth Annual Women in Data Science Conference which is held at Stanford University. I'm your host, Lisa Martin. I'm really excited to be having some great co-hosts today. I've got Hannah Freytag with me, who is a data journalism master student at Stanford. We have yet another inspiring woman in technology to bring to you today. Kelly Hoang joins us, data scientist at Gilead. It's so great to have you, Kelly. >> Hi, thank you for having me today. I'm super excited to be here and share my journey with you guys. >> Let's talk about that journey. You recently got your PhD in information sciences, congratulations. >> Thank you. Yes, I just graduated, I completed my PhD in information sciences from University of Illinois Urbana-Champaign. And right now I moved to Bay Area and started my career as a data scientist at Gilead. >> And you're in better climate. Well, we do get snow here. >> Kelly: That's true. >> We proved that the last... And data science can show us all the climate change that's going on here. >> That's true. That's the topic of the data fund this year, right? To understand the changes in the climate. >> Yeah. Talk a little bit about your background. You were mentioning before we went live that you come from a whole family of STEM students. So you had that kind of in your DNA. >> Well, I consider myself maybe I was a lucky case. I did grew up in a family in the STEM environment. My dad actually was a professor in computer science. So I remember when I was at a very young age, I already see like datas, all of these computer science concepts. So grew up to be a data scientist is always something like in my mind. >> You aspired to be. >> Yes. >> I love that. >> So I consider myself in a lucky place in that way. But also, like during this journey to become a data scientist you need to navigate yourself too, right? Like you have this roots, like this foundation but then you still need to kind of like figure out yourself what is it? Is it really the career that you want to pursue? But I'm happy that I'm end up here today and where I am right now. >> Oh, we're happy to have you. >> Yeah. So you' re with Gilead now after you're completing your PhD. And were you always interested in the intersection of data science and health, or is that something you explored throughout your studies? >> Oh, that's an excellent question. So I did have background in computer science but I only really get into biomedical domain when I did my PhD at school. So my research during my PhD was natural language processing, NLP and machine learning and their applications in biomedical domains. And then when I graduated, I got my first job in Gilead Science. Is super, super close and super relevant to what my research at school. And at Gilead, I am working in the advanced analytics department, and our focus is to bring artificial intelligence and machine learning into supporting clinical decision making. And really the ultimate goal is how to use AI to accelerate the precision medicine. So yes, it's something very like... I'm very lucky to get the first job that which is very close to my research at school. >> That's outstanding. You know, when we talk about AI, we can't not talk about ethics, bias. >> Kelly: Right. >> We know there's (crosstalk) Yes. >> Kelly: In healthcare. >> Exactly. Exactly. Equities in healthcare, equities in so many things. Talk a little bit about what excites you about AI, what you're doing at Gilead to really influence... I mean this, we're talking about something that's influencing life and death situations. >> Kelly: Right. >> How are you using AI in a way that is really maximizing the opportunities that AI can bring and maximizing the value in the data, but helping to dial down some of the challenges that come with AI? >> Yep. So as you may know already with the digitalization of medical records, this is nowaday, we have a tremendous opportunities to fulfill the dream of precision medicine. And what I mean by precision medicines, means now the treatments for people can be really tailored to individual patients depending on their own like characteristic or demographic or whatever. And nature language processing and machine learning, and AI in general really play a key role in that innovation, right? Because like there's a vast amount of information of patients and patient journeys or patient treatment is conducted and recorded in text. So that's why our group was established. Actually our department, advanced analytic department in Gilead is pretty new. We established our department last year. >> Oh wow. >> But really our mission is to bring AI into this field because we see the opportunity now. We have a vast amount of data about patient about their treatments, how we can mine these data how we can understand and tailor the treatment to individuals. And give everyone better care. >> I love that you brought up precision medicine. You know, I always think, if I kind of abstract everything, technology, data, connectivity, we have this expectation in our consumer lives. We can get anything we want. Not only can we get anything we want but we expect whoever we're engaging with, whether it's Amazon or Uber or Netflix to know enough about me to get me that precise next step. I don't think about precision medicine but you bring up such a great point. We expect these tailored experiences in our personal lives. Why not expect that in medicine as well? And have a tailored treatment plan based on whatever you have, based on data, your genetics, and being able to use NLP, machine learning and AI to drive that is really exciting. >> Yeah. You recap it very well, but then you also bring up a good point about the challenges to bring AI into this field right? Definitely this is an emerging field, but also very challenging because we talk about human health. We are doing the work that have direct impact to human health. So everything need to be... Whatever model, machine learning model that you are building, developing you need to be precise. It need to be evaluated properly before like using as a product, apply into the real practice. So it's not like recommendation systems for shopping or anything like that. We're talking about our actual health. So yes, it's challenging that way. >> Yeah. With that, you already answered one of the next questions I had because like medical data and health data is very sensitive. And how you at Gilead, you know, try to protect this data to protect like the human beings, you know, who are the data in the end. >> The security aspect is critical. You bring up a great point about sensitive data. We think of healthcare as sensitive data. Or PII if you're doing a bank transaction. We have to be so careful with that. Where is security, data security, in your everyday work practices within data science? Is it... I imagine it's a fundamental piece. >> Yes, for sure. We at Gilead, for sure, in data science organization we have like intensive trainings for employees about data privacy and security, how you use the data. But then also at the same time, when we work directly with dataset, it's not that we have like direct information about patient at like very granular level. Everything is need to be kind of like anonymized at some points to protect patient privacy. So we do have rules, policies to follow to put that in place in our organization. >> Very much needed. So some of the conversations we heard, were you able to hear the keynote this morning? >> Yes. I did. I attended. Like I listened to all of them. >> Isn't it fantastic? >> Yes, yes. Especially hearing these women from different backgrounds, at different level of their professional life, sharing their journeys. It's really inspiring. >> And Hannah, and I've been talking about, a lot of those journeys look like this. >> I know >> You just kind of go... It's very... Yours is linear, but you're kind of the exception. >> Yeah, this is why I consider my case as I was lucky to grow up in STEM environment. But then again, back to my point at the beginning, sometimes you need to navigate yourself too. Like I did mention about, I did my pa... Sorry, my bachelor degree in Vietnam, in STEM and in computer science. And that time, there's only five girls in a class of 100 students. So I was not the smartest person in the room. And I kept my minority in that areas, right? So at some point I asked myself like, "Huh, I don't know. Is this really my careers." It seems that others, like male people or students, they did better than me. But then you kind of like, I always have this passion of datas. So you just like navigate yourself, keep pushing yourself over those journey. And like being where I am right now. >> And look what you've accomplished. >> Thank you. >> Yeah. That's very inspiring. And yeah, you mentioned how you were in the classroom and you were only one of the few women in the room. And what inspired or motivated you to keep going, even though sometimes you were at these points where you're like, "Okay, is this the right thing?" "Is this the right thing for me?" What motivated you to keep going? >> Well, I think personally for me, as a data scientist or for woman working in data science in general, I always try to find a good story from data. Like it's not, when you have a data set, well it's important for you to come up with methodologies, what are you going to do with the dataset? But I think it's even more important to kind of like getting the context of the dataset. Like think about it like what is the story behind this dataset? What is the thing that you can get out of it and what is the meaning behind? How can we use it to help use it in a useful way. To have in some certain use case. So I always have that like curiosity and encouragement in myself. Like every time someone handed me a data set, I always think about that. So it's helped me to like build up this kind of like passion for me. And then yeah. And then become a data scientist. >> So you had that internal drive. I think it's in your DNA as well. When you were one of five. You were 5% women in your computer science undergrad in Vietnam. Yet as Hannah was asking you, you found a lot of motivation from within. You embrace that, which is so key. When we look at some of the statistics, speaking of data, of women in technical roles. We've seen it hover around 25% the last few years, probably five to 10. I was reading some data from anitab.org over the weekend, and it shows that it's now, in 2022, the number of women in technical roles rose slightly, but it rose, 27.6%. So we're seeing the needle move slowly. But one of the challenges that still remains is attrition. Women who are leaving the role. You've got your PhD. You have a 10 month old, you've got more than one child. What would you advise to women who might be at that crossroads of not knowing should I continue my career in climbing the ladder, or do I just go be with my family or do something else? What's your advice to them in terms of staying the path? >> I think it's really down to that you need to follow your passion. Like in any kind of job, not only like in data science right? If you want to be a baker, or you want to be a chef, or you want to be a software engineer. It's really like you need to ask yourself is it something that you're really passionate about? Because if you really passionate about something, regardless how difficult it is, like regardless like you have so many kids to take care of, you have the whole family to take care of. You have this and that. You still can find your time to spend on it. So it's really like let yourself drive your own passion. Drive the way where you leading to. I guess that's my advice. >> Kind of like following your own North Star, right? Is what you're suggesting. >> Yeah. >> What role have mentors played in your career path, to where you are now? Have you had mentors on the way or people who inspired you? >> Well, I did. I certainly met quite a lot of women who inspired me during my journey. But right now, at this moment, one person, particular person that I just popped into my mind is my current manager. She's also data scientist. She's originally from Caribbean and then came to the US, did her PhDs too, and now led a group, all women. So believe it or not, I am in a group of all women working in data science. So she's really like someone inspire me a lot, like someone I look up to in this career. >> I love that. You went from being one of five females in a class of 100, to now having a PhD in information sciences, and being on an all female data science team. That's pretty cool. >> It's great. Yeah, it's great. And then you see how fascinating that, how things shift right? And now today we are here in a conference that all are women in data science. >> Yeah. >> It's extraordinary. >> So this year we're fortunate to have WIDS coincide this year with the actual International Women's Day, March 8th which is so exciting. Which is always around this time of year, but it's great to have it on the day. The theme of this International Women's Day this year is embrace equity. When you think of that theme, and your career path, and what you're doing now, and who inspires you, how can companies like Gilead benefit from embracing equity? What are your thoughts on that as a theme? >> So I feel like I'm very lucky to get my first job at Gilead. Not only because the work that we are doing here very close to my research at school, but also because of the working environment at Gilead. Inclusion actually is one of the five core values of Gilead. >> Nice. >> So by that, we means we try to create and creating a working environment that all of the differences are valued. Like regardless your background, your gender. So at Gilead, we have women at Gilead which is a global network of female employees, that help us to strengthen our inclusion culture, and also to influence our voices into the company cultural company policy and practice. So yeah, I'm very lucky to work in the environment nowadays. >> It's impressive to not only hear that you're on an all female data science team, but what Gilead is doing and the actions they're taking. It's one thing, we've talked about this Hannah, for companies, and regardless of industry, to say we're going to have 50% women in our workforce by 2030, 2035, 2040. It's a whole other ballgame for companies like Gilead to actually be putting pen to paper. To actually be creating a strategy that they're executing on. That's awesome. And it must feel good to be a part of a company who's really adapting its culture to be more inclusive, because there's so much value that comes from inclusivity, thought diversity, that ultimately will help Gilead produce better products and services. >> Yeah. Yes. Yeah. Actually this here is the first year Gilead is a sponsor of the WIDS Conference. And we are so excited to establish this relationship, and looking forward to like having more collaboration with WIDS in the future. >> Excellent. Kelly we've had such a pleasure having you on the program. Thank you for sharing your linear path. You are definitely a unicorn. We appreciate your insights and your advice to those who might be navigating similar situations. Thank you for being on theCUBE today. >> Thank you so much for having me. >> Oh, it was our pleasure. For our guests, and Hannah Freytag this is Lisa Martin from theCUBE. Coming to you from WIDS 2023, the eighth annual conference. Stick around. Our final guest joins us in just a minute.

(upbeat music) >> Welcome back to theCUBE's coverage of Women in Data Science 2023 live from Stanford University. This is Lisa Martin. My co-host is Tracy Yuan. We're excited to be having great conversations all day but you know, 'cause you've been watching. We've been interviewing some very inspiring women and some men as well, talking about all of the amazing applications of data science. You're not going to want to miss this next conversation. Our guest is Gabriela de Queiroz, Principal Cloud Advocate Manager of Microsoft. Welcome, Gabriela. We're excited to have you. >> Thank you very much. I'm so excited to be talking to you. >> Yeah, you're on theCUBE. >> Yeah, finally. (Lisa laughing) Like a dream come true. (laughs) >> I know and we love that. We're so thrilled to have you. So you have a ton of experience in the data space. I was doing some research on you. You've worked in software, financial advertisement, health. Talk to us a little bit about you. What's your background in? >> So I was trained in statistics. So I'm a statistician and then I worked in epidemiology. I worked with air pollution and public health. So I was a researcher before moving into the industry. So as I was talking today, the weekly paths, it's exactly who I am. I went back and forth and back and forth and stopped and tried something else until I figured out that I want to do data science and that I want to do different things because with data science we can... The beauty of data science is that you can move across domains. So I worked in healthcare, financial, and then different technology companies. >> Well the nice thing, one of the exciting things that data science, that I geek out about and Tracy knows 'cause we've been talking about this all day, it's just all the different, to your point, diverse, pun intended, applications of data science. You know, this morning we were talking about, we had the VP of data science from Meta as a keynote. She came to theCUBE talking and really kind of explaining from a content perspective, from a monetization perspective, and of course so many people in the world are users of Facebook. It makes it tangible. But we also heard today conversations about the applications of data science in police violence, in climate change. We're in California, we're expecting a massive rainstorm and we don't know what to do when it rains or snows. But climate change is real. Everyone's talking about it, and there's data science at its foundation. That's one of the things that I love. But you also have a lot of experience building diverse teams. Talk a little bit about that. You've created some very sophisticated data science solutions. Talk about your recommendation to others to build diverse teams. What's in it for them? And maybe share some data science project or two that you really found inspirational. >> Yeah, absolutely. So I do love building teams. Every time I'm given the task of building teams, I feel the luckiest person in the world because you have the option to pick like different backgrounds and all the diverse set of like people that you can find. I don't think it's easy, like people say, yeah, it's very hard. You have to be intentional. You have to go from the very first part when you are writing the job description through the interview process. So you have to be very intentional in every step. And you have to think through when you are doing that. And I love, like my last team, we had like 10 people and we were so diverse. Like just talking about languages. We had like 15 languages inside a team. So how beautiful it is. Like all different backgrounds, like myself as a statistician, but we had people from engineering background, biology, languages, and so on. So it's, yeah, like every time thinking about building a team, if you wanted your team to be diverse, you need to be intentional. >> I'm so glad you brought up that intention point because that is the fundamental requirement really is to build it with intention. >> Exactly, and I love to hear like how there's different languages. So like I'm assuming, or like different backgrounds, I'm assuming everybody just zig zags their way into the team and now you're all women in data science and I think that's so precious. >> Exactly. And not only woman, right. >> Tracy: Not only woman, you're right. >> The team was diverse not only in terms of like gender, but like background, ethnicity, and spoken languages, and language that they use to program and backgrounds. Like as I mentioned, not everybody did the statistics in school or computer science. And it was like one of my best teams was when we had this combination also like things that I'm good at the other person is not as good and we have this knowledge sharing all the time. Every day I would feel like I'm learning something. In a small talk or if I was reviewing something, there was always something new because of like the richness of the diverse set of people that were in your team. >> Well what you've done is so impressive, because not only have you been intentional with it, but you sound like the hallmark of a great leader of someone who hires and builds teams to fill gaps. They don't have to know less than I do for me to be the leader. They have to have different skills, different areas of expertise. That is really, honestly Gabriela, that's the hallmark of a great leader. And that's not easy to come by. So tell me, who were some of your mentors and sponsors along the way that maybe influenced you in that direction? Or is that just who you are? >> That's a great question. And I joke that I want to be the role model that I never had, right. So growing up, I didn't have anyone that I could see other than my mom probably or my sister. But there was no one that I could see, I want to become that person one day. And once I was tracing my path, I started to see people looking at me and like, you inspire me so much, and I'm like, oh wow, this is amazing and I want to do do this over and over and over again. So I want to be that person to inspire others. And no matter, like I'll be like a VP, CEO, whoever, you know, I want to be, I want to keep inspiring people because that's so valuable. >> Lisa: Oh, that's huge. >> And I feel like when we grow professionally and then go to the next level, we sometimes we lose that, you know, thing that's essential. And I think also like, it's part of who I am as I was building and all my experiences as I was going through, I became what I mentioned is unique person that I think we all are unique somehow. >> You're a rockstar. Isn't she a rockstar? >> You dropping quotes out. >> I'm loving this. I'm like, I've inspired Gabriela. (Gabriela laughing) >> Oh my God. But yeah, 'cause we were asking our other guests about the same question, like, who are your role models? And then we're talking about how like it's very important for women to see that there is a representation, that there is someone they look up to and they want to be. And so that like, it motivates them to stay in this field and to start in this field to begin with. So yeah, I think like you are definitely filling a void and for all these women who dream to be in data science. And I think that's just amazing. >> And you're a founder too. In 2012, you founded R Ladies. Talk a little bit about that. This is present in more than 200 cities in 55 plus countries. Talk about R Ladies and maybe the catalyst to launch it. >> Yes, so you always start, so I'm from Brazil, I always talk about this because it's such, again, I grew up over there. So I was there my whole life and then I moved to here, Silicon Valley. And when I moved to San Francisco, like the doors opened. So many things happening in the city. That was back in 2012. Data science was exploding. And I found out something about Meetup.com, it's a website that you can join and go in all these events. And I was going to this event and I joke that it was kind of like going to the Disneyland, where you don't know if I should go that direction or the other direction. >> Yeah, yeah. >> And I was like, should I go and learn about data visualization? Should I go and learn about SQL or should I go and learn about Hadoop, right? So I would go every day to those meetups. And I was a student back then, so you know, the budget was very restricted as a student. So we don't have much to spend. And then they would serve dinner and you would learn for free. And then I got to a point where I was like, hey, they are doing all of this as a volunteer. Like they are running this meetup and events for free. And I felt like it's a cycle. I need to do something, right. I'm taking all this in. I'm having this huge opportunity to be here. I want to give back. So that's what how everything started. I was like, no, I have to think about something. I need to think about something that I can give back. And I was using R back then and I'm like how about I do something with R. I love R, I'm so passionate about R, what about if I create a community around R but not a regular community, because by going to this events, I felt that as a Latina and as a woman, I was always in the corner and I was not being able to participate and to, you know, be myself and to network and ask questions. I would be in the corner. So I said to myself, what about if I do something where everybody feel included, where everybody can participate, can share, can ask questions without judgment? So that's how R ladies all came together. >> That's awesome. >> Talk about intentions, like you have to, you had that go in mind, but yeah, I wanted to dive a little bit into R. So could you please talk more about where did the passion for R come from, and like how did the special connection between you and R the language, like born, how did that come from? >> It was not a love at first sight. >> No. >> Not at all. Not at all. Because that was back in Brazil. So all the documentation were in English, all the tutorials, only two. We had like very few tutorials. It was not like nowadays that we have so many tutorials and courses. There were like two tutorials, other documentation in English. So it's was hard for me like as someone that didn't know much English to go through the language and then to learn to program was not easy task. But then as I was going through the language and learning and reading books and finding the people behind the language, I don't know how I felt in love. And then when I came to to San Francisco, I saw some of like the main contributors who are speaking in person and I'm like, wow, they are like humans. I don't know, it was like, I have no idea why I had this love. But I think the the people and then the community was the thing that kept me with the R language. >> Yeah, the community factors is so important. And it's so, at WIDS it's so palpable. I mean I literally walk in the door, every WIDS I've done, I think I've been doing them for theCUBE since 2017. theCUBE has been here since the beginning in 2015 with our co-founders. But you walk in, you get this sense of belonging. And this sense of I can do anything, why not? Why not me? Look at her up there, and now look at you speaking in the technical talk today on theCUBE. So inspiring. One of the things that I always think is you can't be what you can't see. We need to be able to see more people that look like you and sound like you and like me and like you as well. And WIDS gives us that opportunity, which is fantastic, but it's also helping to move the needle, really. And I was looking at some of the Anitab.org stats just yesterday about 2022. And they're showing, you know, the percentage of females in technical roles has been hovering around 25% for a while. It's a little higher now. I think it's 27.6 according to any to Anitab. We're seeing more women hired in roles. But what are the challenges, and I would love to get your advice on this, for those that might be in this situation is attrition, women who are leaving roles. What would your advice be to a woman who might be trying to navigate family and work and career ladder to stay in that role and keep pushing forward? >> I'll go back to the community. If you don't have a community around you, it's so hard to navigate. >> That's a great point. >> You are lonely. There is no one that you can bounce ideas off, that you can share what you are feeling or like that you can learn as well. So sometimes you feel like you are the only person that is going through that problem or like, you maybe have a family or you are planning to have a family and you have to make a decision. But you've never seen anyone going through this. So when you have a community, you see people like you, right. So that's where we were saying about having different people and people like you so they can share as well. And you feel like, oh yeah, so they went through this, they succeed. I can also go through this and succeed. So I think the attrition problem is still big problem. And I'm sure will be worse now with everything that is happening in Tech with layoffs. >> Yes and the great resignation. >> Yeah. >> We are going back, you know, a few steps, like a lot of like advancements that we did. I feel like we are going back unfortunately, but I always tell this, make sure that you have a community. Make sure that you have a mentor. Make sure that you have someone or some people, not only one mentor, different mentors, that can support you through this trajectory. Because it's not easy. But there are a lot of us out there. >> There really are. And that's a great point. I love everything about the community. It's all about that network effect and feeling like you belong- >> That's all WIDS is about. >> Yeah. >> Yes. Absolutely. >> Like coming over here, it's like seeing the old friends again. It's like I'm so glad that I'm coming because I'm all my old friends that I only see like maybe once a year. >> Tracy: Reunion. >> Yeah, exactly. And I feel like that our tank get, you know- >> Lisa: Replenished. >> Exactly. For the rest of the year. >> Yes. >> Oh, that's precious. >> I love that. >> I agree with that. I think one of the things that when I say, you know, you can't see, I think, well, how many females in technology would I be able to recognize? And of course you can be female technology working in the healthcare sector or working in finance or manufacturing, but, you know, we need to be able to have more that we can see and identify. And one of the things that I recently found out, I was telling Tracy this earlier that I geeked out about was finding out that the CTO of Open AI, ChatGPT, is a female. I'm like, (gasps) why aren't we talking about this more? She was profiled on Fast Company. I've seen a few pieces on her, Mira Murati. But we're hearing so much about ChatJTP being... ChatGPT, I always get that wrong, about being like, likening it to the launch of the iPhone, which revolutionized mobile and connectivity. And here we have a female in the technical role. Let's put her on a pedestal because that is hugely inspiring. >> Exactly, like let's bring everybody to the front. >> Yes. >> Right. >> And let's have them talk to us because like, you didn't know. I didn't know probably about this, right. You didn't know. Like, we don't know about this. It's kind of like we are hidden. We need to give them the spotlight. Every woman to give the spotlight, so they can keep aspiring the new generation. >> Or Susan Wojcicki who ran, how long does she run YouTube? All the YouTube influencers that probably have no idea who are influential for whatever they're doing on YouTube in different social platforms that don't realize, do you realize there was a female behind the helm that for a long time that turned it into what it is today? That's outstanding. Why aren't we talking about this more? >> How about Megan Smith, was the first CTO on the Obama administration. >> That's right. I knew it had to do with Obama. Couldn't remember. Yes. Let's let's find more pedestals. But organizations like WIDS, your involvement as a speaker, showing more people you can be this because you can see it, >> Yeah, exactly. is the right direction that will help hopefully bring us back to some of the pre-pandemic levels, and keep moving forward because there's so much potential with data science that can impact everyone's lives. I always think, you know, we have this expectation that we have our mobile phone and we can get whatever we want wherever we are in the world and whatever time of day it is. And that's all data driven. The regular average person that's not in tech thinks about data as a, well I'm paying for it. What's all these data charges? But it's powering the world. It's powering those experiences that we all want as consumers or in our business lives or we expect to be able to do a transaction, whether it's something in a CRM system or an Uber transaction like that, and have the app respond, maybe even know me a little bit better than I know myself. And that's all data. So I think we're just at the precipice of the massive impact that data science will make in our lives. And luckily we have leaders like you who can help navigate us along this path. >> Thank you. >> What advice for, last question for you is advice for those in the audience who might be nervous or maybe lack a little bit of confidence to go I really like data science, or I really like engineering, but I don't see a lot of me out there. What would you say to them? >> Especially for people who are from like a non-linear track where like going onto that track. >> Yeah, I would say keep going. Keep going. I don't think it's easy. It's not easy. But keep going because the more you go the more, again, you advance and there are opportunities out there. Sometimes it takes a little bit, but just keep going. Keep going and following your dreams, that you get there, right. So again, data science, such a broad field that doesn't require you to come from a specific background. And I think the beauty of data science exactly is this is like the combination, the most successful data science teams are the teams that have all these different backgrounds. So if you think that we as data scientists, we started programming when we were nine, that's not true, right. You can be 30, 40, shifting careers, starting to program right now. It doesn't matter. Like you get there no matter how old you are. And no matter what's your background. >> There's no limit. >> There was no limits. >> I love that, Gabriela, >> Thank so much. for inspiring. I know you inspired me. I'm pretty sure you probably inspired Tracy with your story. And sometimes like what you just said, you have to be your own mentor and that's okay. Because eventually you're going to turn into a mentor for many, many others and sounds like you're already paving that path and we so appreciate it. You are now officially a CUBE alumni. >> Yes. Thank you. >> Yay. We've loved having you. Thank you so much for your time. >> Thank you. Thank you. >> For our guest and for Tracy's Yuan, this is Lisa Martin. We are live at WIDS 23, the eighth annual Women in Data Science Conference at Stanford. Stick around. Our next guest joins us in just a few minutes. (upbeat music)

(gentle upbeat music) >> Hey, friends of theCUBE. It's Lisa Martin live at Stanford University covering the Eighth Annual Women In Data Science. But you've been a Cube fan for a long time. So you know that we've been here since the beginning of WiDS, which is 2015. We always loved to come and cover this event. We learned great things about data science, about women leaders, underrepresented minorities. And this year we have a special component. We've got two grad students from Stanford's Master's program and Data Journalism joining. One of my them is here with me, Hannah Freitag, my co-host. Great to have you. And we are pleased to welcome from Intuit for the first time, Shir Meir Lador Group Manager at Data Science. Shir, it's great to have you. Thank you for joining us. >> Thank you for having me. >> And I was just secrets girl talking with my boss of theCUBE who informed me that you're in great company. Intuit's Chief Technology Officer, Marianna Tessel is an alumni of theCUBE. She was on at our Supercloud event in January. So welcome back into it. >> Thank you very much. We're happy to be with you. >> Tell us a little bit about what you're doing. You're a data science group manager as I mentioned, but also you've had you've done some cool things I want to share with the audience. You're the co-founder of the PyData Tel Aviv Meetups the co-host of the unsupervised podcast about data science in Israel. You give talks, about machine learning, about data science. Tell us a little bit about your background. Were you always interested in STEM studies from the time you were small? >> So I was always interested in mathematics when I was small, I went to this special program for youth going to university. So I did my test in mathematics earlier and studied in university some courses. And that's when I understood I want to do something in that field. And then when I got to go to university, I went to electrical engineering when I found out about algorithms and how interested it is to be able to find solutions to problems, to difficult problems with math. And this is how I found my way into machine learning. >> Very cool. There's so much, we love talking about machine learning and AI on theCUBE. There's so much potential. Of course, we have to have data. One of the things that I love about WiDS and Hannah and I and our co-host Tracy, have been talking about this all day is the impact of data in everyone's life. If you break it down, I was at Mobile World Congress last week, all about connectivity telecom, and of course we have these expectation that we're going to be connected 24/7 from wherever we are in the world and we can do whatever we want. I can do an Uber transaction, I can watch Netflix, I can do a bank transaction. It all is powered by data. And data science is, some of the great applications of it is what it's being applied to. Things like climate change or police violence or health inequities. Talk about some of the data science projects that you're working on at Intuit. I'm an intuit user myself, but talk to me about some of those things. Give the audience really a feel for what you're doing. >> So if you are a Intuit product user, you probably use TurboTax. >> I do >> In the past. So for those who are not familiar, TurboTax help customers submit their taxes. Basically my group is in charge of getting all the information automatically from your documents, the documents that you upload to TurboTax. We extract that information to accelerate your tax submission to make it less work for our customers. So- >> Thank you. >> Yeah, and this is why I'm so proud to be working at this team because our focus is really to help our customers to simplify all the you know, financial heavy lifting with taxes and also with small businesses. We also do a lot of work in extracting information from small business documents like bill, receipts, different bank statements. Yeah, so this is really exciting for me, the opportunity to work to apply data science and machine learning to solution that actually help people. Yeah >> Yeah, in the past years there have been more and more digital products emerging that needs some sort of data security. And how did your team, or has your team developed in the past years with more and more products or companies offering digital services? >> Yeah, so can you clarify the question again? Sorry. >> Yeah, have you seen that you have more customers? Like has your team expanded in the past years with more digital companies starting that need kind of data security? >> Well, definitely. I think, you know, since I joined Intuit, I joined like five and a half years ago back when I was in Tel Aviv. I recently moved to the Bay Area. So when I joined, there were like a dozens of data scientists and machine learning engineers on Intuit. And now there are a few hundreds. So we've definitely grown with the year and there are so many new places we can apply machine learning to help our customers. So this is amazing, so much we can do with machine learning to get more money in the pocket of our customers and make them do less work. >> I like both of those. More money in my pocket and less work. That's awesome. >> Exactly. >> So keep going Intuit. But one of the things that is so cool is just the the abstraction of the complexity that Intuit's doing. I upload documents or it scans my receipts. I was just in Barcelona last week all these receipts and conversion euros to dollars and it takes that complexity away from the end user who doesn't know all that's going on in the background, but you're making people's lives simpler. Unfortunately, we all have to pay taxes, most of us should. And of course we're in tax season right now. And so it's really cool what you're doing with ML and data science to make fundamental processes to people's lives easier and just a little bit less complicated. >> Definitely. And I think that's what's also really amazing about Intuit it, is how it combines human in the loop as well as AI. Because in some of the tax situation it's very complicated maybe to do it yourself. And then there's an option to work with an expert online that goes on a video with you and helps you do your taxes. And the expert's work is also accelerated by AI because we build tools for those experts to do the work more efficiently. >> And that's what it's all about is you know, using data to be more efficient, to be faster, to be smarter, but also to make complicated processes in our daily lives, in our business lives just a little bit easier. One of the things I've been geeking out about recently is ChatGPT. I was using it yesterday. I was telling everyone I was asking it what's hot in data science and I didn't know would it know what hot is and it did, it gave me trends. But one of the things that I was so, and Hannah knows I've been telling this all day, I was so excited to learn over the weekend that the the CTO of OpenAI is a female. I didn't know that. And I thought why are we not putting her on a pedestal? Because people are likening ChatGPT to like the launch of the iPhone. I mean revolutionary. And here we have what I think is exciting for all of us females, whether you're in tech or not, is another role model. Because really ultimately what WiDS is great at doing is showcasing women in technical roles. Because I always say you can't be what you can't see. We need to be able to see more role models, female role role models, underrepresented minorities of course men, because a lot of my sponsors and mentors are men, but we need more women that we can look up to and see ah, she's doing this, why can't I? Talk to me about how you stay the course in data science. What excites you about the potential, the opportunities based on what you've already accomplished what inspires you to continue and be one of those females that we say oh my God, I could be like Shir. >> I think that what inspires me the most is the endless opportunities that we have. I think we haven't even started tapping into everything that we can do with generative AI, for example. There's so much that can be done to further help you know, people make more money and do less work because there's still so much work that we do that we don't need to. You know, this is with Intuit, but also there are so many other use cases like I heard today you know, with the talk about the police. So that was really exciting how you can apply machine learning and data to actually help people, to help people that been through wrongful things. So I was really moved by that. And I'm also really excited about all the medical applications that we can have with data. >> Yeah, yeah. It's true that data science is so diverse in terms of what fields it can cover but it's equally important to have diverse teams and have like equity and inclusion in your teams. Where is Intuit at promoting women, non-binary minorities in your teams to progress data science? >> Yeah, so I have so much to say on this. >> Good. >> But in my work in Tel Aviv, I had the opportunity to start with Intuit women in data science branch in Tel Aviv. So that's why I'm super excited to be here today for that because basically this is the original conference, but as you know, there are branches all over the world and I got the opportunity to lead the Tel Aviv branch with Israel since 2018. And we've been through already this year it's going to be it's next week, it's going to be the sixth conference. And every year our number of submission to make talk in the conference doubled itself. >> Nice. >> We started with 20 submission, then 50, then 100. This year we have over 200 submissions of females to give talk at the conference. >> Ah, that's fantastic. >> And beyond the fact that there's so much traction, I also feel the great impact it has on the community in Israel because one of the reason we started WiDS was that when I was going to conferences I was seeing so little women on stage in all the technical conferences. You know, kind of the reason why I guess you know, Margaret and team started the WiDS conference. So I saw the same thing in Israel and I was always frustrated. I was organizing PyData Meetups as you mentioned and I was always having such a hard time to get female speakers to talk. I was trying to role model, but that's not enough, you know. We need more. So once we started WiDS and people saw you know, so many examples on the stage and also you know females got opportunity to talk in a place for that. Then it also started spreading and you can see more and more female speakers across other conferences, which are not women in data science. So I think just the fact that Intuits started this conference back in Israel and also in Bangalore and also the support Intuit does for WiDS in Stanford here, it shows how much WiDS values are aligned with our values. Yeah, and I think that to chauffeur that I think we have over 35% females in the data science and machine learning engineering roles, which is pretty amazing I think compared to the industry. >> Way above average. Yeah, absolutely. I was just, we've been talking about some of the AnitaB.org stats from 2022 showing that 'cause usually if we look at the industry to you point, over the last, I don't know, probably five, 10 years we're seeing the number of female technologists around like a quarter, 25% or so. 2022 data from AnitaB.org showed that that number is now 27.6%. So it's very slowly- >> It's very slowly increasing. >> Going in the right direction. >> Too slow. >> And that representation of women technologists increase at every level, except intern, which I thought was really interesting. And I wonder is there a covid relation there? >> I don't know. >> What do we need to do to start opening up the the top of the pipeline, the funnel to go downstream to find kids like you when you were younger and always interested in engineering and things like that. But the good news is that the hiring we've seen improvements, but it sounds like Intuit is way ahead of the curve there with 35% women in data science or technical roles. And what's always nice and refreshing that we've talked, Hannah about this too is seeing companies actually put action into initiatives. It's one thing for a company to say we're going to have you know, 50% females in our organization by 2030. It's a whole other ball game to actually create a strategy, execute on it, and share progress. So kudos to Intuit for what it's doing because that is more companies need to adopt that same sort of philosophy. And that's really cultural. >> Yeah. >> At an organization and culture can be hard to change, but it sounds like you guys kind of have it dialed in. >> I think we definitely do. That's why I really like working and Intuit. And I think that a lot of it is with the role modeling, diversity and inclusion, and by having women leaders. When you see a woman in leadership position, as a woman it makes you want to come work at this place. And as an evidence, when I build the team I started in Israel at Intuit, I have over 50% women in my team. >> Nice. >> Yeah, because when you have a woman in the interviewers panel, it's much easier, it's more inclusive. That's why we always try to have at least you know, one woman and also other minorities represented in our interviews panel. Yeah, and I think that in general it's very important as a leader to kind of know your own biases and trying to have defined standard and rubrics in how you evaluate people to avoid for those biases. So all of that inclusiveness and leadership really helps to get more diversity in your teams. >> It's critical. That thought diversity is so critical, especially if we talk about AI and we're almost out of time, I just wanted to bring up, you brought up a great point about the diversity and equity. With respect to data science and AI, we know in AI there's biases in data. We need to have more inclusivity, more representation to help start shifting that so the biases start to be dialed down and I think a conference like WiDS and it sounds like someone like you and what you've already done so far in the work that you're doing having so many females raise their hands to want to do talks at events is a good situation. It's a good scenario and hopefully it will continue to move the needle on the percentage of females in technical roles. So we thank you Shir for your time sharing with us your story, what you're doing, how Intuit and WiDS are working together. It sounds like there's great alignment there and I think we're at the tip of the iceberg with what we can do with data science and inclusion and equity. So we appreciate all of your insights and your time. >> Thank you very much. >> All right. >> I enjoyed very, very much >> Good. We hope, we aim to please. Thank you for our guests and for Hannah Freitag. This is Lisa Martin coming to you live from Stanford University. This is our coverage of the eighth Annual Women in Data Science Conference. Stick around, next guest will be here in just a minute.

(gentle music) >> Hey! Welcome back to theCUBE's coverage of WiDS 2023, the eighth Annual Women In Data Science Conference. I'm your host, Lisa Martin. We are at Stanford University, as you know we are every year, having some wonderful conversations with some very inspiring women and men in data science and technical roles. I'm very pleased to introduce Tracy Zhang, my co-host, who is in the Data Journalism program at Stanford. And Tracy and I are pleased to welcome our next guest, Rhonda Crate, Principal Data Scientist at Boeing. Great to have you on the program, Rhonda. >> Tracy: Welcome. >> Hey, thanks for having me. >> Were you always interested in data science or STEM from the time you were young? >> No, actually. I was always interested in archeology and anthropology. >> That's right, we were talking about that, anthropology. Interesting. >> We saw the anthropology background, not even a bachelor's degree, but also a master's degree in anthropology. >> So you were committed for a while. >> I was, I was. I actually started college as a fine arts major, but I always wanted to be an archeologist. So at the last minute, 11 credits in, left to switch to anthropology. And then when I did my master's, I focused a little bit more on quantitative research methods and then I got my Stat Degree. >> Interesting. Talk about some of the data science projects that you're working on. When I think of Boeing, I always think of aircraft. But you are doing a lot of really cool things in IT, data analytics. Talk about some of those intriguing data science projects that you're working on. >> Yeah. So when I first started at Boeing, I worked in information technology and data analytics. And Boeing, at the time, had cored up data science in there. And so we worked as a function across the enterprise working on anything from shared services to user experience in IT products, to airplane programs. So, it has a wide range. I worked on environment health and safety projects for a long time as well. So looking at ergonomics and how people actually put parts onto airplanes, along with things like scheduling and production line, part failures, software testing. Yeah, there's a wide spectrum of things. >> But I think that's so fantastic. We've been talking, Tracy, today about just what we often see at WiDS, which is this breadth of diversity in people's background. You talked about anthropology, archeology, you're doing data science. But also all of the different opportunities that you've had at Boeing. To see so many facets of that organization. I always think that breadth of thought diversity can be hugely impactful. >> Yeah. So I will say my anthropology degree has actually worked to my benefit. I'm a huge proponent of integrating liberal arts and sciences together. And it actually helps me. I'm in the Technical Fellowship program at Boeing, so we have different career paths. So you can go into management, you can be a regular employee, or you can go into the Fellowship program. So right now I'm an Associate Technical Fellow. And part of how I got into the Fellowship program was that diversity in my background, what made me different, what made me stand out on projects. Even applying a human aspect to things like ergonomics, as silly as that sounds, but how does a person actually interact in the space along with, here are the actual measurements coming off of whatever system it is that you're working on. So, I think there's a lot of opportunities, especially in safety as well, which is a big initiative for Boeing right now, as you can imagine. >> Tracy: Yeah, definitely. >> I can't go into too specifics. >> No, 'cause we were like, I think a theme for today that kind of we brought up in in all of our talk is how data is about people, how data is about how people understand the world and how these data can make impact on people's lives. So yeah, I think it's great that you brought this up, and I'm very happy that your anthropology background can tap into that and help in your day-to-day data work too. >> Yeah. And currently, right now, I actually switched over to Strategic Workforce Planning. So it's more how we understand our workforce, how we work towards retaining the talent, how do we get the right talent in our space, and making sure overall that we offer a culture and work environment that is great for our employees to come to. >> That culture is so important. You know, I was looking at some anitab.org stats from 2022 and you know, we always talk about the number of women in technical roles. For a long time it's been hovering around that 25% range. The data from anitab.org showed from '22, it's now 27.6%. So, a little increase. But one of the biggest challenges still, and Tracy and I and our other co-host, Hannah, have been talking about this, is attrition. Attrition more than doubled last year. What are some of the things that Boeing is doing on the retention side, because that is so important especially as, you know, there's this pipeline leakage of women leaving technical roles. Tell us about what Boeing's, how they're invested. >> Yeah, sure. We actually have a publicly available Global Diversity Report that anybody can go and look at and see our statistics for our organization. Right now, off the top of my head, I think we're hovering at about 24% in the US for women in our company. It has been a male majority company for many years. We've invested heavily in increasing the number of women in roles. One interesting thing about this year that came out is that even though with the great resignation and those types of things, the attrition level between men and women were actually pretty close to being equal, which is like the first time in our history. Usually it tends on more women leaving. >> Lisa: That's a good sign. >> Right. >> Yes, that's a good sign. >> And we've actually focused on hiring and bringing in more women and diversity in our company. >> Yeah, some of the stats too from anitab.org talked about the increase, and I have to scroll back and find my notes, the increase in 51% more women being hired in 2022 than 2021 for technical roles. So the data, pun intended, is showing us. I mean, the data is there to show the impact that having females in executive leadership positions make from a revenue perspective. >> Tracy: Definitely. >> Companies are more profitable when there's women at the head, or at least in senior leadership roles. But we're seeing some positive trends, especially in terms of representation of women technologists. One of the things though that I found interesting, and I'm curious to get your thoughts on this, Rhonda, is that the representation of women technologists is growing in all areas, except interns. >> Rhonda: Hmm. >> So I think, we've got to go downstream. You teach, I have to go back to my notes on you, did my due diligence, R programming classes through Boeings Ed Wells program, this is for WSU College of Arts and Sciences, talk about what you teach and how do you think that intern kind of glut could be solved? >> Yeah. So, they're actually two separate programs. So I teach a data analytics course at Washington State University as an Adjunct Professor. And then the Ed Wells program is a SPEEA, which is an Aerospace Union, focused on bringing up more technology and skills to the actual workforce itself. So it's kind of a couple different audiences. One is more seasoned employees, right? The other one is our undergraduates. I teach a Capstone class, so it's a great way to introduce students to what it's actually like to work on an industry project. We partner with Google and Microsoft and Boeing on those. The idea is also that maybe those companies have openings for the students when they're done. Since it's Senior Capstone, there's not a lot of opportunities for internships. But the opportunities to actually get hired increase a little bit. In regards to Boeing, we've actually invested a lot in hiring more women interns. I think the number was 40%, but you'd have to double check. >> Lisa: That's great, that's fantastic. >> Tracy: That's way above average, I think. >> That's a good point. Yeah, it is above average. >> Double check on that. That's all from my memory. >> Is this your first WiDS, or have you been before? >> I did virtually last year. >> Okay. One of the things that I love, I love covering this event every year. theCUBE's been covering it since it's inception in 2015. But it's just the inspiration, the vibe here at Stanford is so positive. WiDS is a movement. It's not an initiative, an organization. There are going to be, I think annually this year, there will be 200 different events. Obviously today we're live on International Women's Day. 60 plus countries, 100,000 plus people involved. So, this is such a positive environment for women and men, because we need everybody, underrepresented minorities, to be able to understand the implication that data has across our lives. If we think about stripping away titles in industries, everybody is a consumer, not everybody, most of mobile devices. And we have this expectation, I was in Barcelona last week at a Mobile World Congress, we have this expectation that we're going to be connected 24/7. I can get whatever I want wherever I am in the world, and that's all data driven. And the average person that isn't involved in data science wouldn't understand that. At the same time, they have expectations that depend on organizations like Boeing being data driven so that they can get that experience that they expect in their consumer lives in any aspect of their lives. And that's one of the things I find so interesting and inspiring about data science. What are some of the things that keep you motivated to continue pursuing this? >> Yeah I will say along those lines, I think it's great to invest in K-12 programs for Data Literacy. I know one of my mentors and directors of the Data Analytics program, Dr. Nairanjana Dasgupta, we're really familiar with each other. So, she runs a WSU program for K-12 Data Literacy. It's also something that we strive for at Boeing, and we have an internal Data Literacy program because, believe it or not, most people are in business. And there's a lot of disconnect between interpreting and understanding data. For me, what kind of drives me to continue data science is that connection between people and data and how we use it to improve our world, which is partly why I work at Boeing too 'cause I feel that they produce products that people need like satellites and airplanes, >> Absolutely. >> and everything. >> Well, it's tangible, it's relatable. We can understand it. Can you do me a quick favor and define data literacy for anyone that might not understand what that means? >> Yeah, so it's just being able to understand elements of data, whether that's a bar chart or even in a sentence, like how to read a statistic and interpret a statistic in a sentence, for example. >> Very cool. >> Yeah. And sounds like Boeing's doing a great job in these programs, and also trying to hire more women. So yeah, I wanted to ask, do you think there's something that Boeing needs to work on? Or where do you see yourself working on say the next five years? >> Yeah, I think as a company, we always think that there's always room for improvement. >> It never, never stops. >> Tracy: Definitely. (laughs) >> I know workforce strategy is an area that they're currently really heavily investing in, along with safety. How do we build safer products for people? How do we help inform the public about things like Covid transmission in airports? For example, we had the Confident Traveler Initiative which was a big push that we had, and we had to be able to inform people about data models around Covid, right? So yeah, I would say our future is more about an investment in our people and in our culture from my perspective >> That's so important. One of the hardest things to change especially for a legacy organization like Boeing, is culture. You know, when I talk with CEO's or CIO's or COO's about what's your company's vision, what's your strategy? Especially those companies that are on that digital journey that have no choice these days. Everybody expects to have a digital experience, whether you're transacting an an Uber ride, you're buying groceries, or you're traveling by air. That culture sounds like Boeing is really focused on that. And that's impressive because that's one of the hardest things to morph and mold, but it's so essential. You know, as we look around the room here at WiDS it's obviously mostly females, but we're talking about women, underrepresented minorities. We're talking about men as well who are mentors and sponsors to us. I'd love to get your advice to your younger self. What would you tell yourself in terms of where you are now to become a leader in the technology field? >> Yeah, I mean, it's kind of an interesting question because I always try to think, live with no regrets to an extent. >> Lisa: I like that. >> But, there's lots of failures along the way. (Tracy laughing) I don't know if I would tell myself anything different because honestly, if I did, I wouldn't be where I am. >> Lisa: Good for you. >> I started out in fine arts, and I didn't end up there. >> That's good. >> Such a good point, yeah. >> We've been talking about that and I find that a lot at events like WiDS, is women have these zigzaggy patterns. I studied biology, I have a master's in molecular biology, I'm in media and marketing. We talked about transportable skills. There's a case I made many years ago when I got into tech about, well in science you learn the art of interpreting esoteric data and creating a story from it. And that's a transportable skill. But I always say, you mentioned failure, I always say failure is not a bad F word. It allows us to kind of zig and zag and learn along the way. And I think that really fosters thought diversity. And in data science, that is one of the things we absolutely need to have is that diversity and thought. You know, we talk about AI models being biased, we need the data and we need the diverse brains to help ensure that the biases are identified, extracted, and removed. Speaking of AI, I've been geeking out with ChatGPT. So, I'm on it yesterday and I ask it, "What's hot in data science?" And I was like, is it going to get that? What's hot? And it did it, it came back with trends. I think if I ask anything, "What's hot?", I should be to Paris Hilton, but I didn't. And so I was geeking out. One of the things I learned recently that I thought was so super cool is the CTO of OpenAI is a woman, Mira Murati, which I didn't know until over the weekend. Because I always think if I had to name top females in tech, who would they be? And I always default to Sheryl Sandberg, Carly Fiorina, Susan Wojcicki running YouTube. Who are some of the people in your history, in your current, that are really inspiring to you? Men, women, indifferent. >> Sure. I think Boeing is one of the companies where you actually do see a lot of women in leadership roles. I think we're one of the top companies with a number of women executives, actually. Susan Doniz, who's our Chief Information Officer, I believe she's actually slotted to speak at a WiDS event come fall. >> Lisa: Cool. >> So that will be exciting. Susan's actually relatively newer to Boeing in some ways. A Boeing time skill is like three years is still kind of new. (laughs) But she's been around for a while and she's done a lot of inspiring things, I think, for women in the organization. She does a lot with Latino communities and things like that as well. For me personally, you know, when I started at Boeing Ahmad Yaghoobi was one of my mentors and my Technical Lead. He came from Iran during a lot of hard times in the 1980s. His brother actually wrote a memoir, (laughs) which is just a fun, interesting fact. >> Tracy: Oh my God! >> Lisa: Wow! >> And so, I kind of gravitate to people that I can learn from that's not in my sphere, that might make me uncomfortable. >> And you probably don't even think about how many people you're influencing along the way. >> No. >> We just keep going and learning from our mentors and probably lose sight of, "I wonder how many people actually admire me?" And I'm sure there are many that admire you, Rhonda, for what you've done, going from anthropology to archeology. You mentioned before we went live you were really interested in photography. Keep going and really gathering all that breadth 'cause it's only making you more inspiring to people like us. >> Exactly. >> We thank you so much for joining us on the program and sharing a little bit about you and what brought you to WiDS. Thank you so much, Rhonda. >> Yeah, thank you. >> Tracy: Thank you so much for being here. >> Lisa: Yeah. >> Alright. >> For our guests, and for Tracy Zhang, this is Lisa Martin live at Stanford University covering the eighth Annual Women In Data Science Conference. Stick around. Next guest will be here in just a second. (gentle music)

(upbeat music) >> Hey everyone. Welcome back to "The Cube"'s live coverage of "Women in Data Science 2023". As every year we are here live at Stanford University, profiling some amazing women and men in the fields of data science. I have my co-host for this segment is Hannah Freitag. Hannah is from Stanford's Data Journalism program, really interesting, check it out. We're very pleased to welcome our first guest of the day fresh from the keynote stage, Gayatree Ganu, the VP of Data Science at Meta. Gayatree, It's great to have you on the program. >> Likewise, Thank you for having me. >> So you have a PhD in Computer Science. You shared some really cool stuff. Everyone knows Facebook, everyone uses it. I think my mom might be one of the biggest users (Gayatree laughs) and she's probably watching right now. People don't realize there's so much data behind that and data that drives decisions that we engage with. But talk to me a little bit about you first, PhD in Computer Science, were you always, were you like a STEM kid? Little Gayatree, little STEM, >> Yeah, I was a STEM kid. I grew up in Mumbai, India. My parents are actually pharmacists, so they were not like math or stats or anything like that, but I was always a STEM kid. I don't know, I think it, I think I was in sixth grade when we got our first personal computer and I obviously used it as a Pacman playing machine. >> Oh, that's okay. (all laugh) >> But I was so good at, and I, I honestly believe I think being good at games kind of got me more familiar and comfortable with computers. Yeah. I think I always liked computers, I, yeah. >> And so now you lead, I'm looking at my notes here, the Engagement Ecosystem and Monetization Data Science teams at Facebook, Meta. Talk about those, what are the missions of those teams and how does it impact the everyday user? >> Yeah, so the engagement is basically users coming back to our platform more, there's, no better way for users to tell us that they are finding value on the things that we are doing on Facebook, Instagram, WhatsApp, all the other products than coming back to our platform more. So the Engagement Ecosystem team is looking at trends, looking at where there are needs, looking at how users are changing their behaviors, and you know, helping build strategy for the long term, using that data knowledge. Monetization is very different. You know, obviously the top, top apex goal is have a sustainable business so that we can continue building products for our users. And so, but you know, I said this in my keynote today, it's not about making money, our mission statement is not, you know, maximize as much money as you can make. It's about building a meaningful connection between businesses, customers, users, and, you know especially in these last two or three funky, post-pandemic years, it's been such a big, an important thing to do for small businesses all over all, all around the world for users to find like goods and services and products that they care about and that they can connect to. So, you know, there is truly an connection between my engagement world and the monetization world. And you know, it's not very clear always till you go in to, like, you peel the layers. Everything we do in the ads world is also always first with users as our, you know, guiding principle. >> Yeah, you mentioned how you supported especially small businesses also during the pandemic. You touched a bit upon it in the keynote speech. Can you tell our audience what were like special or certain specific programs you implemented to support especially small businesses during these times? >> Yeah, so there are 200 million businesses on our platform. A lot of them small businesses, 10 million of them run ads. So there is a large number of like businesses on our platform who, you know use the power of social media to connect to the customers that matter to them, to like you, you know use the free products that we built. In the post-pandemic years, we built a lot of stuff very quickly when Covid first hit for business to get the word out, right? Like, they had to announce when special shopping hours existed for at-risk populations, or when certain goods and services were available versus not. We had grants, there's $100 million grant that we gave out to small businesses. Users could show sort of, you know show their support with a bunch of campaigns that we ran, and of course we continue running ads. Our ads are very effective, I guess, and, you know getting a very reliable connection with from the customer to the business. And so, you know, we've run all these studies. We support, I talked about two examples today. One of them is the largest black-owned, woman black-owned wine company, and how they needed to move to an online program and, you know, we gave them a grant, and supported them through their ads campaign and, you know, they saw 60% lift in purchases, or something like that. So, a lot of good stories, small stories, you know, on a scale of 200 million, that really sort of made me feel proud about the work we do. And you know, now more than ever before, I think people can connect so directly with businesses. You can WhatsApp them, I come from India, every business is on WhatsApp. And you can, you know, WhatsApp them, you can send them Facebook messages, and you can build this like direct connection with things that matter to you. >> We have this expectation that we can be connected anywhere. I was just at Mobile World Congress for MWC last week, where, obviously talking about connectivity. We want to be able to do any transaction, whether it's post on Facebook or call an Uber, or watch on Netflix if you're on the road, we expect that we're going to be connected. >> Yeah. >> And what we, I think a lot of us don't realize I mean, those of us in tech do, but how much data science is a facilitator of all of those interactions. >> Yeah! >> As we, Gayatree, as we talk about, like, any business, whether it is the black women-owned wine business, >> Yeah. >> great business, or a a grocer or a car dealer, everybody has to become data-driven. >> Yes. >> Because the consumer has the expectation. >> Yes. >> Talk about data science as a facilitator of just pretty much everything we are doing and conducting in our daily lives. >> Yeah, I think that's a great question. I think data science as a field wasn't really defined like maybe 15 years ago, right? So this is all in our lifetimes that we are seeing this. Even in data science today, People come from so many different backgrounds and bring their own expertise here. And I think we, you know, this conference, all of us get to define what that means and how we can bring data to do good in the world. Everything you do, as you said, there is a lot of data. Facebook has a lot of data, Meta has a lot of data, and how do we responsibly use this data? How do we use this data to make sure that we're, you know representing all diversity? You know, minorities? Like machine learning algorithms don't do well with small data, they do well with big data, but the small data matters. And how do you like, you know, bring that into algorithms? Yeah, so everything we do at Meta is very, very data-driven. I feel proud about that, to be honest, because while data gets a bad rap sometimes, having no data and making decisions in the blind is just the absolute worst thing you can do. And so, you know, we, the job as a data scientist at Facebook is to make sure that we use this data, use this responsibly, make sure that we are representing every aspect of the, you know, 3 billion users who come to our platform. Yeah, data serves all the products that we build here. >> The responsibility factor is, is huge. You know, we can't talk about AI without talking about ethics. One of the things that I was talking with Hannah and our other co-host, Tracy, about during our opening is something I just learned over the weekend. And that is that the CTO of ChatGPT is a woman. (Gayatree laughs) I didn't know that. And I thought, why isn't she getting more awareness? There's a lot of conversations with their CEO. >> Yeah. >> Everyone's using it, playing around with it. I actually asked it yesterday, "What's hot in Data Science?" (all laugh) I was like, should I have asked that to let itself in, what's hot? (Gayatree laughs) But it, I thought that was phenomenal, and we need to be talking about this more. >> Yeah. >> This is something that they're likening to the launch of the iPhone, which has transformed our lives. >> I know, it is. >> ChatGPT, and its chief technologist is a female, how great is that? >> And I don't know whether you, I don't know the stats around this, but I think CTO is even less, it's even more rare to have a woman there, like you have women CEOs because I mean, we are building upon years and years of women not choosing technical fields and not choosing STEM, and it's going to take some time, but yeah, yeah, she's a woman. Isn't it amazing? It's wonderful. >> Yes, there was a great, there's a great "Fast Company" article on her that I was looking at yesterday and I just thought, we need to do what we can to help spread, Mira Murati is her name, because what she's doing is, one of the biggest technological breakthroughs we may ever see in our lifetime. It gives me goosebumps just thinking about it. (Gayatree laughs) I also wanted to share some stats, oh, sorry, go ahead, Hannah. >> Yeah, I was going to follow up on the thing that you mentioned that we had many years with like not enough women choosing a career path in STEM and that we have to overcome this trend. What are some, like what is some advice you have like as the Vice-President Data Science? Like what can we do to make this feel more, you know, approachable and >> Yeah. >> accessible for women? >> Yeah, I, there's so much that we have done already and you know, want to continue, keep doing. Of course conferences like these were, you know and I think there are high school students here there are students from my Alma Mater's undergrad year. It's amazing to like get all these women together to get them to see what success could look like. >> Yeah. >> What being a woman leader in this space could look like. So that's, you know, that's one, at Meta I lead recruiting at Meta and we've done a bunch to sort of open up the thinking around data science and technical jobs for women. Simple things like what you write in your job description. I don't know whether you know this, or this is a story you've heard before, when you see, when you have a job description and there are like 10 things that you need to, you know be good at to apply to this job, a woman sees those 10 and says, okay, I don't meet the qualifications of one of them and she doesn't apply. And a man sees one that he meets the qualifications to and he applies. And so, you know, there's small things you can do, and just how you write your job description, what goals you set for diversity and inclusion for your own organization. We have goals, Facebook's always been pretty up there in like, you know, speaking out for diversity and Sheryl Sandberg has been our Chief Business Officer for a very long time and she's been, like, amazing at like pushing from more women. So yeah, every step of the way, I think, we made a lot of progress, to be honest. I do think women choose STEM fields a lot more than they did. When I did my Computer Science I was often one of one or two women in the Computer Science class. It takes some time to, for it to percolate all the way to like having more CTOs and CEOs, >> Yeah. >> but it's going to happen in our lifetime, and you know, three of us know this, women are going to rule the world, and it (laughs) >> Drop the mic, girl! >> And it's going to happen in our lifetime, so I'm excited about it. >> And we have responsibility in helping make that happen. You know, I'm curious, you were in STEM, you talked about Computer Science, being one of the only females. One of the things that the nadb.org data from 2022 showed, some good numbers, the number of women in technical roles is now 27.6%, I believe, so up from 25, it's up in '22, which is good, more hiring of women. >> Yeah. >> One of the biggest challenges is attrition. What keeps you motivated? >> Yeah. >> To stay what, where you are doing what you're doing, managing a family and helping to drive these experiences at Facebook that we all expect are just going to happen? >> Yeah, two things come to mind. It does take a village. You do need people around you. You know, I'm grateful for my husband. You talked about managing a family, I did the very Indian thing and my parents live with us, and they help take care of the kids. >> Right! (laughs) >> (laughs) My kids are young, six and four, and I definitely needed help over the last few years. It takes mentors, it takes other people that you look up to, who've gone through all of those same challenges and can, you know, advise you to sort of continue working in the field. I remember when my kid was born when he was six months old, I was considering quitting. And my husband's like, to be a good role model for your children, you need to continue working. Like, just being a mother is not enough. And so, you know, so that's one. You know, the village that you build around you your supporters, your mentors who keep encouraging you. Sheryl Sandberg said this to me in my second month at Facebook. She said that women drop out of technical fields, they become managers, they become sort of administrative more, in their nature of their work, and her advice was, "Don't do that, Don't stop the technical". And I think that's the other thing I'd say to a lot of women. Technical stuff is hard, but you know, keeping up with that and keeping sort of on top of it actually does help you in the long run. And it's definitely helped me in my career at Facebook. >> I think one of the things, and Hannah and I and Tracy talked about this in the open, and I think you'll agree with us, is the whole saying of you can't be what you can't see, and I like to way, "Well, you can be what you can see". That visibility, the great thing that WiDS did, of having you on the stage as a speaker this morning so people can understand, everyone, like I said, everyone knows Meta, >> Yeah. >> everyone uses Facebook. And so it's important to bring that connection, >> Yeah. >> of how data is driving the experiences, the fact that it's User First, but we need to be able to see women in positions, >> Yes. >> like you, especially with Sheryl stepping down moving on to something else, or people that are like YouTube influencers, that have no idea that the head of YouTube for a very long time, Susan Wojcicki is a woman. >> (laughs) Yes. Who pioneered streaming, and I mean how often do you are you on YouTube every day? >> Yep, every day. >> But we have to be able to see and and raise the profile of these women and learn from them and be inspired, >> Absolutely. >> to keep going and going. I like what I do, I'm making a difference here. >> Yeah, yeah, absolutely. >> And I can be the, the sponsor or the mentor for somebody down the road. >> Absolutely. >> Yeah, and then referring back to what we talked in the beginning, show that data science is so diverse and it doesn't mean if you're like in IT, you're like sitting in your dark room, >> Right. (laughs) >> coding all day, but you know, >> (laughs) Right! >> to show the different facets of this job and >> Right! >> make this appealing to women, >> Yeah. for sure. >> And I said this in my keynote too, you know, one of the things that helped me most is complimenting the data and the techniques and the algorithms with how you work with people, and you know, empathy and alignment building and leadership, strategic thinking. And I think honestly, I think women do a lot of this stuff really well. We know how to work with people and so, you know, I've seen this at Meta for sure, like, you know, all of these skills soft skills, as we call them, go a long way, and like, you know, doing the right things and having a lasting impact. And like I said, women are going to rule the world, you know, in our lifetimes. (laughs) >> Oh, I can't, I can't wait to see that happen. There's some interesting female candidates that are already throwing their hats in the ring for the next presidential election. >> Yes. >> So we'll have to see where that goes. But some of the things that are so interesting to me, here we are in California and Palo Alto, technically Stanford is its own zip code, I believe. And we're in California, we're freaking out because we've gotten so much rain, it's absolutely unprecedented. We need it, we had a massive drought, an extreme drought, technically, for many years. I've got friends that live up in Tahoe, I've been getting pictures this morning of windows that are >> (laughs) that are covered? >> Yes, actually, yes. (Gayatree laughs) That, where windows like second-story windows are covered in snow. >> Yeah. >> Climate change. >> Climate change. >> There's so much that data science is doing to power and power our understanding of climate change whether it's that, or police violence. >> Yeah. (all talk together) >> We had talk today on that it was amazing. >> Yes. So I want more people to know what data science is really facilitating, that impacts all of us, whether you're in a technical role or not. >> And data wins arguments. >> Yes, I love that! >> I said this is my slide today, like, you know, there's always going to be doubters and naysayers and I mean, but there's hard evidence, there's hard data like, yeah. In all of these fields, I mean the data that climate change, the data science that we have done in the environmental and climate change areas and medical, and you know, medicine professions just so much, so much more opportunity, and like, how much we can learn more about the world. >> Yeah. >> Yeah, it's a pretty exciting time to be a data scientist. >> I feel like, we're just scratching the surface. >> Yeah. >> With the potential and the global impact that we can make with data science. Gayatree, it's been so great having you on theCUBE, thank you. >> Right, >> Thank you so much, Gayatree. >> So much, I love, >> Thank you. >> I'm going to take Data WiD's arguments into my personal life. (Gayatree laughs) I was actually just, just a quick anecdote, funny story. I was listening to the radio this morning and there was a commercial from an insurance company and I guess the joke is, it's an argument between two spouses, and the the voiceover comes in and says, "Let's watch a replay". I'm like, if only they, then they got the data that helped the woman win the argument. (laughs) >> (laughs) I will warn you it doesn't always help with arguments I have with my husband. (laughs) >> Okay, I'm going to keep it in the middle of my mind. >> Yes! >> Gayatree, thank you so much. >> Thank you so much, >> for sharing, >> Thank you both for the opportunity. >> And being a great female that we can look up to, we really appreciate your insights >> Oh, likewise. >> and your time. >> Thank you. >> All right, for our guest, for Hannah Freitag, I'm Lisa Martin, live at Stanford University covering "Women in Data Science '23". Stick around, our next guest joins us in just a minute. (upbeat music) I have been in the software and technology industry for over 12 years now, so I've had the opportunity as a marketer to really understand and interact with customers across the entire buyer's journey. Hi, I'm Lisa Martin and I'm a host of theCUBE. (upbeat music) Being a host on theCUBE has been a dream of mine for the last few years. I had the opportunity to meet Jeff and Dave and John at EMC World a few years ago and got the courage up to say, "Hey, I'm really interested in this. I love talking with customers, gimme a shot, let me come into the studio and do an interview and see if we can work together". I think where I really impact theCUBE is being a female in technology. We interview a lot of females in tech, we do a lot of women in technology events and one of the things I.

(ambient music) >> Good morning, everyone. Lisa Martin with theCUBE, live at the eighth Annual Women in Data Science Conference. This is one of my absolute favorite events of the year. We engage with tons of great inspirational speakers, men and women, and what's happening with WiDS is a global movement. I've got two fabulous co-hosts with me today that you're going to be hearing and meeting. Please welcome Tracy Zhang and Hannah Freitag, who are both from the sata journalism program, master's program, at Stanford. So great to have you guys. >> So excited to be here. >> So data journalism's so interesting. Tracy, tell us a little bit about you, what you're interested in, and then Hannah we'll have you do the same thing. >> Yeah >> Yeah, definitely. I definitely think data journalism is very interesting, and in fact, I think, what is data journalism? Is definitely one of the big questions that we ask during the span of one year, which is the length of our program. And yeah, like you said, I'm in this data journalism master program, and I think coming in I just wanted to pivot from my undergrad studies, which is more like a traditional journalism, into data. We're finding stories through data, so that's why I'm also very excited about meeting these speakers for today because they're all, they have different backgrounds, but they all ended up in data science. So I think they'll be very inspirational and I can't wait to talk to them. >> Data in stories, I love that. Hannah, tell us a little bit about you. >> Yeah, so before coming to Stanford, I was a research assistant at Humboldt University in Berlin, so I was in political science research. And I love to work with data sets and data, but I figured that, for me, I don't want this story to end up in a research paper, which is only very limited in terms of the audience. And I figured, okay, data journalism is the perfect way to tell stories and use data to illustrate anecdotes, but to make it comprehensive and accessible for a broader audience. So then I found this program at Stanford and I was like, okay, that's the perfect transition from political science to journalism, and to use data to tell data-driven stories. So I'm excited to be in this program, I'm excited for the conference today and to hear from these amazing women who work in data science. >> You both brought up great points, and we were chatting earlier that there's a lot of diversity in background. >> Tracy: Definitely. >> Not everyone was in STEM as a young kid or studied computer science. Maybe some are engineering, maybe some are are philosophy or economic, it's so interesting. And what I find year after year at WiDS is it brings in so much thought diversity. And that's what being data-driven really demands. It demands that unbiased approach, that diverse, a spectrum of diverse perspectives, and we definitely get that at WiDS. There's about 350 people in person here, but as I mentioned in the opening, hundreds of thousands will engage throughout the year, tens of thousands probably today at local events going on across the globe. And it just underscores the importance of every organization, whether it's a bank or a grocer, has to be data-driven. We have that expectation as consumers in our consumer lives, and even in our business lives, that I'm going to engage with a business, whatever it is, and they're going to know about me, they're going to deliver me a personalized experience that's relevant to me and my history. And all that is powered by data science, which is I think it's fascinating. >> Yeah, and the great way is if you combine data with people. Because after all, large data sets, they oftentimes consist of stories or data that affects people. And to find these stories or advanced research in whatever fields, maybe in the financial business, or in health, as you mentioned, the variety of fields, it's very powerful, powerful tool to use. >> It's a very power, oh, go ahead Tracy. >> No, definitely. I just wanted to build off of that. It's important to put a face on data. So a dataset without a name is just some numbers, but if there's a story, then I think it means something too. And I think Margot was talking about how data science is about knowing or understanding the past, I think that's very interesting. That's a method for us to know who we are. >> Definitely. There's so many opportunities. I wanted to share some of the statistics from AnitaB.org that I was just looking at from 2022. We always talk at events like WiDS, and some of the other women in tech things, theCUBE is very much pro-women in tech, and has been for a very long, since the beginning of theCUBE. But we've seen the numbers of women technologists historically well below 25%, and we see attrition rates are high. And so we often talk about, well, what can we do? And part of that is raising the awareness. And that's one of the great things about WiDS, especially WiDS happening on International Women's Day, today, March 8th, and around event- >> Tracy: A big holiday. >> Exactly. But one of the nice things I was looking at, the AnitaB.org research, is that representation of tech women is on the rise, still below pre-pandemic levels, but it's actually nearly 27% of women in technical roles. And that's an increase, slow increase, but the needle is moving. We're seeing much more gender diversity across a lot of career levels, which is exciting. But some of the challenges remain. I mean, the representation of women technologists is growing, except at the intern level. And I thought that was really poignant. We need to be opening up that pipeline and going younger. And you'll hear a lot of those conversations today about, what are we doing to reach girls in grade school, 10 year olds, 12 year olds, those in high school? How do we help foster them through their undergrad studies- >> And excite them about science and all these fields, for sure. >> What do you think, Hannah, on that note, and I'll ask you the same question, what do you think can be done? The theme of this year's International Women's Day is Embrace Equity. What do you think can be done on that intern problem to help really dial up the volume on getting those younger kids interested, one, earlier, and two, helping them stay interested? >> Yeah. Yeah, that's a great question. I think it's important to start early, as you said, in school. Back in the day when I went to high school, we had this one day per year where we could explore as girls, explore a STEM job and go into the job for one day and see how it's like to work in a, I dunno, in IT or in data science, so that's a great first step. But as you mentioned, it's important to keep girls and women excited about this field and make them actually pursue this path. So I think conferences or networking is very powerful. Also these days with social media and technology, we have more ability and greater ways to connect. And I think we should even empower ourselves even more to pursue this path if we're interested in data science, and not be like, okay, maybe it's not for me, or maybe as a woman I have less chances. So I think it's very important to connect with other women, and this is what WiDS is great about. >> WiDS is so fantastic for that network effect, as you talked about. It's always such, as I was telling you about before we went live, I've covered five or six WiDS for theCUBE, and it's always such a day of positivity, it's a day of of inclusivity, which is exactly what Embrace Equity is really kind of about. Tracy, talk a little bit about some of the things that you see that will help with that hashtag Embrace Equity kind of pulling it, not just to tech. Because we're talking and we saw Meta was a keynote who's going to come to talk with Hannah and me in a little bit, we see Total Energies on the program today, we see Microsoft, Intuit, Boeing Air Company. What are some of the things you think that can be done to help inspire, say, little Tracy back in the day to become interested in STEM or in technology or in data? What do you think companies can and should be doing to dial up the volume for those youngsters? >> Yeah, 'cause I think somebody was talking about, one of the keynote speakers was talking about how there is a notion that girls just can't be data scientists. girls just can't do science. And I think representation definitely matters. If three year old me see on TV that all the scientists are women, I think I would definitely have the notion that, oh, this might be a career choice for me and I can definitely also be a scientist if I want. So yeah, I think representation definitely matters and that's why conference like this will just show us how these women are great in their fields. They're great data scientists that are bringing great insight to the company and even to the social good as well. So yeah, I think that's very important just to make women feel seen in this data science field and to listen to the great woman who's doing amazing work. >> Absolutely. There's a saying, you can't be what you can't see. >> Exactly. >> And I like to say, I like to flip it on its head, 'cause we can talk about some of the negatives, but there's a lot of positives and I want to share some of those in a minute, is that we need to be, that visibility that you talked about, the awareness that you talked about, it needs to be there but it needs to be sustained and maintained. And an organization like WiDS and some of the other women in tech events that happen around the valley here and globally, are all aimed at raising the profile of these women so that the younger, really, all generations can see what they can be. We all, the funny thing is, we all have this expectation whether we're transacting on Uber ride or we are on Netflix or we're buying something on Amazon, we can get it like that. They're going to know who I am, they're going to know what I want, they're going to want to know what I just bought or what I just watched. Don't serve me up something that I've already done that. >> Hannah: Yeah. >> Tracy: Yeah. >> So that expectation that everyone has is all about data, though we don't necessarily think about it like that. >> Hannah: Exactly. >> Tracy: Exactly. >> But it's all about the data that, the past data, the data science, as well as the realtime data because we want to have these experiences that are fresh, in the moment, and super relevant. So whether women recognize it or not, they're data driven too. Whether or not you're in data science, we're all driven by data and we have these expectations that every business is going to meet it. >> Exactly. >> Yeah. And circling back to young women, I think it's crucial and important to have role models. As you said, if you see someone and you're younger and you're like, oh I want to be like her. I want to follow this path, and have inspiration and a role model, someone you look up to and be like, okay, this is possible if I study the math part or do the physics, and you kind of have a goal and a vision in mind, I think that's really important to drive you. >> Having those mentors and sponsors, something that's interesting is, I always, everyone knows what a mentor is, somebody that you look up to, that can guide you, that you admire. I didn't learn what a sponsor was until a Women in Tech event a few years ago that we did on theCUBE. And I was kind of, my eyes were open but I didn't understand the difference between a mentor and a sponsor. And then it got me thinking, who are my sponsors? And I started going through LinkedIn, oh, he's a sponsor, she's a sponsor, people that help really propel you forward, your recommenders, your champions, and it's so important at every level to build that network. And we have, to your point, Hannah, there's so much potential here for data drivenness across the globe, and there's so much potential for women. One of the things I also learned recently , and I wanted to share this with you 'cause I'm not sure if you know this, ChatGPT, exploding, I was on it yesterday looking at- >> Everyone talking about it. >> What's hot in data science? And it was kind of like, and I actually asked it, what was hot in data science in 2023? And it told me that it didn't know anything prior to 2021. >> Tracy: Yes. >> Hannah: Yeah. >> So I said, Oh, I'm so sorry. But everyone's talking about ChatGPT, it is the most advanced AI chatbot ever released to the masses, it's on fire. They're likening it to the launch of the iPhone, 100 million-plus users. But did you know that the CTO of ChatGPT is a woman? >> Tracy: I did not know, but I learned that. >> I learned that a couple days ago, Mira Murati, and of course- >> I love it. >> She's been, I saw this great profile piece on her on Fast Company, but of course everything that we're hearing about with respect to ChatGPT, a lot on the CEO. But I thought we need to help dial up the profile of the CTO because she's only 35, yet she is at the helm of one of the most groundbreaking things in our lifetime we'll probably ever see. Isn't that cool? >> That is, yeah, I completely had no idea. >> I didn't either. I saw it on LinkedIn over the weekend and I thought, I have to talk about that because it's so important when we talk about some of the trends, other trends from AnitaB.org, I talked about some of those positive trends. Overall hiring has rebounded in '22 compared to pre-pandemic levels. And we see also 51% more women being hired in '22 than '21. So the data, it's all about data, is showing us things are progressing quite slowly. But one of the biggest challenges that's still persistent is attrition. So we were talking about, Hannah, what would your advice be? How would you help a woman stay in tech? We saw that attrition last year in '22 according to AnitaB.org, more than doubled. So we're seeing women getting into the field and dropping out for various reasons. And so that's still an extent concern that we have. What do you think would motivate you to stick around if you were in a technical role? Same question for you in a minute. >> Right, you were talking about how we see an increase especially in the intern level for women. And I think if, I don't know, this is a great for a start point for pushing the momentum to start growth, pushing the needle rightwards. But I think if we can see more increase in the upper level, the women representation in the upper level too, maybe that's definitely a big goal and something we should work towards to. >> Lisa: Absolutely. >> But if there's more representation up in the CTO position, like in the managing level, I think that will definitely be a great factor to keep women in data science. >> I was looking at some trends, sorry, Hannah, forgetting what this source was, so forgive me, that was showing that there was a trend in the last few years, I think it was Fast Company, of more women in executive positions, specifically chief operating officer positions. What that hasn't translated to, what they thought it might translate to, is more women going from COO to CEO and we're not seeing that. We think of, if you ask, name a female executive that you'd recognize, everyone would probably say Sheryl Sandberg. But I was shocked to learn the other day at a Women in Tech event I was doing, that there was a survey done by this organization that showed that 78% of people couldn't identify. So to your point, we need more of them in that visible role, in the executive suite. >> Tracy: Exactly. >> And there's data that show that companies that have women, companies across industries that have women in leadership positions, executive positions I should say, are actually more profitable. So it's kind of like, duh, the data is there, it's telling you this. >> Hannah: Exactly. >> Right? >> And I think also a very important point is work culture and the work environment. And as a woman, maybe if you enter and you work two or three years, and then you have to oftentimes choose, okay, do I want family or do I want my job? And I think that's one of the major tasks that companies face to make it possible for women to combine being a mother and being a great data scientist or an executive or CEO. And I think there's still a lot to be done in this regard to make it possible for women to not have to choose for one thing or the other. And I think that's also a reason why we might see more women at the entry level, but not long-term. Because they are punished if they take a couple years off if they want to have kids. >> I think that's a question we need to ask to men too. >> Absolutely. >> How to balance work and life. 'Cause we never ask that. We just ask the woman. >> No, they just get it done, probably because there's a woman on the other end whose making it happen. >> Exactly. So yeah, another thing to think about, another thing to work towards too. >> Yeah, it's a good point you're raising that we have this conversation together and not exclusively only women, but we all have to come together and talk about how we can design companies in a way that it works for everyone. >> Yeah, and no slight to men at all. A lot of my mentors and sponsors are men. They're just people that I greatly admire who saw raw potential in me 15, 18 years ago, and just added a little water to this little weed and it started to grow. In fact, theCUBE- >> Tracy: And look at you now. >> Look at me now. And theCUBE, the guys Dave Vellante and John Furrier are two of those people that are sponsors of mine. But it needs to be diverse. It needs to be diverse and gender, it needs to include non-binary people, anybody, shouldn't matter. We should be able to collectively work together to solve big problems. Like the propaganda problem that was being discussed in the keynote this morning with respect to China, or climate change. Climate change is a huge challenge. Here, we are in California, we're getting an atmospheric river tomorrow. And Californians and rain, we're not so friendly. But we know that there's massive changes going on in the climate. Data science can help really unlock a lot of the challenges and solve some of the problems and help us understand better. So there's so much real-world implication potential that being data-driven can really lead to. And I love the fact that you guys are studying data journalism. You'll have to help me understand that even more. But we're going to going to have great conversations today, I'm so excited to be co-hosting with both of you. You're going to be inspired, you're going to learn, they're going to learn from us as well. So let's just kind of think of this as a community of men, women, everything in between to really help inspire the current generations, the future generations. And to your point, let's help women feel confident to be able to stay and raise their hand for fast-tracking their careers. >> Exactly. >> What are you guys, last minute, what are you looking forward to most for today? >> Just meeting these great women, I can't wait. >> Yeah, learning from each other. Having this conversation about how we can make data science even more equitable and hear from the great ideas that all these women have. >> Excellent, girls, we're going to have a great day. We're so glad that you're here with us on theCUBE, live at Stanford University, Women in Data Science, the eighth annual conference. I'm Lisa Martin, my two co-hosts for the day, Tracy Zhang, Hannah Freitag, you're going to be seeing a lot of us, we appreciate. Stick around, our first guest joins Hannah and me in just a minute. (ambient music)

(upbeat music) >> Hello, everyone. Welcome to theCUBE's special presentation of International Women's Day. I'm John Furrier, host of theCUBE. Jenni Troutman is here, Director of Products and Services, and Training and Certification at AWS, and Heather Ruden, Director of Education Programs, Training and Certification. Thanks for coming on theCUBE and for the International Women's Day special program. >> Thanks so much for having us. >> So, I'll just get it out of the way. I'm a big fan of what you guys do. I've been shouting at the top of my lungs, "It's free. Get cloud training and you'll have a six figure job." Pretty much. I'm over amplifying. But this is really a big opportunity in the industry, education and the skills gap, and the skill velocities that's changing. New roles are coming on around cloud native, cloud native operators, cybersecurity. There's so much excitement going on around the industry, and all these open positions, and they need new talent. So you can't get a degree for some of these things. So, nope, it doesn't matter what school you went to, everyone's kind of level. This is a really big deal. So, Heather, share with us your thoughts as well on this topic. Jenni, you too. Like, where are you guys at? 'Cause this is a big opportunity for women and anyone to level up in the industry. >> Absolutely. So I'll jump in and then I'll hand it over to Jenni. We're your dream team here. We can talk about both sides of this. So I run a set of programs here at AWS that are really intended to help build the next generation of cloud builders. And we do that with a variety of programs, whether it is targeting young learners from kind of 12 and up. We have AWS GetIT that is designed to get women ambassadors or women mentors in front of girls 12 to 14 and get them curious about a career in STEM. We also have a program that is all digital online. It's available in 11 languages. It's got hundreds of courses. That's called AWS Educate that is designed to do exactly what you just talked about, expose the opportunities and start building cloud skills for learners at age 13 and up. They can go online and register with an email and start learning. We want them to understand not only what the opportunity is for them, but the ways that they can help influence and bring more diversity and more inclusion and into the cloud technology space, and just keep building all those amazing builders that we need here for our customers and partners. And those are the programs that I manage, but Jenni also has an amazing program, a set of programs. And so I'll hand it over to her as you get into the professional side of this things. >> So Jenni, you're on the product side. You've got the keys to the kingdom on all the materials and shaping it. What's your view on this? 'Cause this is a huge opportunity and it's always changing. What's the latest and greatest? >> It is a massive opportunity and to give you a sense, there was a study in '21 where IT executives said that talent availability is the biggest challenge to emerging tech adoption. 64% of IT executives said that up from only 4% the year before. So the challenge is growing really fast, which for everyone that's ready to go out there and learn and try something new is a massive opportunity. And that's really why I'm here. We provide all kinds of learning experiences for people across different cloud technologies to be able to not only gain the knowledge around cloud, but also the confidence to be able to build in the cloud. And so we look across different learner levels, different roles, different opportunities, and we provide those experiences where people can actually get hands-on in a totally risk-free environment and practice building in the cloud so they can go and be ready to get their certifications, their AWS certifications, give them the credentials to be able to show an employer they can do it, and then go out and get these jobs. It's really exciting. And we go kind of end to end from the very beginning. What is cloud? I want to know what it is all the way through to I can prove that I can build in the cloud and I'm ready for a job. >> So Jenni, you nailed that confidence word. I think I want to double click on that. And Heather, you talked about you're the dream team. You guys, you're the go to market, you bring this to the marketplace. Jenni, you get the products. This is the key, but to me the the international women days angle is, is that what I hear over and over again is that, "It's too technical. I'm not qualified." It can be scary. We had a guest on who has two double E degrees in robotics and aerospace and she's hard charging. She almost lost her confidence twice she said in her career. But she was hard charging. It can get scary, but also the ability to level up fast is just as good. So if you can break through that confidence and keep the curiosity and be a builder, talk about that dynamic 'cause you guys are in the middle of it, you're in the industry, how do you handle that? 'Cause I think that's a big thing that comes up over and over again. And confidence is not just women, it's men too. But women can always, that comes up as a theme. >> It is. It is a big challenge. I mean, I've struggled with it personally and I mentor a lot of women and that is the number one challenge that is holding women back from really being able to advance is the confidence to step out there and show what they can do. And what I love about some of the products we've put out recently is we have AWS Skill Builder. You can go online, you can get all kinds of free core training and if you want to go deeper, you can go deeper. And there's a lot of different options on there. But what it does is not only gives you that based knowledge, but you can actually go in. We have something called AWS Labs. You can go in and you can actually practice on the AWS console with the services that people are using in their jobs every day without any risk of doing something that is going to blow up in your face. You're not going to suddenly get this big AWS bill. You're not going to break something that's out there running. You just go in. It's your own little environment that gets wiped when you're done and you can practice. And there's lots of different ways to learn as well. So if you go in there and you're watching a video and to your point you're like, "Oh my gosh, this is too technical. I can't understand it. I don't know what I'm going to go do." You can go another route. There's something called AWS Cloud Quest. It's a game. You go in and it's like you're gaming and it walks you through. You're actually in a virtual world. You're walking through and it's telling you, "Hey, go build this and if you need help, here's hints and here's tips." And it continues to build on itself. So you're learning and you're applying practical skills and it's at your own pace. You don't have to watch somebody else talking that is going at a pace that maybe accelerates beyond what you're ready. You can do it at your own pace, you can redo it, you can try it again until you feel confident that you know it and you're really ready to move on to the next thing. Personally, I find that hugely valuable. I go in and do these myself and I sit there and I have a lot of engineers on my team, very smart people. And I have my own imposter syndrome. I get nervous to go talk to them. Like, are they going to think I'm totally lost? And so I go in and I learn some of this myself by experiment. And then I feel like, okay, now I can go ask them some intelligent questions and they're not going to be like, "Oh gosh, my leader is totally unaware of what we're doing." And so I think that we all struggle with confidence. I think everybody does, but I see it especially in women as I mentor them. And that's what I encourage them to do is go and on your own time, practice a bit, get a little bit of experience and once you feel like you can throw a couple words out there that you know what they mean and suddenly other people look at you like, "Oh, she knows what she's talking about." And you can kind of get past that feeling. >> Well Jenni, you nailed it. Heather, she just mentioned she's in the job and she's going and she's still leveling up. That's the end when you're in, but it's also the barriers to entry are lowering. You guys are doing a good job of getting people in, but also growing fast too. So there's two dynamics at play here. How do people do this? What's the playbook? Because I think that's really key, easy to get in. And then once you're in, you can level up fast at your own pace to ride the wave. And then there's new stuff coming. I mean, every re:Invent there's 5,000 announcements. So it's like zillion new things and AI taught now. >> re:Invent is a perfect example of that ongoing imposter syndrome or confidence check for all of us. I think something that that Jenni said too is we really try and meet learners where they are and make sure that we have the support, whether it's accessibility requirements or we have the content that is built for the age that we're talking to, or we have a workforce development program called re/Start that is for people that have very little tech experience and really want to talk about a career in cloud, but they need a little bit more handholding. They need a combination of instructor-led and digital. But then we have AWS educators, I mentioned. If you want to be more self-directed, all of these tools are intended to work well together and to be complimentary and to take you on a journey as a learner. And the more skills you have, the more you increase your knowledge, the more you can take on more. But meeting folks where they are with a variety of programs, tools, languages, and accessibility really helps ensure that we can do that for learners throughout the world. >> That's awesome. Let's get into it. Let's get into the roadmaps of people and their personas. And you guys can share the programs that you have and where people could fit in. 'Cause this comes up a lot when I talk to folks. There's the young person who's I'm a gamer or whatever, I want to get a job. I'm in high school or an elementary or I want to tinker around or I'm in college or I'm learning, I'm an entry level kind of entry. Then you have the re-skilling. I'm going to change my careers, I'm kind of bored, I want to do something compelling. How do I get into the cloud game? And then the advanced re-skill is I want to get into cyber and AI and then there's other. Could you break down? Did I get that right or did I miss anything? And then what's available for those kind of lanes? So those persona lanes? >> Well, let's see, I could start with maybe the high schooler stuff and then we can bring Jenni in as well. I would say a great place to start for anyone is aws.amazon.com/training. That's going to give them the full suite of options that they could take on. If you're in high school, you can go onto AWS Educate. All you need is an email. And if you're 13 years and older, you can start exploring the types of jobs that are available in the cloud and you could start taking some introductory classes. You can do some of those labs in a safe environment that Jenni mentioned. That's a great place to start. If you are in an environment where you have an educator that is willing to go on this with you, this journey with you, we have this AWS GetIT program that is, again, educator-led. So it's an afterschool or it's an a program where we match mentors and students up with cloud professionals and they do some real-time experimentation. They build an app, they work on things together, and do a presentation at the end. The other thing I would say too is that if you are in a university, I would double check and see if the AWS Academy curriculum is already in your university. And if so, explore some of those classes there. We have instructor-led, educator-ready. course curriculum that we've designed that help people get to those certifications and get closer to those jobs and as well as hopefully then lead people right into skill builder and all the things that Jenni talked about to help them as they start out in a professional environment. >> So is the GetIT, is that an instructor-led that the person has to find someone for? Or is this available for them? >> It is through teachers. It's through educators. We are in, we've reached over 19,000 students. We're available in eight countries. There are ways for educators to lead this, but we want to make sure that we are helping the kids be successful and giving them an educator environment to do that. If they want to do it on their own, then they can absolutely go through AWS Educate or even and to explore where they want to get started. >> So what about someone who's educated in their middle of their career, might want to switch from being a biologist to a cloud cybersecurity guru or a cloud native operator? >> Yeah, so in that case, AWS re/Start is one of the great program for them to explore. We run that program with collaborating organizations in 160 cities in 80 countries throughout the world. That is a multi-week cohort-based program where we do take folks through a very clear path towards certification and job skilling that will help them get into those opportunities. Over 98% of the cohorts, the graduates of those cohorts get an interview and are hopefully on their path to getting a job. So that really has global reach. The partnership with collaborating organizations helps us ensure that we find communities that are often unreached by cloud skills training and we really work to keep a diverse focus on those cohorts and bring those folks into the cloud. >> Okay. Jenni, you've got the Skill Builder action here. What's going on on your side? Because you must have to manage all the change. I mean, AI is hot right now. I'm sure you're cranking away on curriculum and content for SageMaker, large language models, computer vision, cybersecurity. >> We do. There are a lot of options. >> How is your world? Tell us about what people can take out of way from your side. >> Yeah. So a great way to think about it is if they're already out in the workforce or they're entering the workforce, but they are technical, have technical skills is what are the roles that are interesting in the technologies that are interesting. Because the way we put out our training and our certifications is aligned to paths. So if you're look interested in a specific role. If you're interested in architecting a cloud environment or in security as you mentioned, and you want to go deep in security, there are AWS certifications that give you that. If you achieve them, they're very difficult. But if you work to them and achieve them, they give you the credential that you can take to an employer and say, "Look, I can do this job." And they are in very high demand. In fact that's where if you look at some of the publications that have come out, they talk about, what are people making if they have different certifications? What are the most in-demand certifications that are out there? And those are what help people get jobs. And so you identify what is that role or that technology area I want to learn. And then you have multiple options for how you build those skills depending on how you want to learn. And again, that's really our focus, is on providing experiences based on how people learn and making it accessible to them. 'Cause not everybody wants to learn in the same way. And so there is AWS Skill Builder where people can go learn on their own that is really great particularly for people who maybe are already working and have to learn in the evenings, on the weekends. People who like to learn at their own pace, who just want to be hands-on, but are self-starters. And they can get those whole learning plans through there all the way aligned to the certification and then they can go get their certification. There's also classroom training. So a lot of people maybe want to do continuous learning through an online, but want to go really deep with an expert in the room and maybe have a more focused period of time if they can go for a couple days. And so they can do classroom training. We provide a lot of classroom training. We have partners all over the globe who provide classroom training. And so there's that and what we find to be the most powerful is when you couple the two. If you can really get deep, you have an expert, you can ask questions, but first before you go do that, you get some of that foundational that you've kind of learned on your own. And then after you go back and reinforce, you go back online, you try out things that maybe you learned in the classroom, but you didn't quite, you hadn't used it enough yet to quite know how to do it. Now you can go back and actually use it, experiment and play around. And so we really encourage that kind of, figure out what are some areas you're interested in, go learn it and then go get a job and continue to learn because then once you learn that first area, you start to build confidence in it. Suddenly other areas become interesting. 'Cause as you said, cloud is changing fast. And once you learn a space, first of all you have to keep going back to stay up on it as it changes. But you quickly find that there are other areas that are really interesting too. >> I've observed that the training side, it's just like cloud itself, it's very agile. You can get hands-on quickly, you don't need to take a class, and then get in weeks later. You're in it like it's real time. So you're immersed in gamification and all kinds of ways to funnel into the either advanced tracks and certification. So you guys do a great job and I want to give you props for that and a shout out. The question I have for you guys is can you scope the opportunity for these certifications and opportunities for women in particular? What are some of the top jobs pulling down? Scope out the opportunity because I think when people hear that they really fall out of their chair, they go, "Wow, I didn't know I could make $200,000 doing cybersecurity." Well, yeah or maybe more. I just made the number, I don't actually know, but like I know people do make that much in cyber, but there are huge financial opportunities with certifications and education. Can you scope that order of magnitude? Can you share any data? >> Yeah, so in the US they certainly are. Certifications on average aligned to six digit type jobs. And if you go out and do a search, there are research studies out there that are refreshed every year that say what are the top IT industry certifications and how much money do they make? And the reason I don't put a number out there is because it's constantly changing and in fact it keeps going up, >> It's going up, not going down. >> But I would encourage people to do that quick search. What are the top IT industry certifications. Again, based on the country you're in, it makes a difference. But if you're US, there's a lot of data out there for the US and then there is some for other countries as well around how much on average people make. >> Do you list like the higher level certifications, stack rank them in terms of order? Like say, I'm a type A personnel, I want to climb Mount Everest, I want to get the highest level certification. How do I know that? Is it like laddered up or is like how do you guys present that? >> Yeah, so we have different types of certifications. There is a foundational, which we call the cloud practitioner. That one is more about just showing that you know something about cloud. It's not aligned to a specific job role. But then we have what we call associate level certifications, which are aligned to roles. So there's the solutions architect, cloud developer, so developer operations. And so you can tell by the role and associate is kind of that next level. And then the roles often have a professional level, which is even more advanced. And basically that's saying you're kind of an Uber expert at that point. And then there are technology specialties, which are less about a specific role, although some would argue a security technology specialty might align very well to a security role, but they're more about showing the technology. And so typically, it goes foundational, advanced, professional, and then the specialties are more on the side. They're not aligned, but they're deep. They're deep within that area. >> So you can go dig and pick your deep dive and jump into where you're comfortable. Heather, talk about the commitment in terms of dollars. I know Amazon's flaunted some numbers like 30 million or something, people they want to have trained, hundreds of millions of dollars in investment. This is key, obviously, more people trained on cloud, more operators, more cloud usage, obviously. I see the business connection. What's the women relationship to the numbers? Or what the experience is? How do you guys see that? Obviously International Women's Day, get the confidence, got the curiosity. You're a builder, you're in. It's that easy. >> It doesn't always feel that way, I'm sure to everybody, but we'd like to think that it is. Amazon and AWS do invest hundreds of millions of dollars in free training every year that is accessible to everyone out there. I think that sometimes the hardest obstacles to get overcome are getting started and we try and make it as easy as possible to get started with the tools that we've talked about already today. We run into plenty of cohorts of women as part of our re/Start program that are really grateful for the opportunity to see something, see a new way of thinking, see a new opportunity for them. We don't necessarily break out our funding by women versus men. We want to make sure that we are open and diverse for everybody to come in and get the training that they need to. But we definitely want to make sure that we are accessible and available to women and all genders outside of the US and inside the US. >> Well, I know the number's a lot lower than they should be and that's obviously why we're promoting this heavily. There's a lot more interest I see in tech. So digital transformation is gender neutral. I mean, it's like the world eats software and uses software, uses the cloud. So it has to get 50/50 in my opinion. So you guys do a great job. Now that we're done kind of promoting Amazon, which I wanted to do 'cause I think it's super important. Let's talk about you guys. What got you guys involved in tech? What was the inspiration and share some stories about your experiences and advice for folks watching? >> So I've always been in traditionally male dominated roles. I actually started in aviation and then moved to tech. And what I found was I got a mentor early on, a woman who was senior to me and who was kind of who I saw as the smartest person out there. She was incredibly smart, she was incredibly kind, and she was always lifting women up. And I kind of latched onto her and followed her around and she was such an amazing mentor. She brought me from throughout tech, from company to company, job to job, was always positioning me in front of other people as the go-to person. And I realized, "Wow, I want to be like her." And so that's been my focus as well in tech is you can be deeply technical in tech or you can be not deeply technical and be in tech and you can be successful both ways, but the way you're going to be most successful is if you find other people, build them up and help put them out in front. And so I personally love to mentor women and to put them in places where they can feel comfortable being out in front of people. And that's really been my career. I have tried to model her approach as much as I can. >> That's a really interesting observation. It's the pattern we've been seeing in all these interviews for the past two years of doing the International Women's Day is that networking, mentoring and sponsorship are one thing. So it's all one thing. It's not just mentoring. It's like people think, "Oh, just mentoring. What does that mean? Advice?" No, it's sponsorship, it's lifting people up, creating a keiretsu, creating networks. Really important. Heather, what's your experience? >> Yeah, I'm sort of the example of somebody who never thought they'd be in tech, but I happened to graduate from college in the Silicon Valley in the early nineties and next thing you know, it's more than a couple years later and I'm deeply in tech and I think it when we were having the conversation about confidence and willingness to learn and try that really spoke to me as well. I think I had to get out of my own way sometimes and just be willing to not be the smartest person in the room and just be willing to ask a lot of questions. And with every opportunity to ask questions, I think somebody, I ended up with good mentors, male and female, that saw the willingness to ask questions and the willingness to be humble in my approach to learning. And that really helped. I'm also very aware that nobody's journey is the same and I need to create an environment on my team and I need to be a role model within AWS and Amazon for allowing people to show up in the way that they're going to be most successful. And sometimes that will mean giving them learning opportunities. Sometimes that will be hooking them up with a mentor. Sometimes that will be giving them the freedom to do what they need for their family or their personal life. And modeling that behavior regardless of gender has always been how I choose to show up and what I ask my leaders to do. And the more we can do that, I've seen the team been able to grow and flourish in that way and support our entire team. >> I love that story. You also have a great leader, Maureen Lonergan, who I've met many conversations with, but also it starts at the top. Andy Jassy who can come across, he's kind of technical, he's dirty, he's a builder mentality. He has first principles and you're bringing up this first principles concept and whether that's passing it forward, what you've learned, having first principles helps in an organization. Can you guys talk about what that's like at your company? 'Cause everyone's different. And sometimes whether, and I sometimes I worry about what I say, but I also have my first principles. So talk about how principles matter in how you guys interface with others and letting people be their authentic self. >> Yeah, I'll jump in Jenni and then you can. The Amazon leadership principles are super important to how we interact with each other and it really does provide a set of guidelines for how we work with each other and how we work for our customers and with our partners. But most of all it gives us a common language and a common set of expectations. And I will be honest, they're not always easy. When you come from an environment that tends to be less open to feedback and less open to direct conversations than you find at Amazon, it could take a while to get used to that, but for me at least, it was extremely empowering to have those tools and those principles as guidance for how to operate and to gain the confidence in using them. I've also been able to participate in hundreds and hundreds of interviews in the time that I've been here as part of an interview team of bar raisers. I think that really helps us understand whether or not folks are going to be successful at AWS and at Amazon and helps them understand if they're going to be able to be successful. >> Bar raising is an Amazon term and it's gender neutral, right Jenni? >> It is gender neutral. >> Bar is a bar, it raises. >> That's right. And it's funny, we say that our culture here is peculiar. And when I started, I had been in consulting for several years, so I worked with a lot of different companies in tech and so I thought I'd seen everything and I came here and I went, "Hmm." I see what they mean by peculiar. It is very different environment. >> In the fullness of time, it'll all work out. >> That's right, that's right. Well and it's funny because when you first started, it's a lot to figure out to how to operate in an environment where people do use a 16 leadership principles. I've worked at a lot of companies with three or four core values and nobody can state those. We could state all 16 leadership principles and we use them in our regular everyday dialogue. That is an awkward thing when you first come to have people saying, "Oh, I'm going to use bias for action in this situation and I'm going to go move fast. And they're actually used in everyday conversations. But after a couple years suddenly you realize, "Oh, I'm doing that." And maybe even sometimes at the dinner table I'm doing that, which can get to be a bit much. But it creates an environment where we can all be different. We can all think differently. We can all have different ways of doing things, but we have a common overall approach to what we're trying to achieve. And that's really, it gives us a good framework for that. >> Jenni, it's great insight. Heather, thank you so much for sharing your stories. We're going to do this not once a year. We're going to continue this Women in Tech program every quarter. We'll check in with you guys and find out what's new. And thank you for what you do. We appreciate that getting the word out and really is an opportunity for everyone with education and cloud and it's only going to get more opportunities at the edge in AI and so much more tech. Thank you for coming on the program. >> Thank you for having us. >> Thanks, John. >> Thank you. That's the International Women's Day segment here with leaders from AWS. I'm John Furrier. Thanks for watching. (upbeat musiC)

(gentle music) >> Hello everyone. Welcome to this CUBE conversation here in Palo Alto, California. I'm John Furrier, host of theCUBE. We got a great remote guest coming in. Joseph Nelson, co-founder and CEO of RoboFlow hot startup in AI, computer vision. Really interesting topic in this wave of AI next gen hitting. Joseph, thanks for coming on this CUBE conversation. >> Thanks for having me. >> Yeah, I love the startup tsunami that's happening here in this wave. RoboFlow, you're in the middle of it. Exciting opportunities, you guys are in the cutting edge. I think computer vision's been talked about more as just as much as the large language models and these foundational models are merging. You're in the middle of it. What's it like right now as a startup and growing in this new wave hitting? >> It's kind of funny, it's, you know, I kind of describe it like sometimes you're in a garden of gnomes. It's like we feel like we've got this giant headstart with hundreds of thousands of people building with computer vision, training their own models, but that's a fraction of what it's going to be in six months, 12 months, 24 months. So, as you described it, a wave is a good way to think about it. And the wave is still building before it gets to its full size. So it's a ton of fun. >> Yeah, I think it's one of the most exciting areas in computer science. I wish I was in my twenties again, because I would be all over this. It's the intersection, there's so many disciplines, right? It's not just tech computer science, it's computer science, it's systems, it's software, it's data. There's so much aperture of things going on around your world. So, I mean, you got to be batting all the students away kind of trying to get hired in there, probably. I can only imagine you're hiring regiment. I'll ask that later, but first talk about what the company is that you're doing. How it's positioned, what's the market you're going after, and what's the origination story? How did you guys get here? How did you just say, hey, want to do this? What was the origination story? What do you do and how did you start the company? >> Yeah, yeah. I'll give you the what we do today and then I'll shift into the origin. RoboFlow builds tools for making the world programmable. Like anything that you see should be read write access if you think about it with a programmer's mind or legible. And computer vision is a technology that enables software to be added to these real world objects that we see. And so any sort of interface, any sort of object, any sort of scene, we can interact with it, we can make it more efficient, we can make it more entertaining by adding the ability for the tools that we use and the software that we write to understand those objects. And at RoboFlow, we've empowered a little over a hundred thousand developers, including those in half the Fortune 100 so far in that mission. Whether that's Walmart understanding the retail in their stores, Cardinal Health understanding the ways that they're helping their patients, or even electric vehicle manufacturers ensuring that they're making the right stuff at the right time. As you mentioned, it's early. Like I think maybe computer vision has touched one, maybe 2% of the whole economy and it'll be like everything in a very short period of time. And so we're focused on enabling that transformation. I think it's it, as far as I think about it, I've been fortunate to start companies before, start, sell these sorts of things. This is the last company I ever wanted to start and I think it will be, should we do it right, the world's largest in riding the wave of bringing together the disparate pieces of that technology. >> What was the motivating point of the formation? Was it, you know, you guys were hanging around? Was there some catalyst? What was the moment where it all kind of came together for you? >> You know what's funny is my co-founder, Brad and I, we were making computer vision apps for making board games more fun to play. So in 2017, Apple released AR kit, augmented reality kit for building augmented reality applications. And Brad and I are both sort of like hacker persona types. We feel like we don't really understand the technology until we build something with it and so we decided that we should make an app that if you point your phone at a Sudoku puzzle, it understands the state of the board and then it kind of magically fills in that experience with all the digits in real time, which totally ruins the game of Sudoku to be clear. But it also just creates this like aha moment of like, oh wow, like the ability for our pocket devices to understand and see the world as good or better than we can is possible. And so, you know, we actually did that as I mentioned in 2017, and the app went viral. It was, you know, top of some subreddits, top of Injure, Reddit, the hacker community as well as Product Hunt really liked it. So it actually won Product Hunt AR app of the year, which was the same year that the Tesla model three won the product of the year. So we joked that we share an award with Elon our shared (indistinct) But frankly, so that was 2017. RoboFlow wasn't incorporated as a business until 2019. And so, you know, when we made Magic Sudoku, I was running a different company at the time, Brad was running a different company at the time, and we kind of just put it out there and were excited by how many people liked it. And we assumed that other curious developers would see this inevitable future of, oh wow, you know. This is much more than just a pedestrian point your phone at a board game. This is everything can be seen and understood and rewritten in a different way. Things like, you know, maybe your fridge. Knowing what ingredients you have and suggesting recipes or auto ordering for you, or we were talking about some retail use cases of automated checkout. Like anything can be seen and observed and we presume that that would kick off a Cambrian explosion of applications. It didn't. So you fast forward to 2019, we said, well we might as well be the guys to start to tackle this sort of problem. And because of our success with board games before, we returned to making more board game solving applications. So we made one that solves Boggle, you know, the four by four word game, we made one that solves chess, you point your phone at a chess board and it understands the state of the board and then can make move recommendations. And each additional board game that we added, we realized that the tooling was really immature. The process of collecting images, knowing which images are actually going to be useful for improving model performance, training those models, deploying those models. And if we really wanted to make the world programmable, developers waiting for us to make an app for their thing of interest is a lot less efficient, less impactful than taking our tool chain and releasing that externally. And so, that's what RoboFlow became. RoboFlow became the internal tools that we used to make these game changing applications readily available. And as you know, when you give developers new tools, they create new billion dollar industries, let alone all sorts of fun hobbyist projects along the way. >> I love that story. Curious, inventive, little radical. Let's break the rules, see how we can push the envelope on the board games. That's how companies get started. It's a great story. I got to ask you, okay, what happens next? Now, okay, you realize this new tooling, but this is like how companies get built. Like they solve their own problem that they had 'cause they realized there's one, but then there has to be a market for it. So you actually guys knew that this was coming around the corner. So okay, you got your hacker mentality, you did that thing, you got the award and now you're like, okay, wow. Were you guys conscious of the wave coming? Was it one of those things where you said, look, if we do this, we solve our own problem, this will be big for everybody. Did you have that moment? Was that in 2019 or was that more of like, it kind of was obvious to you guys? >> Absolutely. I mean Brad puts this pretty effectively where he describes how we lived through the initial internet revolution, but we were kind of too young to really recognize and comprehend what was happening at the time. And then mobile happened and we were working on different companies that were not in the mobile space. And computer vision feels like the wave that we've caught. Like, this is a technology and capability that rewrites how we interact with the world, how everyone will interact with the world. And so we feel we've been kind of lucky this time, right place, right time of every enterprise will have the ability to improve their operations with computer vision. And so we've been very cognizant of the fact that computer vision is one of those groundbreaking technologies that every company will have as a part of their products and services and offerings, and we can provide the tooling to accelerate that future. >> Yeah, and the developer angle, by the way, I love that because I think, you know, as we've been saying in theCUBE all the time, developer's the new defacto standard bodies because what they adopt is pure, you know, meritocracy. And they pick the best. If it's sell service and it's good and it's got open source community around it, its all in. And they'll vote. They'll vote with their code and that is clear. Now I got to ask you, as you look at the market, we were just having this conversation on theCUBE in Barcelona at recent Mobile World Congress, now called MWC, around 5G versus wifi. And the debate was specifically computer vision, like facial recognition. We were talking about how the Cleveland Browns were using facial recognition for people coming into the stadium they were using it for ships in international ports. So the question was 5G versus wifi. My question is what infrastructure or what are the areas that need to be in place to make computer vision work? If you have developers building apps, apps got to run on stuff. So how do you sort that out in your mind? What's your reaction to that? >> A lot of the times when we see applications that need to run in real time and on video, they'll actually run at the edge without internet. And so a lot of our users will actually take their models and run it in a fully offline environment. Now to act on that information, you'll often need to have internet signal at some point 'cause you'll need to know how many people were in the stadium or what shipping crates are in my port at this point in time. You'll need to relay that information somewhere else, which will require connectivity. But actually using the model and creating the insights at the edge does not require internet. I mean we have users that deploy models on underwater submarines just as much as in outer space actually. And those are not very friendly environments to internet, let alone 5g. And so what you do is you use an edge device, like an Nvidia Jetson is common, mobile devices are common. Intel has some strong edge devices, the Movidius family of chips for example. And you use that compute that runs completely offline in real time to process those signals. Now again, what you do with those signals may require connectivity and that becomes a question of the problem you're solving of how soon you need to relay that information to another place. >> So, that's an architectural issue on the infrastructure. If you're a tactical edge war fighter for instance, you might want to have highly available and maybe high availability. I mean, these are words that mean something. You got storage, but it's not at the edge in real time. But you can trickle it back and pull it down. That's management. So that's more of a business by business decision or environment, right? >> That's right, that's right. Yeah. So I mean we can talk through some specifics. So for example, the RoboFlow actually powers the broadcaster that does the tennis ball tracking at Wimbledon. That runs completely at the edge in real time in, you know, technically to track the tennis ball and point the camera, you actually don't need internet. Now they do have internet of course to do the broadcasting and relay the signal and feeds and these sorts of things. And so that's a case where you have both edge deployment of running the model and high availability act on that model. We have other instances where customers will run their models on drones and the drone will go and do a flight and it'll say, you know, this many residential homes are in this given area, or this many cargo containers are in this given shipping yard. Or maybe we saw these environmental considerations of soil erosion along this riverbank. The model in that case can run on the drone during flight without internet, but then you only need internet once the drone lands and you're going to act on that information because for example, if you're doing like a study of soil erosion, you don't need to be real time. You just need to be able to process and make use of that information once the drone finishes its flight. >> Well I can imagine a zillion use cases. I heard of a use case interview at a company that does computer vision to help people see if anyone's jumping the fence on their company. Like, they know what a body looks like climbing a fence and they can spot it. Pretty easy use case compared to probably some of the other things, but this is the horizontal use cases, its so many use cases. So how do you guys talk to the marketplace when you say, hey, we have generative AI for commuter vision. You might know language models that's completely different animal because vision's like the world, right? So you got a lot more to do. What's the difference? How do you explain that to customers? What can I build and what's their reaction? >> Because we're such a developer centric company, developers are usually creative and show you the ways that they want to take advantage of new technologies. I mean, we've had people use things for identifying conveyor belt debris, doing gas leak detection, measuring the size of fish, airplane maintenance. We even had someone that like a hobby use case where they did like a specific sushi identifier. I dunno if you know this, but there's a specific type of whitefish that if you grew up in the western hemisphere and you eat it in the eastern hemisphere, you get very sick. And so there was someone that made an app that tells you if you happen to have that fish in the sushi that you're eating. But security camera analysis, transportation flows, plant disease detection, really, you know, smarter cities. We have people that are doing curb management identifying, and a lot of these use cases, the fantastic thing about building tools for developers is they're a creative bunch and they have these ideas that if you and I sat down for 15 minutes and said, let's guess every way computer vision can be used, we would need weeks to list all the example use cases. >> We'd miss everything. >> And we'd miss. And so having the community show us the ways that they're using computer vision is impactful. Now that said, there are of course commercial industries that have discovered the value and been able to be out of the gate. And that's where we have the Fortune 100 customers, like we do. Like the retail customers in the Walmart sector, healthcare providers like Medtronic, or vehicle manufacturers like Rivian who all have very difficult either supply chain, quality assurance, in stock, out of stock, anti-theft protection considerations that require successfully making sense of the real world. >> Let me ask you a question. This is maybe a little bit in the weeds, but it's more developer focused. What are some of the developer profiles that you're seeing right now in terms of low-hanging fruit applications? And can you talk about the academic impact? Because I imagine if I was in school right now, I'd be all over it. Are you seeing Master's thesis' being worked on with some of your stuff? Is the uptake in both areas of younger pre-graduates? And then inside the workforce, What are some of the devs like? Can you share just either what their makeup is, what they work on, give a little insight into the devs you're working with. >> Leading developers that want to be on state-of-the-art technology build with RoboFlow because they know they can use the best in class open source. They know that they can get the most out of their data. They know that they can deploy extremely quickly. That's true among students as you mentioned, just as much as as industries. So we welcome students and I mean, we have research grants that will regularly support for people to publish. I mean we actually have a channel inside our internal slack where every day, more student publications that cite building with RoboFlow pop up. And so, that helps inspire some of the use cases. Now what's interesting is that the use case is relatively, you know, useful or applicable for the business or the student. In other words, if a student does a thesis on how to do, we'll say like shingle damage detection from satellite imagery and they're just doing that as a master's thesis, in fact most insurance businesses would be interested in that sort of application. So, that's kind of how we see uptick and adoption both among researchers who want to be on the cutting edge and publish, both with RoboFlow and making use of open source tools in tandem with the tool that we provide, just as much as industry. And you know, I'm a big believer in the philosophy that kind of like what the hackers are doing nights and weekends, the Fortune 500 are doing in a pretty short order period of time and we're experiencing that transition. Computer vision used to be, you know, kind of like a PhD, multi-year investment endeavor. And now with some of the tooling that we're working on in open source technologies and the compute that's available, these science fiction ideas are possible in an afternoon. And so you have this idea of maybe doing asset management or the aerial observation of your shingles or things like this. You have a few hundred images and you can de-risk whether that's possible for your business today. So there's pretty broad-based adoption among both researchers that want to be on the state of the art, as much as companies that want to reduce the time to value. >> You know, Joseph, you guys and your partner have got a great front row seat, ground floor, presented creation wave here. I'm seeing a pattern emerging from all my conversations on theCUBE with founders that are successful, like yourselves, that there's two kind of real things going on. You got the enterprises grabbing the products and retrofitting into their legacy and rebuilding their business. And then you have startups coming out of the woodwork. Young, seeing greenfield or pick a specific niche or focus and making that the signature lever to move the market. >> That's right. >> So can you share your thoughts on the startup scene, other founders out there and talk about that? And then I have a couple questions for like the enterprises, the old school, the existing legacy. Little slower, but the startups are moving fast. What are some of the things you're seeing as startups are emerging in this field? >> I think you make a great point that independent of RoboFlow, very successful, especially developer focused businesses, kind of have three customer types. You have the startups and maybe like series A, series B startups that you're building a product as fast as you can to keep up with them, and they're really moving just as fast as as you are and pulling the product out at you for things that they need. The second segment that you have might be, call it SMB but not enterprise, who are able to purchase and aren't, you know, as fast of moving, but are stable and getting value and able to get to production. And then the third type is enterprise, and that's where you have typically larger contract value sizes, slower moving in terms of adoption and feedback for your product. And I think what you see is that successful companies balance having those three customer personas because you have the small startups, small fast moving upstarts that are discerning buyers who know the market and elect to build on tooling that is best in class. And so you basically kind of pass the smell test of companies who are quite discerning in their purchases, plus are moving so quick they're pulling their product out of you. Concurrently, you have a product that's enterprise ready to service the scalability, availability, and trust of enterprise buyers. And that's ultimately where a lot of companies will see tremendous commercial success. I mean I remember seeing the Twilio IPO, Uber being like a full 20% of their revenue, right? And so there's this very common pattern where you have the ability to find some of those upstarts that you make bets on, like the next Ubers of the world, the smaller companies that continue to get developed with the product and then the enterprise whom allows you to really fund the commercial success of the business, and validate the size of the opportunity in market that's being creative. >> It's interesting, there's so many things happening there. It's like, in a way it's a new category, but it's not a new category. It becomes a new category because of the capabilities, right? So, it's really interesting, 'cause that's what you're talking about is a category, creating. >> I think developer tools. So people often talk about B to B and B to C businesses. I think developer tools are in some ways a third way. I mean ultimately they're B to B, you're selling to other businesses and that's where your revenue's coming from. However, you look kind of like a B to C company in the ways that you measure product adoption and kind of go to market. In other words, you know, we're often tracking the leading indicators of commercial success in the form of usage, adoption, retention. Really consumer app, traditionally based metrics of how to know you're building the right stuff, and that's what product led growth companies do. And then you ultimately have commercial traction in a B to B way. And I think that that actually kind of looks like a third thing, right? Like you can do these sort of funny zany marketing examples that you might see historically from consumer businesses, but yet you ultimately make your money from the enterprise who has these de-risked high value problems you can solve for them. And I selfishly think that that's the best of both worlds because I don't have to be like Evan Spiegel, guessing the next consumer trend or maybe creating the next consumer trend and catching lightning in a bottle over and over again on the consumer side. But I still get to have fun in our marketing and make sort of fun, like we're launching the world's largest game of rock paper scissors being played with computer vision, right? Like that's sort of like a fun thing you can do, but then you can concurrently have the commercial validation and customers telling you the things that they need to be built for them next to solve commercial pain points for them. So I really do think that you're right by calling this a new category and it really is the best of both worlds. >> It's a great call out, it's a great call out. In fact, I always juggle with the VC. I'm like, it's so easy. Your job is so easy to pick the winners. What are you talking about its so easy? I go, just watch what the developers jump on. And it's not about who started, it could be someone in the dorm room to the boardroom person. You don't know because that B to C, the C, it's B to D you know? You know it's developer 'cause that's a human right? That's a consumer of the tool which influences the business that never was there before. So I think this direct business model evolution, whether it's media going direct or going direct to the developers rather than going to a gatekeeper, this is the reality. >> That's right. >> Well I got to ask you while we got some time left to describe, I want to get into this topic of multi-modality, okay? And can you describe what that means in computer vision? And what's the state of the growth of that portion of this piece? >> Multi modality refers to using multiple traditionally siloed problem types, meaning text, image, video, audio. So you could treat an audio problem as only processing audio signal. That is not multimodal, but you could use the audio signal at the same time as a video feed. Now you're talking about multi modality. In computer vision, multi modality is predominantly happening with images and text. And one of the biggest releases in this space is actually two years old now, was clip, contrastive language image pre-training, which took 400 million image text pairs and basically instead of previously when you do classification, you basically map every single image to a single class, right? Like here's a bunch of images of chairs, here's a bunch of images of dogs. What clip did is used, you can think about it like, the class for an image being the Instagram caption for the image. So it's not one single thing. And by training on understanding the corpora, you basically see which words, which concepts are associated with which pixels. And this opens up the aperture for the types of problems and generalizability of models. So what does this mean? This means that you can get to value more quickly from an existing trained model, or at least validate that what you want to tackle with a computer vision, you can get there more quickly. It also opens up the, I mean. Clip has been the bedrock of some of the generative image techniques that have come to bear, just as much as some of the LLMs. And increasingly we're going to see more and more of multi modality being a theme simply because at its core, you're including more context into what you're trying to understand about the world. I mean, in its most basic sense, you could ask yourself, if I have an image, can I know more about that image with just the pixels? Or if I have the image and the sound of when that image was captured or it had someone describe what they see in that image when the image was captured, which one's going to be able to get you more signal? And so multi modality helps expand the ability for us to understand signal processing. >> Awesome. And can you just real quick, define clip for the folks that don't know what that means? >> Yeah. Clip is a model architecture, it's an acronym for contrastive language image pre-training and like, you know, model architectures that have come before it captures the almost like, models are kind of like brands. So I guess it's a brand of a model where you've done these 400 million image text pairs to match up which visual concepts are associated with which text concepts. And there have been new releases of clip, just at bigger sizes of bigger encoding's, of longer strings of texture, or larger image windows. But it's been a really exciting advancement that OpenAI released in January, 2021. >> All right, well great stuff. We got a couple minutes left. Just I want to get into more of a company-specific question around culture. All startups have, you know, some sort of cultural vibe. You know, Intel has Moore's law doubles every whatever, six months. What's your culture like at RoboFlow? I mean, if you had to describe that culture, obviously love the hacking story, you and your partner with the games going number one on Product Hunt next to Elon and Tesla and then hey, we should start a company two years later. That's kind of like a curious, inventing, building, hard charging, but laid back. That's my take. How would you describe the culture? >> I think that you're right. The culture that we have is one of shipping, making things. So every week each team shares what they did for our customers on a weekly basis. And we have such a strong emphasis on being better week over week that those sorts of things compound. So one big emphasis in our culture is getting things done, shipping, doing things for our customers. The second is we're an incredibly transparent place to work. For example, how we think about giving decisions, where we're progressing against our goals, what problems are biggest and most important for the company is all open information for those that are inside the company to know and progress against. The third thing that I'd use to describe our culture is one that thrives with autonomy. So RoboFlow has a number of individuals who have founded companies before, some of which have sold their businesses for a hundred million plus upon exit. And the way that we've been able to attract talent like that is because the problems that we're tackling are so immense, yet individuals are able to charge at it with the way that they think is best. And this is what pairs well with transparency. If you have a strong sense of what the company's goals are, how we're progressing against it, and you have this ownership mentality of what can I do to change or drive progress against that given outcome, then you create a really healthy pairing of, okay cool, here's where the company's progressing. Here's where things are going really well, here's the places that we most need to improve and work on. And if you're inside that company as someone who has a preponderance to be a self-starter and even a history of building entire functions or companies yourself, then you're going to be a place where you can really thrive. You have the inputs of the things where we need to work on to progress the company's goals. And you have the background of someone that is just necessarily a fast moving and ambitious type of individual. So I think the best way to describe it is a transparent place with autonomy and an emphasis on getting things done. >> Getting shit done as they say. Getting stuff done. Great stuff. Hey, final question. Put a plug out there for the company. What are you going to hire? What's your pipeline look like for people? What jobs are open? I'm sure you got hiring all around. Give a quick plug for the company what you're looking for. >> I appreciate you asking. Basically you're either building the product or helping customers be successful with the product. So in the building product category, we have platform engineering roles, machine learning engineering roles, and we're solving some of the hardest and most impactful problems of bringing such a groundbreaking technology to the masses. And so it's a great place to be where you can kind of be your own user as an engineer. And then if you're enabling people to be successful with the products, I mean you're working in a place where there's already such a strong community around it and you can help shape, foster, cultivate, activate, and drive commercial success in that community. So those are roles that tend themselves to being those that build the product for developer advocacy, those that are account executives that are enabling our customers to realize commercial success, and even hybrid roles like we call it field engineering, where you are a technical resource to drive success within customer accounts. And so all this is listed on roboflow.com/careers. And one thing that I actually kind of want to mention John that's kind of novel about the thing that's working at RoboFlow. So there's been a lot of discussion around remote companies and there's been a lot of discussion around in-person companies and do you need to be in the office? And one thing that we've kind of recognized is you can actually chart a third way. You can create a third way which we call satellite, which basically means people can work from where they most like to work and there's clusters of people, regular onsite's. And at RoboFlow everyone gets, for example, $2,500 a year that they can use to spend on visiting coworkers. And so what's sort of organically happened is team numbers have started to pull together these resources and rent out like, lavish Airbnbs for like a week and then everyone kind of like descends in and works together for a week and makes and creates things. And we call this lighthouses because you know, a lighthouse kind of brings ships into harbor and we have an emphasis on shipping. >> Yeah, quality people that are creative and doers and builders. You give 'em some cash and let the self-governing begin, you know? And like, creativity goes through the roof. It's a great story. I think that sums up the culture right there, Joseph. Thanks for sharing that and thanks for this great conversation. I really appreciate it and it's very inspiring. Thanks for coming on. >> Yeah, thanks for having me, John. >> Joseph Nelson, co-founder and CEO of RoboFlow. Hot company, great culture in the right place in a hot area, computer vision. This is going to explode in value. The edge is exploding. More use cases, more development, and developers are driving the change. Check out RoboFlow. This is theCUBE. I'm John Furrier, your host. Thanks for watching. (gentle music)