(mellow music) >> Hello everyone, welcome to theCUBE's coverage of AWS Startup Showcase. This is the closing panel session on AI machine learning, the top startups generating generative AI on AWS. It's a great panel. This is going to be the experts talking about riding the wave in generative AI. We got Ankur Mehrotra, who's the director and general manager of AI and machine learning at AWS, and Clem Delangue, co-founder and CEO of Hugging Face, and Ori Goshen, who's the co-founder and CEO of AI21 Labs. Ori from Tel Aviv dialing in, and rest coming in here on theCUBE. Appreciate you coming on for this closing session for the Startup Showcase. >> Thanks for having us. >> Thank you for having us. >> Thank you. >> I'm super excited to have you all on. Hugging Face was recently in the news with the AWS relationship, so congratulations. Open source, open science, really driving the machine learning. And we got the AI21 Labs access to the LLMs, generating huge scale live applications, commercial applications, coming to the market, all powered by AWS. So everyone, congratulations on all your success, and thank you for headlining this panel. Let's get right into it. AWS is powering this wave here. We're seeing a lot of push here from applications. Ankur, set the table for us on the AI machine learning. It's not new, it's been goin' on for a while. Past three years have been significant advancements, but there's been a lot of work done in AI machine learning. Now it's released to the public. Everybody's super excited and now says, "Oh, the future's here!" It's kind of been going on for a while and baking. Now it's kind of coming out. What's your view here? Let's get it started. >> Yes, thank you. So, yeah, as you may be aware, Amazon has been in investing in machine learning research and development since quite some time now. And we've used machine learning to innovate and improve user experiences across different Amazon products, whether it's Alexa or Amazon.com. But we've also brought in our expertise to extend what we are doing in the space and add more generative AI technology to our AWS products and services, starting with CodeWhisperer, which is an AWS service that we announced a few months ago, which is, you can think of it as a coding companion as a service, which uses generative AI models underneath. And so this is a service that customers who have no machine learning expertise can just use. And we also are talking to customers, and we see a lot of excitement about generative AI, and customers who want to build these models themselves, who have the talent and the expertise and resources. For them, AWS has a number of different options and capabilities they can leverage, such as our custom silicon, such as Trainium and Inferentia, as well as distributed machine learning capabilities that we offer as part of SageMaker, which is an end-to-end machine learning development service. At the same time, many of our customers tell us that they're interested in not training and building these generative AI models from scratch, given they can be expensive and can require specialized talent and skills to build. And so for those customers, we are also making it super easy to bring in existing generative AI models into their machine learning development environment within SageMaker for them to use. So we recently announced our partnership with Hugging Face, where we are making it super easy for customers to bring in those models into their SageMaker development environment for fine tuning and deployment. And then we are also partnering with other proprietary model providers such as AI21 and others, where we making these generative AI models available within SageMaker for our customers to use. So our approach here is to really provide customers options and choices and help them accelerate their generative AI journey. >> Ankur, thank you for setting the table there. Clem and Ori, I want to get your take, because the riding the waves, the theme of this session, and to me being in California, I imagine the big surf, the big waves, the big talent out there. This is like alpha geeks, alpha coders, developers are really leaning into this. You're seeing massive uptake from the smartest people. Whether they're young or around, they're coming in with their kind of surfboards, (chuckles) if you will. These early adopters, they've been on this for a while; Now the waves are hitting. This is a big wave, everyone sees it. What are some of those early adopter devs doing? What are some of the use cases you're seeing right out of the gate? And what does this mean for the folks that are going to come in and get on this wave? Can you guys share your perspective on this? Because you're seeing the best talent now leaning into this. >> Yeah, absolutely. I mean, from Hugging Face vantage points, it's not even a a wave, it's a tidal wave, or maybe even the tide itself. Because actually what we are seeing is that AI and machine learning is not something that you add to your products. It's very much a new paradigm to do all technology. It's this idea that we had in the past 15, 20 years, one way to build software and to build technology, which was writing a million lines of code, very rule-based, and then you get your product. Now what we are seeing is that every single product, every single feature, every single company is starting to adopt AI to build the next generation of technology. And that works both to make the existing use cases better, if you think of search, if you think of social network, if you think of SaaS, but also it's creating completely new capabilities that weren't possible with the previous paradigm. Now AI can generate text, it can generate image, it can describe your image, it can do so many new things that weren't possible before. >> It's going to really make the developers really productive, right? I mean, you're seeing the developer uptake strong, right? >> Yes, we have over 15,000 companies using Hugging Face now, and it keeps accelerating. I really think that maybe in like three, five years, there's not going to be any company not using AI. It's going to be really kind of the default to build all technology. >> Ori, weigh in on this. APIs, the cloud. Now I'm a developer, I want to have live applications, I want the commercial applications on this. What's your take? Weigh in here. >> Yeah, first, I absolutely agree. I mean, we're in the midst of a technology shift here. I think not a lot of people realize how big this is going to be. Just the number of possibilities is endless, and I think hard to imagine. And I don't think it's just the use cases. I think we can think of it as two separate categories. We'll see companies and products enhancing their offerings with these new AI capabilities, but we'll also see new companies that are AI first, that kind of reimagine certain experiences. They build something that wasn't possible before. And that's why I think it's actually extremely exciting times. And maybe more philosophically, I think now these large language models and large transformer based models are helping us people to express our thoughts and kind of making the bridge from our thinking to a creative digital asset in a speed we've never imagined before. I can write something down and get a piece of text, or an image, or a code. So I'll start by saying it's hard to imagine all the possibilities right now, but it's certainly big. And if I had to bet, I would say it's probably at least as big as the mobile revolution we've seen in the last 20 years. >> Yeah, this is the biggest. I mean, it's been compared to the Enlightenment Age. I saw the Wall Street Journal had a recent story on this. We've been saying that this is probably going to be bigger than all inflection points combined in the tech industry, given what transformation is coming. I guess I want to ask you guys, on the early adopters, we've been hearing on these interviews and throughout the industry that there's already a set of big companies, a set of companies out there that have a lot of data and they're already there, they're kind of tinkering. Kind of reminds me of the old hyper scaler days where they were building their own scale, and they're eatin' glass, spittin' nails out, you know, they're hardcore. Then you got everybody else kind of saying board level, "Hey team, how do I leverage this?" How do you see those two things coming together? You got the fast followers coming in behind the early adopters. What's it like for the second wave coming in? What are those conversations for those developers like? >> I mean, I think for me, the important switch for companies is to change their mindset from being kind of like a traditional software company to being an AI or machine learning company. And that means investing, hiring machine learning engineers, machine learning scientists, infrastructure in members who are working on how to put these models in production, team members who are able to optimize models, specialized models, customized models for the company's specific use cases. So it's really changing this mindset of how you build technology and optimize your company building around that. Things are moving so fast that I think now it's kind of like too late for low hanging fruits or small, small adjustments. I think it's important to realize that if you want to be good at that, and if you really want to surf this wave, you need massive investments. If there are like some surfers listening with this analogy of the wave, right, when there are waves, it's not enough just to stand and make a little bit of adjustments. You need to position yourself aggressively, paddle like crazy, and that's how you get into the waves. So that's what companies, in my opinion, need to do right now. >> Ori, what's your take on the generative models out there? We hear a lot about foundation models. What's your experience running end-to-end applications for large foundation models? Any insights you can share with the app developers out there who are looking to get in? >> Yeah, I think first of all, it's start create an economy, where it probably doesn't make sense for every company to create their own foundation models. You can basically start by using an existing foundation model, either open source or a proprietary one, and start deploying it for your needs. And then comes the second round when you are starting the optimization process. You bootstrap, whether it's a demo, or a small feature, or introducing new capability within your product, and then start collecting data. That data, and particularly the human feedback data, helps you to constantly improve the model, so you create this data flywheel. And I think we're now entering an era where customers have a lot of different choice of how they want to start their generative AI endeavor. And it's a good thing that there's a variety of choices. And the really amazing thing here is that every industry, any company you speak with, it could be something very traditional like industrial or financial, medical, really any company. I think peoples now start to imagine what are the possibilities, and seriously think what's their strategy for adopting this generative AI technology. And I think in that sense, the foundation model actually enabled this to become scalable. So the barrier to entry became lower; Now the adoption could actually accelerate. >> There's a lot of integration aspects here in this new wave that's a little bit different. Before it was like very monolithic, hardcore, very brittle. A lot more integration, you see a lot more data coming together. I have to ask you guys, as developers come in and grow, I mean, when I went to college and you were a software engineer, I mean, I got a degree in computer science, and software engineering, that's all you did was code, (chuckles) you coded. Now, isn't it like everyone's a machine learning engineer at this point? Because that will be ultimately the science. So, (chuckles) you got open source, you got open software, you got the communities. Swami called you guys the GitHub of machine learning, Hugging Face is the GitHub of machine learning, mainly because that's where people are going to code. So this is essentially, machine learning is computer science. What's your reaction to that? >> Yes, my co-founder Julien at Hugging Face have been having this thing for quite a while now, for over three years, which was saying that actually software engineering as we know it today is a subset of machine learning, instead of the other way around. People would call us crazy a few years ago when we're seeing that. But now we are realizing that you can actually code with machine learning. So machine learning is generating code. And we are starting to see that every software engineer can leverage machine learning through open models, through APIs, through different technology stack. So yeah, it's not crazy anymore to think that maybe in a few years, there's going to be more people doing AI and machine learning. However you call it, right? Maybe you'll still call them software engineers, maybe you'll call them machine learning engineers. But there might be more of these people in a couple of years than there is software engineers today. >> I bring this up as more tongue in cheek as well, because Ankur, infrastructure's co is what made Cloud great, right? That's kind of the DevOps movement. But here the shift is so massive, there will be a game-changing philosophy around coding. Machine learning as code, you're starting to see CodeWhisperer, you guys have had coding companions for a while on AWS. So this is a paradigm shift. How is the cloud playing into this for you guys? Because to me, I've been riffing on some interviews where it's like, okay, you got the cloud going next level. This is an example of that, where there is a DevOps-like moment happening with machine learning, whether you call it coding or whatever. It's writing code on its own. Can you guys comment on what this means on top of the cloud? What comes out of the scale? What comes out of the benefit here? >> Absolutely, so- >> Well first- >> Oh, go ahead. >> Yeah, so I think as far as scale is concerned, I think customers are really relying on cloud to make sure that the applications that they build can scale along with the needs of their business. But there's another aspect to it, which is that until a few years ago, John, what we saw was that machine learning was a data scientist heavy activity. They were data scientists who were taking the data and training models. And then as machine learning found its way more and more into production and actual usage, we saw the MLOps become a thing, and MLOps engineers become more involved into the process. And then we now are seeing, as machine learning is being used to solve more business critical problems, we're seeing even legal and compliance teams get involved. We are seeing business stakeholders more engaged. So, more and more machine learning is becoming an activity that's not just performed by data scientists, but is performed by a team and a group of people with different skills. And for them, we as AWS are focused on providing the best tools and services for these different personas to be able to do their job and really complete that end-to-end machine learning story. So that's where, whether it's tools related to MLOps or even for folks who cannot code or don't know any machine learning. For example, we launched SageMaker Canvas as a tool last year, which is a UI-based tool which data analysts and business analysts can use to build machine learning models. So overall, the spectrum in terms of persona and who can get involved in the machine learning process is expanding, and the cloud is playing a big role in that process. >> Ori, Clem, can you guys weigh in too? 'Cause this is just another abstraction layer of scale. What's it mean for you guys as you look forward to your customers and the use cases that you're enabling? >> Yes, I think what's important is that the AI companies and providers and the cloud kind of work together. That's how you make a seamless experience and you actually reduce the barrier to entry for this technology. So that's what we've been super happy to do with AWS for the past few years. We actually announced not too long ago that we are doubling down on our partnership with AWS. We're excited to have many, many customers on our shared product, the Hugging Face deep learning container on SageMaker. And we are working really closely with the Inferentia team and the Trainium team to release some more exciting stuff in the coming weeks and coming months. So I think when you have an ecosystem and a system where the AWS and the AI providers, AI startups can work hand in hand, it's to the benefit of the customers and the companies, because it makes it orders of magnitude easier for them to adopt this new paradigm to build technology AI. >> Ori, this is a scale on reasoning too. The data's out there and making sense out of it, making it reason, getting comprehension, having it make decisions is next, isn't it? And you need scale for that. >> Yes. Just a comment about the infrastructure side. So I think really the purpose is to streamline and make these technologies much more accessible. And I think we'll see, I predict that we'll see in the next few years more and more tooling that make this technology much more simple to consume. And I think it plays a very important role. There's so many aspects, like the monitoring the models and their kind of outputs they produce, and kind of containing and running them in a production environment. There's so much there to build on, the infrastructure side will play a very significant role. >> All right, that's awesome stuff. I'd love to change gears a little bit and get a little philosophy here around AI and how it's going to transform, if you guys don't mind. There's been a lot of conversations around, on theCUBE here as well as in some industry areas, where it's like, okay, all the heavy lifting is automated away with machine learning and AI, the complexity, there's some efficiencies, it's horizontal and scalable across all industries. Ankur, good point there. Everyone's going to use it for something. And a lot of stuff gets brought to the table with large language models and other things. But the key ingredient will be proprietary data or human input, or some sort of AI whisperer kind of role, or prompt engineering, people are saying. So with that being said, some are saying it's automating intelligence. And that creativity will be unleashed from this. If the heavy lifting goes away and AI can fill the void, that shifts the value to the intellect or the input. And so that means data's got to come together, interact, fuse, and understand each other. This is kind of new. I mean, old school AI was, okay, got a big model, I provisioned it long time, very expensive. Now it's all free flowing. Can you guys comment on where you see this going with this freeform, data flowing everywhere, heavy lifting, and then specialization? >> Yeah, I think- >> Go ahead. >> Yeah, I think, so what we are seeing with these large language models or generative models is that they're really good at creating stuff. But I think it's also important to recognize their limitations. They're not as good at reasoning and logic. And I think now we're seeing great enthusiasm, I think, which is justified. And the next phase would be how to make these systems more reliable. How to inject more reasoning capabilities into these models, or augment with other mechanisms that actually perform more reasoning so we can achieve more reliable results. And we can count on these models to perform for critical tasks, whether it's medical tasks, legal tasks. We really want to kind of offload a lot of the intelligence to these systems. And then we'll have to get back, we'll have to make sure these are reliable, we'll have to make sure we get some sort of explainability that we can understand the process behind the generated results that we received. So I think this is kind of the next phase of systems that are based on these generated models. >> Clem, what's your view on this? Obviously you're at open community, open source has been around, it's been a great track record, proven model. I'm assuming creativity's going to come out of the woodwork, and if we can automate open source contribution, and relationships, and onboarding more developers, there's going to be unleashing of creativity. >> Yes, it's been so exciting on the open source front. We all know Bert, Bloom, GPT-J, T5, Stable Diffusion, that work up. The previous or the current generation of open source models that are on Hugging Face. It has been accelerating in the past few months. So I'm super excited about ControlNet right now that is really having a lot of impact, which is kind of like a way to control the generation of images. Super excited about Flan UL2, which is like a new model that has been recently released and is open source. So yeah, it's really fun to see the ecosystem coming together. Open source has been the basis for traditional software, with like open source programming languages, of course, but also all the great open source that we've gotten over the years. So we're happy to see that the same thing is happening for machine learning and AI, and hopefully can help a lot of companies reduce a little bit the barrier to entry. So yeah, it's going to be exciting to see how it evolves in the next few years in that respect. >> I think the developer productivity angle that's been talked about a lot in the industry will be accelerated significantly. I think security will be enhanced by this. I think in general, applications are going to transform at a radical rate, accelerated, incredible rate. So I think it's not a big wave, it's the water, right? I mean, (chuckles) it's the new thing. My final question for you guys, if you don't mind, I'd love to get each of you to answer the question I'm going to ask you, which is, a lot of conversations around data. Data infrastructure's obviously involved in this. And the common thread that I'm hearing is that every company that looks at this is asking themselves, if we don't rebuild our company, start thinking about rebuilding our business model around AI, we might be dinosaurs, we might be extinct. And it reminds me that scene in Moneyball when, at the end, it's like, if we're not building the model around your model, every company will be out of business. What's your advice to companies out there that are having those kind of moments where it's like, okay, this is real, this is next gen, this is happening. I better start thinking and putting into motion plans to refactor my business, 'cause it's happening, business transformation is happening on the cloud. This kind of puts an exclamation point on, with the AI, as a next step function. Big increase in value. So it's an opportunity for leaders. Ankur, we'll start with you. What's your advice for folks out there thinking about this? Do they put their toe in the water? Do they jump right into the deep end? What's your advice? >> Yeah, John, so we talk to a lot of customers, and customers are excited about what's happening in the space, but they often ask us like, "Hey, where do we start?" So we always advise our customers to do a lot of proof of concepts, understand where they can drive the biggest ROI. And then also leverage existing tools and services to move fast and scale, and try and not reinvent the wheel where it doesn't need to be. That's basically our advice to customers. >> Get it. Ori, what's your advice to folks who are scratching their head going, "I better jump in here. "How do I get started?" What's your advice? >> So I actually think that need to think about it really economically. Both on the opportunity side and the challenges. So there's a lot of opportunities for many companies to actually gain revenue upside by building these new generative features and capabilities. On the other hand, of course, this would probably affect the cogs, and incorporating these capabilities could probably affect the cogs. So I think we really need to think carefully about both of these sides, and also understand clearly if this is a project or an F word towards cost reduction, then the ROI is pretty clear, or revenue amplifier, where there's, again, a lot of different opportunities. So I think once you think about this in a structured way, I think, and map the different initiatives, then it's probably a good way to start and a good way to start thinking about these endeavors. >> Awesome. Clem, what's your take on this? What's your advice, folks out there? >> Yes, all of these are very good advice already. Something that you said before, John, that I disagreed a little bit, a lot of people are talking about the data mode and proprietary data. Actually, when you look at some of the organizations that have been building the best models, they don't have specialized or unique access to data. So I'm not sure that's so important today. I think what's important for companies, and it's been the same for the previous generation of technology, is their ability to build better technology faster than others. And in this new paradigm, that means being able to build machine learning faster than others, and better. So that's how, in my opinion, you should approach this. And kind of like how can you evolve your company, your teams, your products, so that you are able in the long run to build machine learning better and faster than your competitors. And if you manage to put yourself in that situation, then that's when you'll be able to differentiate yourself to really kind of be impactful and get results. That's really hard to do. It's something really different, because machine learning and AI is a different paradigm than traditional software. So this is going to be challenging, but I think if you manage to nail that, then the future is going to be very interesting for your company. >> That's a great point. Thanks for calling that out. I think this all reminds me of the cloud days early on. If you went to the cloud early, you took advantage of it when the pandemic hit. If you weren't native in the cloud, you got hamstrung by that, you were flatfooted. So just get in there. (laughs) Get in the cloud, get into AI, you're going to be good. Thanks for for calling that. Final parting comments, what's your most exciting thing going on right now for you guys? Ori, Clem, what's the most exciting thing on your plate right now that you'd like to share with folks? >> I mean, for me it's just the diversity of use cases and really creative ways of companies leveraging this technology. Every day I speak with about two, three customers, and I'm continuously being surprised by the creative ideas. And the future is really exciting of what can be achieved here. And also I'm amazed by the pace that things move in this industry. It's just, there's not at dull moment. So, definitely exciting times. >> Clem, what are you most excited about right now? >> For me, it's all the new open source models that have been released in the past few weeks, and that they'll keep being released in the next few weeks. I'm also super excited about more and more companies getting into this capability of chaining different models and different APIs. I think that's a very, very interesting development, because it creates new capabilities, new possibilities, new functionalities that weren't possible before. You can plug an API with an open source embedding model, with like a no-geo transcription model. So that's also very exciting. This capability of having more interoperable machine learning will also, I think, open a lot of interesting things in the future. >> Clem, congratulations on your success at Hugging Face. Please pass that on to your team. Ori, congratulations on your success, and continue to, just day one. I mean, it's just the beginning. It's not even scratching the service. Ankur, I'll give you the last word. What are you excited for at AWS? More cloud goodness coming here with AI. Give you the final word. >> Yeah, so as both Clem and Ori said, I think the research in the space is moving really, really fast, so we are excited about that. But we are also excited to see the speed at which enterprises and other AWS customers are applying machine learning to solve real business problems, and the kind of results they're seeing. So when they come back to us and tell us the kind of improvement in their business metrics and overall customer experience that they're driving and they're seeing real business results, that's what keeps us going and inspires us to continue inventing on their behalf. >> Gentlemen, thank you so much for this awesome high impact panel. Ankur, Clem, Ori, congratulations on all your success. We'll see you around. Thanks for coming on. Generative AI, riding the wave, it's a tidal wave, it's the water, it's all happening. All great stuff. This is season three, episode one of AWS Startup Showcase closing panel. This is the AI ML episode, the top startups building generative AI on AWS. I'm John Furrier, your host. Thanks for watching. (mellow music)

>> From The Cube Studios in Palo Alto and Boston, bringing you data driven insights from The Cube and ETR, this is Breaking Analysis with Dave Vellante. >> Despite it's faster growth and infrastructure as a service, relative to AWS and Azure, Google Cloud platform remains a third wheel in the race for cloud dominance. Google begins its Cloud Next online event starting July fourteenth in a series of nine rolling sessions that go through early September. Ahead of that, we want to update you on our most current data on Google's cloud business. Hello everyone, this is Dave Vellante, and welcome to this week's Wikibon Cube insights, powered by ETR. In this session, we'll review the current state of cloud, and Google's position in the market. We'll drill into the ETR data and share fresh insights from our partner and the Cube community. So let's get right into it. You know, Google, if you think about it, was actually very early into the cloud game. Google's 2004 IPO was a milestone event for the tech industry, and in you know many ways, it really marked the end of the post-dotcom malaise. It signaled the beginning of a new era of innovation. During this time, Google was busy building out its massive, global cloud infrastructure, probably the largest in the world, with undersea cables, global data centers, and tools like the Google file system, and of course Bigtable. But it took many years for Google to pull its head out of its ad serving butt and realize the opportunity to sell its cloud services to global enterprises. Bigtable, Google's no-sequel database, for example, was released in 2005, but it wasn't until 2015 that Google made this service available to its customers. That was the same year Google brought in VMware founder, Diane Greene to begin its enterprise journey in earnest. Now Google, they have a dizzying array of services in compute, storage, database, networking, IT ops, dev tools, machine learning, AI, analytics, big data, security, on and on and on. Name a category and it's likely that Google has something in it as a cloud service. But Google, to this day, still hasn't figured out how to sell to the enterprise. It really struggles to find the right formula. So, as you know, Google brought in Thomas Kurian from Oracle, to figure this out. Of course Kurian is, he's going to go with Google's strengths like analytics and database, but it has to have differentiation, so it comes up with unique pricing models like sustained discounts, which automatically apply discount for heavy usage, as opposed to forcing users to buy reserved instances such as what AWS does. You know Google is more aggressive partnering around multi-cloud, for instance, with Anthos, and it's smartly open-sourced Kubernetes really to minimize the importance of, physically, where workloads run. The bottom-line, however, is that these moves are necessary for Google to compete because it lags behind the leaders. And it has a long way to go before it's going to be satisfied with its cloud business. Let's look at the IaaS market in context. Now, I don't want to say it's all gloom and doom for Google. Far from it. Earnings for Q2, they're going to start rolling out later this month, but this chart shows our latest estimates of IaaS and PaaS for the big three cloud players. Now, I got to caution you, as I did before, other than AWS, which reports very clean numbers each quarter on IaaS and PaaS, we have to estimate Azure and GCP revenue because they bundle in other things. I'll give an example. Google reports its overall cloud numbers which include G Suite. Microsoft reports a category they call intelligent cloud. Now that includes public, private clouds, hybrid, sequel server, Windows server, system center, GitHub, enterprise support and consulting services. And Azure, the IaaS and PaaS numbers are also in there too. So what we have to do is to squint through the earnings reports and the 10 Ks and try to get a clean IaaS and PaaS figure for these players, and that's what we show here. Now there's really two points that we want to stress with this data. First, on a trailing 12 month basis, the big three cloud players now account for nearly 60 billion dollars in IaaS and PaaS revenue. And this 60 billion dollars, on a weighted average basis, is growing in the mid 40% range. So well on its way to being a 100 billion dollar business. Just for these three firms. And as we've reported, that's eating directly into the on-premises infrastructure install base, which is a flat to declining market. And that trend is going to play out in a big way this decade. We've predicted that public cloud is going to out pace on-prem infrastructure by more that 1800 basis points over the next 10 years, from a spending standpoint. Now the second point that I want to make relates to Google IaaS and PaaS growth. We peg it at greater than 70%, based on public statements, reading the 10 Ks and ETR data, which we'll discuss in a moment. So, very healthy growth, but from a much smaller install base than, or base than AWS and Azure. But in our view it's not enough, because AWS and Azure are so large and strong still, growth wise, that we feel Google is going to remain a distant third, really indefinitely. Nonetheless, a lot of companies would be thrilled to have a four billion dollar cloud business and there's certainly good news in the data for Google. So let's look at some of that survey data. Now, as we've reported in the past, Google pushes G Suite very hard, as part of its cloud story, and it leads often times with G Suite in its messaging. You know, but to us that's never really been that compelling. So let me start with some anecdotal data from ETR. ETR runs a regular program, they call it VENN, and in the VENN they invite clients into a private session to listen to named CIOs talk about their experience with vendors and overall spending intentions. It's a facilitated session. And we've had ETR's Eric Bradley on as a guest who directs the VENN program, and does much of the facilitation, and here's a statement from a recent VENN session quoting a CIO at a midsize Telco, that I think sums it up nicely. He says Google's G Suite is fine and dandy, but I don't see that truly as an enterprise solution. And frankly, it's still not of the quality of an Office application, talking about Microsoft. All in all I really like the infrastructure-as-a-service and the platform-as-a-service components that GCP had. And I thought they were coming along very very well in that space. Now, the reason that I share this is because the IT buyers that we speak with, you know they're very serious about exploring Google. They want options other than Azure and AWS and they see Google as having great tech and as a viable alternative. So let's talk about GCP and the enterprise. We looking, when we look into the ETR data for the most recent survey, which ran in June and early July, GCP is showing strength in one really important bellwether category, the giant public and private companies. These are the largest firms in the ETR dataset and often point to secular trends. Now, before we get into that, let's look at the picture for GCP using ETR's net score up methodology. This is fundamental to the ETR approach, and remember, each quarter ETR goes out and asks its respondents, are you planning to spend more or less? In its July survey, ETR focuses on second half spending. The next chart captures results across Google's entire portfolio. So here's the breakdown for, for Google across all sectors. 14% of the respondents are adopting new, that's the lime green. 39% plan to increase spending in the second half versus the first half, that's the forest green. Then there's a big fat middle, that's flat, and you see that in the gray area. And the 7% are spending less, with 2% replacing, that's the pinkish and dark red, respectively. So, I would say this result is mixed, in my opinion. Yeah, it's not bad, don't get me wrong, and we've, we'll see once ETR comes out of its quite period, how this compares to Azure and AWR, so remember, I can only share limited data until ETR clients get the data and have time to act on it. But this calculates out to a net score of 44%, which is respectable, but frankly not overly inspiring. So let's look across the GCP portfolio using the ETR taxonomy and see what it looks like. This chart shows the net score comparisons across three different surveys, October 19, April 20, and July 20. So reading the bars left to right, you can see Google's strong suit really is machine learning and AI. Container platforms are also very strong, as are functions, or server-less, and databases, very solid, we'll talk more about that in a minute. You know, video conferencing was just added by ETR and sure it pops up with the work from home. Cloud is actually holding firm when compared to October of last year. But surprisingly, analytics is looking a bit softer. And ETR for the first time added G Suite with, it shows a 26% net score, first time out, which is pretty tepid. I mean not very impressive at all. But overall, the picture looks pretty good for Google. So let's dig further into the giant public and private sector, that bellwether I talked about. And let's peal the onion a bit and look closer at the results from the largest companies in the dataset. So this chart shows the giant public, plus private organizations. So it would include like monster public companies but also large companies like a Cargill or a Coke Industries, if in fact they responded in this survey. And you can see, in that all important sector, it's a story of a lot of green with hardly any red, so quite a positive sign for Google within those bellwethers. Here's what I think is happening here. Is these large, and often far flung organizations, have realized that they have multiple cloud vendors, and they're asking their senior IT leadership to bring some consistency and sanity to their cloud strategies. So they look at the big three and say, okay, what's the best strategic fit for each workload? So they might say for instance let's use AWS for core IaaS, let's use Azure for productivity workloads, and we'll sprinkle some Google in for machine learning and related projects. So we do see some real strength in some of the larger strongholds for Google, although interestingly ETR sort of tells me that there's softness in the midsize and smaller companies that have powered AWS for so many years. And of course this, with Google's base, but compare that to AWS and AWS is much stronger in those smaller companies, start-ups and the like, and of course COVID's the wild car in all this. You know, we have to take that into account, and we will with Sagar Kadakia, who's ETR's director of research in the coming weeks. But I want to look at Google in the all important database category. So before we wrap, let's look at database. You remember, Google's playing catch up in the cloud and its marketing takes a more open posture around partners and things like multi-cloud and you know you can contrast that with AWS for example, but look, make no mistake, Google wants you data in their cloud, and that's why database is so strategic and so important. Look, it's the mother of all lock specs. All you got to do is look at Oracle and their success. Now, as we've reported many times, there's a new workload emerging in the cloud around this idea of the modern data warehouse. I mean I don't even like that term anymore, data warehouse, because it sounds just so static. But anyway, any rate, I'm talking about workloads that bring database, machine learning, AI, data science, compute and storage along with visualization tools to deliver real-time insights and operational analytics. Database is at the heart of everything here. Win the database and everything else falls into place. Now, Google has six or seven database products and one of the most impressive, in my opinion, is BigQuery. I mean, for those who have followed me over the years you know I love the technology behind Google's banner, but BigQuery is where much of the action is around this new workload that I'm talking about. So, let's look at, deeper at Google's position in database. This chart shows one of my favorite views. On the Y axis is the net score, or spending momentum, and on the X axis is market share or pervasiveness in the ETR dataset. The chart plots various database companies and their position within the all important giant public plus private sector. So these are the companies in the ETR survey that are the largest, and oftentimes, again, are a bellwether. And you can see Microsoft and Oracle and AWS have very strong presence on the horizontal axis. Mongo, MongoDB looms large, MemSQL, they just raised 50 million dollars this past May, MariaDB just raised another 25 million this month. You can see Couchbase and Redis, they show up, and they're on my radar. I'm learning more about those companies. Folks, database is hot. VC's are pouring money in and it's something that's very important to the Cube community to look at. And of course you see Google in the chart, with a strong net score, you know, but not the type of market presence that you see from the other big cloud players. In fact, they've pulled back a little somewhat in this last ETR survey. So despite some bright spots in the enterprise in terms of spending momentum, just not quite enough presence yet. Oh, by the way, look who's right there with Google. I know I sound like a broken record, but Snowflake is everywhere. You'll find them in AWS, you'll find them in Azure and on GCP. Now remember, Snowflake is only about one tenth the size of Google's IaaS and PaaS business. But it has stronger spending momentum than all the big guys, and it continues to creep its way to the right in terms of market share or presence. You know, but Google has great database tech and BigQuery is at the heart of its strategy to support analytics at scale, and automate the data pipeline. BigQuery's very well designed, it started as a cloud native database, it's based on server-less, it's highly scalable, and it's very cost-effective. In fact, ESG, enterprise strategy group, wrote a report comparing the TCO of the cloud databases. Let me pull that up and show you. Now the report was commissioned by Google, so I got to caution you there. But it was very well done in my opinion by a guy named Aviv Kaufmann, and you can see here it compares BigQuery with the other cloud databases, and of course, you know, BigQuery wins, got the lowest TCO, but again I thought the report was really detailed and well researched. I have no doubt that Snowflake has an answer for the big brown bar, which is on-demand cloud cost. I think ESG was making certain assumptions, maybe worst case assumptions, about the need to over-provision resources for Snowflake, which I'm sure ESG can defend, but I'll bet dollars to donuts that Snowflake, you know, has an answer to that or a comeback. I'm going to ask them. But the point I want to make here is that BigQuery was designed from day one, again, as a cloud-native database. We've been talking about that a lot. It's very efficient and is going to be competitive. So you can see, there are some bright spots in the enterprise, for Google. Okay, let's wrap up. Now, having called out some of the positives, and there are many, Google is still not getting it done in the enterprise, in my opinion. I certainly would not say too little too late, but I would say they spotted the competition a huge lead, and the only reason is Google just didn't act on the opportunity staring them in the face, within the enterprise, fast enough, and they finally woke up. But enterprise sales are, they're really hard. Thomas Kurian, for all his experience, is coming from way, way behind with regard to the enterprise go to market, systems and processes, pricing, partnerships, special deals for the enterprise. Google's still learning how to sell the business outcomes and is relying far too much on its technology chops, which, while impressive, are not going to win the day without better enterprise sales, marketing, and ecosystem integration. Now I feel like for years, Google has said to the enterprise market, give me heat and I'll add the wood. Meaning we have the best tech, go ahead and use it. That strategy just doesn't work in the enterprise. Kurian knows it and I suspect that's why Google's showing some strength within these large, giant public and private companies. They're probably applying focused sales resources to nail customer success with some of its top accounts where they have a presence, and then once they nail that they'll broaden to the market. But they got to move fast. We'll learn more about Google's intentions and its progress over the next few, next few months as they try their online event experiment, and of course we'll be there providing our wall to wall coverage. Remember, these Breaking Analysis episodes, they're all available as podcasts. ETR is shortly exiting its quiet period, this week, and will be rolling out the data, so check out etr.plus. I publish weekly on wikibon.com and siloconeangle.com and as always please comment on my LinkedIn posts, I really appreciate the feedback. This is Dave Vellante for the Cube Insights, powered by ETR. Thanks for watching everyone. We'll see you next time.

(Gentle music) >> Hello and welcome to the Cube here in Palo Alto, California. I'm John Furrier here, joined by Platform nine, Amelia Bell the Chief Revenue Officer, really digging into the conversation around Kubernetes Cloud native and the journey this next generation cloud. Amelia, thanks for coming in and joining me today. >> Thank you, thank you. Great pleasure to be here. >> So, CRO, chief Revenue Officer. So you're mainly in charge of serving the customers, making sure they're they're happy with the solution you guys have. >> That's right. >> And this market must be pretty exciting. >> Oh, it's very exciting and we are seeing a lot of new use cases coming up all the time. So part of my job is to obtain new customers but then of course, service our existing customers and then there's a constant evolution. Nothing is standing still right now. >> We've had all your co-founders on, on the show here and we've kind of talked about the trends and where you guys have come from, where you guys are going now. And it's interesting, if you look at the cloud native market, the scale is still huge. You seeing now this next wave of AI coming on, which I call that's the real web three in my mind in terms of like the next experiences really still points to data infrastructure scale. These next gen apps are coming. And so that's being built on the previous generation of DevSecOps. >> Right >> And so a lot of enterprises are having to grow up really, really fast >> Right. >> And figure out, okay, I got to have scale I got large scale data, I got horizontal scalability I got to apply machine learning now the new software engineering practice. And then, oh, by the way I got the Kubernetes clusters I got to manage >> Right. >> I got what's containers weather, the security problems. This is a really complicated but important area of build out right now in the marketplace. >> Right. What are you seeing? >> So it's, it's really important that the infrastructure is not the hindrance in these cases. And we, one of our customers is in fact a large AI company and we, I met with them yesterday and asked them, you know, why are you giving that to us? You've got really smart engineers. They can run and create the infrastructure, you know in a custom way that you want it. And they said, we've got to be core to our business. There's plenty of work to do just on delivering the AI capabilities, and there's plenty of work to do. We can't get bogged down in the infrastructure. We don't want to have people running the engine we want them driving the car. We want them creating value on top of that. so they can't have the infrastructure being the bottleneck for them. >> It's interesting, the AI companies, that's their value proposition to their customers is that they don't want the technical talent. >> Right. >> Working on, you know, non-differentiated heavy lifting things. >> Right. >> And automate those and scale it up. Can you talk about the problem that you guys are solving? Because there's a lot going on here. >> Yeah. >> You can look at all aspects of the DevOps scale. There's a lot of little problems, some big problems. What are you guys focusing on? What's the bullseye for Platform known? >> Okay, so the bullseye is that Kubernetes infrastructure is really hard, right? It's really hard to create and run. So we introduce a time to market efficiency, let's get this up and running and let's get you into production and and producing results for your customers fast. But at the same time, let's reduce your cost and complexity and increase reliability. So, >> And what are some of the things that they're having problems with that are breaking? Is it more of updates on code? Is it size of the, I mean clusters they have, what what is it more operational? What are the, what are some of the things that are that kind of get them to call you guys up? What's the main thing? >> It's the operations. It's all operations. So what, what happens is that if you have a look at Kubernetes platform it's made up of many, many components. And that's where it gets complex. It's not just Kubernetes. There's load balances, networking, there's observability. All these things have to operate together. And all the piece parts have to be upgraded and maintained. The integrations need to work, you need to have probes into the system to predict where problems can be coming. So the operational part of it is complex. So you need to be observing not only your clusters in the health of the clusters and the nodes and so on but the health of the platform itself. >> We're going to get Peter Frey in on here after I talk about some of the technical issues on deployments. But what's the, what's the big decision for the customer? Because there's kind of, there's two schools of thought. One is, I'm going to build my own and have my team build it or I'm going to go with a partner >> Right. >> Say platform nine, what's the trade offs there? Because it seems to me that, that there's a there's a certain area of where it's core competency but I can outsource it or partner with it and, and work with platform nine versus trying to take it all on internally >> Right. >> Of which requires more costs. So there's a, there's a line where you kind of like figure out that customers have to figure out that, that piece >> Right >> What do, what's your view on that? Because I'm hearing that more people are saying, hey I want to, I want to focus my people on solutions. The app side, not so much the ops >> Right. >> What's the trade off? How do you talk about? >> It's a really interesting question because most companies think they have two options. It's either a DIY option and they love that engineers love playing with the new and on the latest. And then they think the other option is going to cloud, public cloud and have it semi managed by them. And you get very different out of those. So in the DIY you get flexibility coz you get to choose your infrastructure but then you've got all the complexities of the DIY piece. You've got to not only choose all your components but you've got to keep them working. Now if you go to public cloud option, you lose flexibility because a lot of those choices are made for you but you gain agility because quite frankly it's really easy to spin up clusters. So what we are, is that in the middle we bring the agility and the flexibility because we bring the control plane that allows you to spin up clusters and and lifecycle manage them very quickly. So the agility's there but you can do it on the infrastructure of your choice. And in the DIY culture, one of the hardest things to do actually is to convince them they don't have to do it themselves. They can focus on higher value activities, which are more focused on delivering outcomes to their customers. >> So you provide the solution that allows them to feel like they're billing it themselves. >> Correct. >> And get these scale and speed and the efficiencies of the op side. So it's kind of the best of both worlds. It's not a full outsource. >> Right, right. >> You're bringing them in to make their jobs easier >> Right, That's right. So they get choices. >> Yeah. >> We, we, they get choices on how they build it and then we run and operate it for them. But they, they have all the observability. The benefit is that if we are managing their operations and most of our customers choose the managed operations piece of it, then they don't. If something goes wrong, we fix that and they, they they get told, oh, by the way, you had a problem. We've dealt with it. But in the other model is they've got to create all that observability themselves and they've got to get ahead of the issues themselves, and then they've got to raise tickets to whoever they need to raise tickets to. Whereas we have things like auto ticket generation and so on where, look, just drive the car let us worry about the engine and all of that. Let us deal with that. And you can choose whatever you want about the engine but let us manage it for you. So >> What do you, what do you say to folks out there that are may have a need for platform nine? What's the signals inside their company that they should be calling you guys up and, and leaning in with platform nine? >> Right. >> Is it more sprawl on on clusters? Is it more errors? Is it more tickets? Is it more hassle? What are some of the signs? If someone's watching this say, hey I have, I have an issue with this. >> I would say, if there's operational inefficiencies you can't get things to market fast enough because you are building this and it's just taking too long you're spending way too much time operationally on the infrastructure, then you are, you are not using your resources where they should best be used. And, and that is delivering services to the customer. >> Ed me Hora on for International Women's Day. And she was talking about how they love to solve complex problems on the engineering team at Platform nine. It's going to get pretty complex with the edge emerging >> Indeed >> and cloud native on-premises distributed computing. >> Indeed. >> essentially is what it is. That's kind of the core DNA of the team. >> Yeah. >> What, how does that translate to the customers? Because IT seems to be, okay, I have virtual machines were great, now I got to scale up and and convert over a transform to containers, Kubernetes >> Right. >> And then large scale app, app applications. >> Right, so when it comes to Edge it gets complex pretty fast because it's highly distributed. So how do you have standardization and governance across all the different edge locations? So what we bring into play is an ability to, um, at each edge, location eh, provision from bare metal up all the way up to the application. So let's say you have thousands of stores and you want to modernize those stores, you know rather than having a server being sent somewhere to have an image loaded up and then sent that and then you've got to send a technical guide to the store and you've got to implement it all there. Forget all that. That's just, that's just a ridiculous waste of time. So what we've done is we've created the ability where the server can just be sent to the store. You can get your barista or your chef just to plug it in, right? You don't need to send any technical person over there. As long as we have access to it, we get access to it and we provision the whole thing from bare metal up and then we can maintain it according to the standards that are needed and upgrade accordingly. And that gives standardization across all your stores or edge locations or 5G towers or whatever it is, distribution centers. And we can create nice governance and good standardization which allows them to innovate fast as well. >> So this is a real opportunity for you guys. >> Yeah. >> This is an advantage from your expertise. >> Yes. >> The edge piece, dropping in a box, self-provisioning. >> That's right. So yeah. >> Can people do that? What's the, >> No, actually it, it's, it's very difficult to do. I I, from my understanding, we're the only people that can provision it from bare metal up, right? So if anyone has a different story, I'd love to hear about that. But that's my understanding today. >> That's a good value purpose. So talk about the value of the customer. What kind of scope do you got? Can you scope some of the customer environments you have from >> Sure. >> From, you know, small to the large, how give us an idea of the order of magnitude of the >> Yeah, so, so small customers may have 20 clusters or something like that. 20 nodes, I beg your pardon. Our large customers, like we're we are scaling one particular distributed environment from 2200 nodes to 10,000 nodes by the end of this year and 26,000 nodes next year. We have another customer that's scaling up to 10,000 nodes this year as well. So we have some very large scale, but some smaller ones too. And we're, we're happy to work with either end. >> Okay, so pretend I'm a customer. I'm really, I got pain and Kubernetes like I want to, I can't hire enough people. I want to have my all focus. What's the pitch? >> Okay. So skill shortage is something that that everyone is facing right now. And if, if you've got skill shortage it's going to be really hard to hire if you are competing against really, you know, high salary you know, offering companies that are out there. So the pitch is, let us do it for you. We have, we have a team of excellent probably the best Kubernetes engineers on the planet. We will create your environment for you. We will get it up and running. We will allow you to, you know, run your applica, just consume the platform, we'll run it for you. We'll have SLAs and up times guaranteed and you can just focus on delivering the software and the value needed to your customers. >> What are some of the testimonials that you get from people? Just anecdotally, what do they say? Oh my god, you guys save. >> Yeah. >> Our butts. >> Yeah. >> This is amazing. We just shipped our code out much faster. >> Yeah. >> What are some of the things that you hear? >> So, so the number one thing I hear is it just works right? It's, we don't have to worry about it, it just works. So that, that's a really great feedback that we get. The other thing I hear is if we do have issues that your team are amazing, they they fix things, they're proactive, you know, they're we really enjoy working with you. So from, from that perspective, that's great. But the other side of it is we hear things like if we were to do that ourselves we would've taken six to 12 months to build that. And you guys have just saved us six to 12 months. The other thing that we hear is with the same two engineers we started on, you know, a hundred nodes we're now running thousands of nodes. We have not had to increase the size of the team and expand and scale exponentially. >> Awesome. What's next for you guys? What's on your, your plate? >> Yeah. >> With CRO, what's some of the goals you have? >> Yeah, so growth of course as a CRO, you don't get away from that. We've got some very exciting, actually, initiatives coming up. One of the things that we are seeing a lot of demand for and is, is in the area of virtualization bringing virtual machine, virtual virtual containers, sorry I'm saying that all wrong. Bringing virtual machine, the virtual machines onto the cloud native infrastructure using Kubernetes technology. So that provides a, an excellent stepping stone for those guys who are in the virtualization world. And they can't move to containers, they can't refactor their applications and workloads fast enough. So just bring your virtual machine and put it onto the container infrastructure. So we're seeing a lot of demand for that, because it provides an excellent stepping stone. Why not use Kubernetes to orchestrate virtual the virtual world? And then we've got some really interesting cost optimization. >> So a lot of migration kind of thinking around VMs and >> Oh, tremendous. The, the VM world is just massively bigger than the container world right now. So you can't ignore that. So we are providing basically the evolution, the the journey for the customers to utilize the greatest of technologies without having to do that in a, in a in a way that just breaks the bank and they can't get there fast enough. So we provide those stepping stones for them. Yeah. >> Amelia thank you for coming on. Sharing. >> Thank you. >> The update on platform nine. Congratulations on your big accounts you have and >> thank you. >> And the world could get more complex, which Means >> indeed >> have more customers. >> Thank you, thank you John. Appreciate that. Thank you. >> I'm John Furry. You're watching Platform nine and the Cube Conversations here. Thanks for watching. (gentle music)

(chill electronic music) >> Hello everyone, welcome to theCUBE's presentation of the AWS Startups Showcase, AI and machine learning, the top startups building generative AI on AWS. This is the season three, episode one of the ongoing series covering the exciting startups from the AWS ecosystem, talk about AI and machine learning. Can't believe it's three years and season one. I'm your host, John Furrier. Got a great guest today, we're joined by Joseph Nelson, the co-founder and CEO of Roboflow, doing some cutting edge stuff around computer vision and really at the front end of this massive wave coming around, large language models, computer vision. The next gen AI is here, and it's just getting started. We haven't even scratched a service. Thanks for joining us today. >> Thanks for having me. >> So you got to love the large language model, foundation models, really educating the mainstream world. ChatGPT has got everyone in the frenzy. This is educating the world around this next gen AI capabilities, enterprise, image and video data, all a big part of it. I mean the edge of the network, Mobile World Conference is happening right now, this month, and it's just ending up, it's just continue to explode. Video is huge. So take us through the company, do a quick explanation of what you guys are doing, when you were founded. Talk about what the company's mission is, and what's your North Star, why do you exist? >> Yeah, Roboflow exists to really kind of make the world programmable. I like to say make the world be read and write access. And our North Star is enabling developers, predominantly, to build that future. If you look around, anything that you see will have software related to it, and can kind of be turned into software. The limiting reactant though, is how to enable computers and machines to understand things as well as people can. And in a lot of ways, computer vision is that missing element that enables anything that you see to become software. So in the virtue of, if software is eating the world, computer vision kind of makes the aperture infinitely wide. It's something that I kind of like, the way I like to frame it. And the capabilities are there, the open source models are there, the amount of data is there, the computer capabilities are only improving annually, but there's a pretty big dearth of tooling, and an early but promising sign of the explosion of use cases, models, and data sets that companies, developers, hobbyists alike will need to bring these capabilities to bear. So Roboflow is in the game of building the community around that capability, building the use cases that allow developers and enterprises to use computer vision, and providing the tooling for companies and developers to be able to add computer vision, create better data sets, and deploy to production, quickly, easily, safely, invaluably. >> You know, Joseph, the word in production is actually real now. You're seeing a lot more people doing in production activities. That's a real hot one and usually it's slower, but it's gone faster, and I think that's going to be more the same. And I think the parallel between what we're seeing on the large language models coming into computer vision, and as you mentioned, video's data, right? I mean we're doing video right now, we're transcribing it into a transcript, linking up to your linguistics, times and the timestamp, I mean everything's data and that really kind of feeds. So this connection between what we're seeing, the large language and computer vision are coming together kind of cousins, brothers. I mean, how would you compare, how would you explain to someone, because everyone's like on this wave of watching people bang out their homework assignments, and you know, write some hacks on code with some of the open AI technologies, there is a corollary directly related to to the vision side. Can you explain? >> Yeah, the rise of large language models are showing what's possible, especially with text, and I think increasingly will get multimodal as the images and video become ingested. Though there's kind of this still core missing element of basically like understanding. So the rise of large language models kind of create this new area of generative AI, and generative AI in the context of computer vision is a lot of, you know, creating video and image assets and content. There's also this whole surface area to understanding what's already created. Basically digitizing physical, real world things. I mean the Metaverse can't be built if we don't know how to mirror or create or identify the objects that we want to interact with in our everyday lives. And where computer vision comes to play in, especially what we've seen at Roboflow is, you know, a little over a hundred thousand developers now have built with our tools. That's to the tune of a hundred million labeled open source images, over 10,000 pre-trained models. And they've kind of showcased to us all of the ways that computer vision is impacting and bringing the world to life. And these are things that, you know, even before large language models and generative AI, you had pretty impressive capabilities, and when you add the two together, it actually unlocks these kind of new capabilities. So for example, you know, one of our users actually powers the broadcast feeds at Wimbledon. So here we're talking about video, we're streaming, we're doing things live, we've got folks that are cropping and making sure we look good, and audio/visual all plugged in correctly. When you broadcast Wimbledon, you'll notice that the camera controllers need to do things like track the ball, which is moving at extremely high speeds and zoom crop, pan tilt, as well as determine if the ball bounced in or out. The very controversial but critical key to a lot of tennis matches. And a lot of that has been historically done with the trained, but fallible human eye and computer vision is, you know, well suited for this task to say, how do we track, pan, tilt, zoom, and see, track the tennis ball in real time, run at 30 plus frames per second, and do it all on the edge. And those are capabilities that, you know, were kind of like science fiction, maybe even a decade ago, and certainly five years ago. Now the interesting thing, is that with the advent of of generative AI, you can start to do things like create your own training data sets, or kind of create logic around once you have this visual input. And teams at Tesla have actually been speaking about, of course the autopilot team's focused on doing vision tasks, but they've combined large language models to add reasoning and logic. So given that you see, let's say the tennis ball, what do you want to do? And being able to combine the capabilities of what LLM's represent, which is really a lot of basically, core human reasoning and logic, with computer vision for the inputs of what's possible, creates these new capabilities, let alone multimodality, which I'm sure we'll talk more about. >> Yeah, and it's really, I mean it's almost intoxicating. It's amazing that this is so capable because the cloud scales here, you got the edge developing, you can decouple compute power, and let Moore's law and all the new silicone and the processors and the GPUs do their thing, and you got open source booming. You're kind of getting at this next segment I wanted to get into, which is the, how people should be thinking about these advances of the computer vision. So this is now a next wave, it's here. I mean I'd love to have that for baseball because I'm always like, "Oh, it should have been a strike." I'm sure that's going to be coming soon, but what is the computer vision capable of doing today? I guess that's my first question. You hit some of it, unpack that a little bit. What does general AI mean in computer vision? What's the new thing? Because there are old technology's been around, proprietary, bolted onto hardware, but hardware advances at a different pace, but now you got new capabilities, generative AI for vision, what does that mean? >> Yeah, so computer vision, you know, at its core is basically enabling machines, computers, to understand, process, and act on visual data as effective or more effective than people can. Traditionally this has been, you know, task types like classification, which you know, identifying if a given image belongs in a certain category of goods on maybe a retail site, is the shoes or is it clothing? Or object detection, which is, you know, creating bounding boxes, which allows you to do things like count how many things are present, or maybe measure the speed of something, or trigger an alert when something becomes visible in frame that wasn't previously visible in frame, or instant segmentation where you're creating pixel wise segmentations for both instance and semantic segmentation, where you often see these kind of beautiful visuals of the polygon surrounding objects that you see. Then you have key point detection, which is where you see, you know, athletes, and each of their joints are kind of outlined is another more traditional type problem in signal processing and computer vision. With generative AI, you kind of get a whole new class of problem types that are opened up. So in a lot of ways I think about generative AI in computer vision as some of the, you know, problems that you aimed to tackle, might still be better suited for one of the previous task types we were discussing. Some of those problem types may be better suited for using a generative technique, and some are problem types that just previously wouldn't have been possible absent generative AI. And so if you make that kind of Venn diagram in your head, you can think about, okay, you know, visual question answering is a task type where if I give you an image and I say, you know, "How many people are in this image?" We could either build an object detection model that might count all those people, or maybe a visual question answering system would sufficiently answer this type of problem. Let alone generative AI being able to create new training data for old systems. And that's something that we've seen be an increasingly prominent use case for our users, as much as things that we advise our customers and the community writ large to take advantage of. So ultimately those are kind of the traditional task types. I can give you some insight, maybe, into how I think about what's possible today, or five years or ten years as you sort go back. >> Yes, definitely. Let's get into that vision. >> So I kind of think about the types of use cases in terms of what's possible. If you just imagine a very simple bell curve, your normal distribution, for the longest time, the types of things that are in the center of that bell curve are identifying objects that are very common or common objects in context. Microsoft published the COCO Dataset in 2014 of common objects and contexts, of hundreds of thousands of images of chairs, forks, food, person, these sorts of things. And you know, the challenge of the day had always been, how do you identify just those 80 objects? So if we think about the bell curve, that'd be maybe the like dead center of the curve, where there's a lot of those objects present, and it's a very common thing that needs to be identified. But it's a very, very, very small sliver of the distribution. Now if you go out to the way long tail, let's go like deep into the tail of this imagined visual normal distribution, you're going to have a problem like one of our customers, Rivian, in tandem with AWS, is tackling, to do visual quality assurance and manufacturing in production processes. Now only Rivian knows what a Rivian is supposed to look like. Only they know the imagery of what their goods that are going to be produced are. And then between those long tails of proprietary data of highly specific things that need to be understood, in the center of the curve, you have a whole kind of messy middle, type of problems I like to say. The way I think about computer vision advancing, is it's basically you have larger and larger and more capable models that eat from the center out, right? So if you have a model that, you know, understands the 80 classes in COCO, well, pretty soon you have advances like Clip, which was trained on 400 million image text pairs, and has a greater understanding of a wider array of objects than just 80 classes in context. And over time you'll get more and more of these larger models that kind of eat outwards from that center of the distribution. And so the question becomes for companies, when can you rely on maybe a model that just already exists? How do you use your data to get what may be capable off the shelf, so to speak, into something that is usable for you? Or, if you're in those long tails and you have proprietary data, how do you take advantage of the greatest asset you have, which is observed visual information that you want to put to work for your customers, and you're kind of living in the long tails, and you need to adapt state of the art for your capabilities. So my mental model for like how computer vision advances is you have that bell curve, and you have increasingly powerful models that eat outward. And multimodality has a role to play in that, larger models have a role to play in that, more compute, more data generally has a role to play in that. But it will be a messy and I think long condition. >> Well, the thing I want to get, first of all, it's great, great mental model, I appreciate that, 'cause I think that makes a lot of sense. The question is, it seems now more than ever, with the scale and compute that's available, that not only can you eat out to the middle in your example, but there's other models you can integrate with. In the past there was siloed, static, almost bespoke. Now you're looking at larger models eating into the bell curve, as you said, but also integrating in with other stuff. So this seems to be part of that interaction. How does, first of all, is that really happening? Is that true? And then two, what does that mean for companies who want to take advantage of this? Because the old model was operational, you know? I have my cameras, they're watching stuff, whatever, and like now you're in this more of a, distributed computing, computer science mindset, not, you know, put the camera on the wall kind of- I'm oversimplifying, but you know what I'm saying. What's your take on that? >> Well, to the first point of, how are these advances happening? What I was kind of describing was, you know, almost uni-dimensional in that you have like, you're only thinking about vision, but the rise of generative techniques and multi-modality, like Clip is a multi-modal model, it has 400 million image text pairs. That will advance the generalizability at a faster rate than just treating everything as only vision. And that's kind of where LLMs and vision will intersect in a really nice and powerful way. Now in terms of like companies, how should they be thinking about taking advantage of these trends? The biggest thing that, and I think it's different, obviously, on the size of business, if you're an enterprise versus a startup. The biggest thing that I think if you're an enterprise, and you have an established scaled business model that is working for your customers, the question becomes, how do you take advantage of that established data moat, potentially, resource moats, and certainly, of course, establish a way of providing value to an end user. So for example, one of our customers, Walmart, has the advantage of one of the largest inventory and stock of any company in the world. And they also of course have substantial visual data, both from like their online catalogs, or understanding what's in stock or out of stock, or understanding, you know, the quality of things that they're going from the start of their supply chain to making it inside stores, for delivery of fulfillments. All these are are visual challenges. Now they already have a substantial trove of useful imagery to understand and teach and train large models to understand each of the individual SKUs and products that are in their stores. And so if I'm a Walmart, what I'm thinking is, how do I make sure that my petabytes of visual information is utilized in a way where I capture the proprietary benefit of the models that I can train to do tasks like, what item was this? Or maybe I'm going to create AmazonGo-like technology, or maybe I'm going to build like delivery robots, or I want to automatically know what's in and out of stock from visual input fees that I have across my in-store traffic. And that becomes the question and flavor of the day for enterprises. I've got this large amount of data, I've got an established way that I can provide more value to my own customers. How do I ensure I take advantage of the data advantage I'm already sitting on? If you're a startup, I think it's a pretty different question, and I'm happy to talk about. >> Yeah, what's startup angle on this? Because you know, they're going to want to take advantage. It's like cloud startups, cloud native startups, they were born in the cloud, they never had an IT department. So if you're a startup, is there a similar role here? And if I'm a computer vision startup, what's that mean? So can you share your your take on that, because there'll be a lot of people starting up from this. >> So the startup on the opposite advantage and disadvantage, right? Like a startup doesn't have an proven way of delivering repeatable value in the same way that a scaled enterprise does. But it does have the nimbleness to identify and take advantage of techniques that you can start from a blank slate. And I think the thing that startups need to be wary of in the generative AI enlarged language model, in multimodal world, is building what I like to call, kind of like sandcastles. A sandcastle is maybe a business model or a capability that's built on top of an assumption that is going to be pretty quickly wiped away by improving underlying model technology. So almost like if you imagine like the ocean, the waves are coming in, and they're going to wipe away your progress. You don't want to be in the position of building sandcastle business where, you don't want to bet on the fact that models aren't going to get good enough to solve the task type that you might be solving. In other words, don't take a screenshot of what's capable today. Assume that what's capable today is only going to continue to become possible. And so for a startup, what you can do, that like enterprises are quite comparatively less good at, is embedding these capabilities deeply within your products and delivering maybe a vertical based experience, where AI kind of exists in the background. >> Yeah. >> And we might not think of companies as, you know, even AI companies, it's just so embedded in the experience they provide, but that's like the vertical application example of taking AI and making it be immediately usable. Or, of course there's tons of picks and shovels businesses to be built like Roboflow, where you're enabling these enterprises to take advantage of something that they have, whether that's their data sets, their computes, or their intellect. >> Okay, so if I hear that right, by the way, I love, that's horizontally scalable, that's the large language models, go up and build them the apps, hence your developer focus. I'm sure that's probably the reason that the tsunami of developer's action. So you're saying picks and shovels tools, don't try to replicate the platform of what could be the platform. Oh, go to a VC, I'm going to build a platform. No, no, no, no, those are going to get wiped away by the large language models. Is there one large language model that will rule the world, or do you see many coming? >> Yeah, so to be clear, I think there will be useful platforms. I just think a lot of people think that they're building, let's say, you know, if we put this in the cloud context, you're building a specific type of EC2 instance. Well, it turns out that Amazon can offer that type of EC2 instance, and immediately distribute it to all of their customers. So you don't want to be in the position of just providing something that actually ends up looking like a feature, which in the context of AI, might be like a small incremental improvement on the model. If that's all you're doing, you're a sandcastle business. Now there's a lot of platform businesses that need to be built that enable businesses to get to value and do things like, how do I monitor my models? How do I create better models with my given data sets? How do I ensure that my models are doing what I want them to do? How do I find the right models to use? There's all these sorts of platform wide problems that certainly exist for businesses. I just think a lot of startups that I'm seeing right now are making the mistake of assuming the advances we're seeing are not going to accelerate or even get better. >> So if I'm a customer, if I'm a company, say I'm a startup or an enterprise, either one, same question. And I want to stand up, and I have developers working on stuff, I want to start standing up an environment to start doing stuff. Is that a service provider? Is that a managed service? Is that you guys? So how do you guys fit into your customers leaning in? Is it just for developers? Are you targeting with a specific like managed service? What's the product consumption? How do you talk to customers when they come to you? >> The thing that we do is enable, we give developers superpowers to build automated inventory tracking, self-checkout systems, identify if this image is malignant cancer or benign cancer, ensure that these products that I've produced are correct. Make sure that that the defect that might exist on this electric vehicle makes its way back for review. All these sorts of problems are immediately able to be solved and tackled. In terms of the managed services element, we have solutions as integrators that will often build on top of our tools, or we'll have companies that look to us for guidance, but ultimately the company is in control of developing and building and creating these capabilities in house. I really think the distinction is maybe less around managed service and tool, and more around ownership in the era of AI. So for example, if I'm using a managed service, in that managed service, part of their benefit is that they are learning across their customer sets, then it's a very different relationship than using a managed service where I'm developing some amount of proprietary advantages for my data sets. And I think that's a really important thing that companies are becoming attuned to, just the value of the data that they have. And so that's what we do. We tell companies that you have this proprietary, immense treasure trove of data, use that to your advantage, and think about us more like a set of tools that enable you to get value from that capability. You know, the HashiCorp's and GitLab's of the world have proven like what these businesses look like at scale. >> And you're targeting developers. When you go into a company, do you target developers with freemium, is there a paid service? Talk about the business model real quick. >> Sure, yeah. The tools are free to use and get started. When someone signs up for Roboflow, they may elect to make their work open source, in which case we're able to provide even more generous usage limits to basically move the computer vision community forward. If you elect to make your data private, you can use our hosted data set managing, data set training, model deployment, annotation tooling up to some limits. And then usually when someone validates that what they're doing gets them value, they purchase a subscription license to be able to scale up those capabilities. So like most developer centric products, it's free to get started, free to prove, free to poke around, develop what you think is possible. And then once you're getting to value, then we're able to capture the commercial upside in the value that's being provided. >> Love the business model. It's right in line with where the market is. There's kind of no standards bodies these days. The developers are the ones who are deciding kind of what the standards are by their adoption. I think making that easy for developers to get value as the model open sources continuing to grow, you can see more of that. Great perspective Joseph, thanks for sharing that. Put a plug in for the company. What are you guys doing right now? Where are you in your growth? What are you looking for? How should people engage? Give the quick commercial for the company. >> So as I mentioned, Roboflow is I think one of the largest, if not the largest collections of computer vision models and data sets that are open source, available on the web today, and have a private set of tools that over half the Fortune 100 now rely on those tools. So we're at the stage now where we know people want what we're working on, and we're continuing to drive that type of adoption. So companies that are looking to make better models, improve their data sets, train and deploy, often will get a lot of value from our tools, and certainly reach out to talk. I'm sure there's a lot of talented engineers that are tuning in too, we're aggressively hiring. So if you are interested in being a part of making the world programmable, and being at the ground floor of the company that's creating these capabilities to be writ large, we'd love to hear from you. >> Amazing, Joseph, thanks so much for coming on and being part of the AWS Startup Showcase. Man, if I was in my twenties, I'd be knocking on your door, because it's the hottest trend right now, it's super exciting. Generative AI is just the beginning of massive sea change. Congratulations on all your success, and we'll be following you guys. Thanks for spending the time, really appreciate it. >> Thanks for having me. >> Okay, this is season three, episode one of the ongoing series covering the exciting startups from the AWS ecosystem, talking about the hottest things in tech. I'm John Furrier, your host. Thanks for watching. (chill electronic music)

(upbeat music) >> Welcome everyone to theCUBE's presentation of the AWS Startup Showcase AI Machine Learning Top Startups Building Generative AI on AWS. This is season 3, episode 1 of the ongoing series covering the exciting startup from the AWS ecosystem to talk about AI and machine learning. I'm your host, John Furrier. I'm joined by two great guests here, Adam Wenchel, who's the CEO of Arthur, and Chief Scientist of Arthur, John Dickerson. Talk about how they help people build better LLM AI systems to get them into the market faster. Gentlemen, thank you for coming on. >> Yeah, thanks for having us, John. >> Well, I got to say I got to temper my enthusiasm because the last few months explosion of interest in LLMs with ChatGPT, has opened the eyes to everybody around the reality of that this is going next gen, this is it, this is the moment, this is the the point we're going to look back and say, this is the time where AI really hit the scene for real applications. So, a lot of Large Language Models, also known as LLMs, foundational models, and generative AI is all booming. This is where all the alpha developers are going. This is where everyone's focusing their business model transformations on. This is where developers are seeing action. So it's all happening, the wave is here. So I got to ask you guys, what are you guys seeing right now? You're in the middle of it, it's hitting you guys right on. You're in the front end of this massive wave. >> Yeah, John, I don't think you have to temper your enthusiasm at all. I mean, what we're seeing every single day is, everything from existing enterprise customers coming in with new ways that they're rethinking, like business things that they've been doing for many years that they can now do an entirely different way, as well as all manner of new companies popping up, applying LLMs to everything from generating code and SQL statements to generating health transcripts and just legal briefs. Everything you can imagine. And when you actually sit down and look at these systems and the demos we get of them, the hype is definitely justified. It's pretty amazing what they're going to do. And even just internally, we built, about a month ago in January, we built an Arthur chatbot so customers could ask questions, technical questions from our, rather than read our product documentation, they could just ask this LLM a particular question and get an answer. And at the time it was like state of the art, but then just last week we decided to rebuild it because the tooling has changed so much that we, last week, we've completely rebuilt it. It's now way better, built on an entirely different stack. And the tooling has undergone a full generation worth of change in six weeks, which is crazy. So it just tells you how much energy is going into this and how fast it's evolving right now. >> John, weigh in as a chief scientist. I mean, you must be blown away. Talk about kid in the candy store. I mean, you must be looking like this saying, I mean, she must be super busy to begin with, but the change, the acceleration, can you scope the kind of change you're seeing and be specific around the areas you're seeing movement and highly accelerated change? >> Yeah, definitely. And it is very, very exciting actually, thinking back to when ChatGPT was announced, that was a night our company was throwing an event at NeurIPS, which is maybe the biggest machine learning conference out there. And the hype when that happened was palatable and it was just shocking to see how well that performed. And then obviously over the last few months since then, as LLMs have continued to enter the market, we've seen use cases for them, like Adam mentioned all over the place. And so, some things I'm excited about in this space are the use of LLMs and more generally, foundation models to redesign traditional operations, research style problems, logistics problems, like auctions, decisioning problems. So moving beyond the already amazing news cases, like creating marketing content into more core integration and a lot of the bread and butter companies and tasks that drive the American ecosystem. And I think we're just starting to see some of that. And in the next 12 months, I think we're going to see a lot more. If I had to make other predictions, I think we're going to continue seeing a lot of work being done on managing like inference time costs via shrinking models or distillation. And I don't know how to make this prediction, but at some point we're going to be seeing lots of these very large scale models operating on the edge as well. So the time scales are extremely compressed, like Adam mentioned, 12 months from now, hard to say. >> We were talking on theCUBE prior to this session here. We had theCUBE conversation here and then the Wall Street Journal just picked up on the same theme, which is the printing press moment created the enlightenment stage of the history. Here we're in the whole nother automating intellect efficiency, doing heavy lifting, the creative class coming back, a whole nother level of reality around the corner that's being hyped up. The question is, is this justified? Is there really a breakthrough here or is this just another result of continued progress with AI? Can you guys weigh in, because there's two schools of thought. There's the, "Oh my God, we're entering a new enlightenment tech phase, of the equivalent of the printing press in all areas. Then there's, Ah, it's just AI (indistinct) inch by inch. What's your guys' opinion? >> Yeah, I think on the one hand when you're down in the weeds of building AI systems all day, every day, like we are, it's easy to look at this as an incremental progress. Like we have customers who've been building on foundation models since we started the company four years ago, particular in computer vision for classification tasks, starting with pre-trained models, things like that. So that part of it doesn't feel real new, but what does feel new is just when you apply these things to language with all the breakthroughs and computational efficiency, algorithmic improvements, things like that, when you actually sit down and interact with ChatGPT or one of the other systems that's out there that's building on top of LLMs, it really is breathtaking, like, the level of understanding that they have and how quickly you can accelerate your development efforts and get an actual working system in place that solves a really important real world problem and makes people way faster, way more efficient. So I do think there's definitely something there. It's more than just incremental improvement. This feels like a real trajectory inflection point for the adoption of AI. >> John, what's your take on this? As people come into the field, I'm seeing a lot of people move from, hey, I've been coding in Python, I've been doing some development, I've been a software engineer, I'm a computer science student. I'm coding in C++ old school, OG systems person. Where do they come in? Where's the focus, where's the action? Where are the breakthroughs? Where are people jumping in and rolling up their sleeves and getting dirty with this stuff? >> Yeah, all over the place. And it's funny you mentioned students in a different life. I wore a university professor hat and so I'm very, very familiar with the teaching aspects of this. And I will say toward Adam's point, this really is a leap forward in that techniques like in a co-pilot for example, everybody's using them right now and they really do accelerate the way that we develop. When I think about the areas where people are really, really focusing right now, tooling is certainly one of them. Like you and I were chatting about LangChain right before this interview started, two or three people can sit down and create an amazing set of pipes that connect different aspects of the LLM ecosystem. Two, I would say is in engineering. So like distributed training might be one, or just understanding better ways to even be able to train large models, understanding better ways to then distill them or run them. So like this heavy interaction now between engineering and what I might call traditional machine learning from 10 years ago where you had to know a lot of math, you had to know calculus very well, things like that. Now you also need to be, again, a very strong engineer, which is exciting. >> I interviewed Swami when he talked about the news. He's ahead of Amazon's machine learning and AI when they announced Hugging Face announcement. And I reminded him how Amazon was easy to get into if you were developing a startup back in 2007,8, and that the language models had that similar problem. It's step up a lot of content and a lot of expense to get provisioned up, now it's easy. So this is the next wave of innovation. So how do you guys see that from where we are right now? Are we at that point where it's that moment where it's that cloud-like experience for LLMs and large language models? >> Yeah, go ahead John. >> I think the answer is yes. We see a number of large companies that are training these and serving these, some of which are being co-interviewed in this episode. I think we're at that. Like, you can hit one of these with a simple, single line of Python, hitting an API, you can boot this up in seconds if you want. It's easy. >> Got it. >> So I (audio cuts out). >> Well let's take a step back and talk about the company. You guys being featured here on the Showcase. Arthur, what drove you to start the company? How'd this all come together? What's the origination story? Obviously you got a big customers, how'd get started? What are you guys doing? How do you make money? Give a quick overview. >> Yeah, I think John and I come at it from slightly different angles, but for myself, I have been a part of a number of technology companies. I joined Capital One, they acquired my last company and shortly after I joined, they asked me to start their AI team. And so even though I've been doing AI for a long time, I started my career back in DARPA. It was the first time I was really working at scale in AI at an organization where there were hundreds of millions of dollars in revenue at stake with the operation of these models and that they were impacting millions of people's financial livelihoods. And so it just got me hyper-focused on these issues around making sure that your AI worked well and it worked well for your company and it worked well for the people who were being affected by it. At the time when I was doing this 2016, 2017, 2018, there just wasn't any tooling out there to support this production management model monitoring life phase of the life cycle. And so we basically left to start the company that I wanted. And John has a his own story. I'll let let you share that one, John. >> Go ahead John, you're up. >> Yeah, so I'm coming at this from a different world. So I'm on leave now from a tenured role in academia where I was leading a large lab focusing on the intersection of machine learning and economics. And so questions like fairness or the response to the dynamism on the underlying environment have been around for quite a long time in that space. And so I've been thinking very deeply about some of those more like R and D style questions as well as having deployed some automation code across a couple of different industries, some in online advertising, some in the healthcare space and so on, where concerns of, again, fairness come to bear. And so Adam and I connected to understand the space of what that might look like in the 2018 20 19 realm from a quantitative and from a human-centered point of view. And so booted things up from there. >> Yeah, bring that applied engineering R and D into the Capital One, DNA that he had at scale. I could see that fit. I got to ask you now, next step, as you guys move out and think about LLMs and the recent AI news around the generative models and the foundational models like ChatGPT, how should we be looking at that news and everyone watching might be thinking the same thing. I know at the board level companies like, we should refactor our business, this is the future. It's that kind of moment, and the tech team's like, okay, boss, how do we do this again? Or are they prepared? How should we be thinking? How should people watching be thinking about LLMs? >> Yeah, I think they really are transformative. And so, I mean, we're seeing companies all over the place. Everything from large tech companies to a lot of our large enterprise customers are launching significant projects at core parts of their business. And so, yeah, I would be surprised, if you're serious about becoming an AI native company, which most leading companies are, then this is a trend that you need to be taking seriously. And we're seeing the adoption rate. It's funny, I would say the AI adoption in the broader business world really started, let's call it four or five years ago, and it was a relatively slow adoption rate, but I think all that kind of investment in and scaling the maturity curve has paid off because the rate at which people are adopting and deploying systems based on this is tremendous. I mean, this has all just happened in the few months and we're already seeing people get systems into production. So, now there's a lot of things you have to guarantee in order to put these in production in a way that basically is added into your business and doesn't cause more headaches than it solves. And so that's where we help customers is where how do you put these out there in a way that they're going to represent your company well, they're going to perform well, they're going to do their job and do it properly. >> So in the use case, as a customer, as I think about this, there's workflows. They might have had an ML AI ops team that's around IT. Their inference engines are out there. They probably don't have a visibility on say how much it costs, they're kicking the tires. When you look at the deployment, there's a cost piece, there's a workflow piece, there's fairness you mentioned John, what should be, I should be thinking about if I'm going to be deploying stuff into production, I got to think about those things. What's your opinion? >> Yeah, I'm happy to dive in on that one. So monitoring in general is extremely important once you have one of these LLMs in production, and there have been some changes versus traditional monitoring that we can dive deeper into that LLMs are really accelerated. But a lot of that bread and butter style of things you should be looking out for remain just as important as they are for what you might call traditional machine learning models. So the underlying environment of data streams, the way users interact with these models, these are all changing over time. And so any performance metrics that you care about, traditional ones like an accuracy, if you can define that for an LLM, ones around, for example, fairness or bias. If that is a concern for your particular use case and so on. Those need to be tracked. Now there are some interesting changes that LLMs are bringing along as well. So most ML models in production that we see are relatively static in the sense that they're not getting flipped in more than maybe once a day or once a week or they're just set once and then not changed ever again. With LLMs, there's this ongoing value alignment or collection of preferences from users that is often constantly updating the model. And so that opens up all sorts of vectors for, I won't say attack, but for problems to arise in production. Like users might learn to use your system in a different way and thus change the way those preferences are getting collected and thus change your system in ways that you never intended. So maybe that went through governance already internally at the company and now it's totally, totally changed and it's through no fault of your own, but you need to be watching over that for sure. >> Talk about the reinforced learnings from human feedback. How's that factoring in to the LLMs? Is that part of it? Should people be thinking about that? Is that a component that's important? >> It certainly is, yeah. So this is one of the big tweaks that happened with InstructGPT, which is the basis model behind ChatGPT and has since gone on to be used all over the place. So value alignment I think is through RLHF like you mentioned is a very interesting space to get into and it's one that you need to watch over. Like, you're asking humans for feedback over outputs from a model and then you're updating the model with respect to that human feedback. And now you've thrown humans into the loop here in a way that is just going to complicate things. And it certainly helps in many ways. You can ask humans to, let's say that you're deploying an internal chat bot at an enterprise, you could ask humans to align that LLM behind the chatbot to, say company values. And so you're listening feedback about these company values and that's going to scoot that chatbot that you're running internally more toward the kind of language that you'd like to use internally on like a Slack channel or something like that. Watching over that model I think in that specific case, that's a compliance and HR issue as well. So while it is part of the greater LLM stack, you can also view that as an independent bit to watch over. >> Got it, and these are important factors. When people see the Bing news, they freak out how it's doing great. Then it goes off the rails, it goes big, fails big. (laughing) So these models people see that, is that human interaction or is that feedback, is that not accepting it or how do people understand how to take that input in and how to build the right apps around LLMs? This is a tough question. >> Yeah, for sure. So some of the examples that you'll see online where these chatbots go off the rails are obviously humans trying to break the system, but some of them clearly aren't. And that's because these are large statistical models and we don't know what's going to pop out of them all the time. And even if you're doing as much in-house testing at the big companies like the Go-HERE's and the OpenAI's of the world, to try to prevent things like toxicity or racism or other sorts of bad content that might lead to bad pr, you're never going to catch all of these possible holes in the model itself. And so, again, it's very, very important to keep watching over that while it's in production. >> On the business model side, how are you guys doing? What's the approach? How do you guys engage with customers? Take a minute to explain the customer engagement. What do they need? What do you need? How's that work? >> Yeah, I can talk a little bit about that. So it's really easy to get started. It's literally a matter of like just handing out an API key and people can get started. And so we also offer alternative, we also offer versions that can be installed on-prem for models that, we find a lot of our customers have models that deal with very sensitive data. So you can run it in your cloud account or use our cloud version. And so yeah, it's pretty easy to get started with this stuff. We find people start using it a lot of times during the validation phase 'cause that way they can start baselining performance models, they can do champion challenger, they can really kind of baseline the performance of, maybe they're considering different foundation models. And so it's a really helpful tool for understanding differences in the way these models perform. And then from there they can just flow that into their production inferencing, so that as these systems are out there, you have really kind of real time monitoring for anomalies and for all sorts of weird behaviors as well as that continuous feedback loop that helps you make make your product get better and observability and you can run all sorts of aggregated reports to really understand what's going on with these models when they're out there deciding. I should also add that we just today have another way to adopt Arthur and that is we are in the AWS marketplace, and so we are available there just to make it that much easier to use your cloud credits, skip the procurement process, and get up and running really quickly. >> And that's great 'cause Amazon's got SageMaker, which handles a lot of privacy stuff, all kinds of cool things, or you can get down and dirty. So I got to ask on the next one, production is a big deal, getting stuff into production. What have you guys learned that you could share to folks watching? Is there a cost issue? I got to monitor, obviously you brought that up, we talked about the even reinforcement issues, all these things are happening. What is the big learnings that you could share for people that are going to put these into production to watch out for, to plan for, or be prepared for, hope for the best plan for the worst? What's your advice? >> I can give a couple opinions there and I'm sure Adam has. Well, yeah, the big one from my side is, again, I had mentioned this earlier, it's just the input data streams because humans are also exploring how they can use these systems to begin with. It's really, really hard to predict the type of inputs you're going to be seeing in production. Especially, we always talk about chatbots, but then any generative text tasks like this, let's say you're taking in news articles and summarizing them or something like that, it's very hard to get a good sampling even of the set of news articles in such a way that you can really predict what's going to pop out of that model. So to me, it's, adversarial maybe isn't the word that I would use, but it's an unnatural shifting input distribution of like prompts that you might see for these models. That's certainly one. And then the second one that I would talk about is, it can be hard to understand the costs, the inference time costs behind these LLMs. So the pricing on these is always changing as the models change size, it might go up, it might go down based on model size, based on energy cost and so on, but your pricing per token or per a thousand tokens and that I think can be difficult for some clients to wrap their head around. Again, you don't know how these systems are going to be used after all so it can be tough. And so again that's another metric that really should be tracked. >> Yeah, and there's a lot of trade off choices in there with like, how many tokens do you want at each step and in the sequence and based on, you have (indistinct) and you reject these tokens and so based on how your system's operating, that can make the cost highly variable. And that's if you're using like an API version that you're paying per token. A lot of people also choose to run these internally and as John mentioned, the inference time on these is significantly higher than a traditional classifi, even NLP classification model or tabular data model, like orders of magnitude higher. And so you really need to understand how that, as you're constantly iterating on these models and putting out new versions and new features in these models, how that's affecting the overall scale of that inference cost because you can use a lot of computing power very quickly with these profits. >> Yeah, scale, performance, price all come together. I got to ask while we're here on the secret sauce of the company, if you had to describe to people out there watching, what's the secret sauce of the company? What's the key to your success? >> Yeah, so John leads our research team and they've had a number of really cool, I think AI as much as it's been hyped for a while, it's still commercial AI at least is really in its infancy. And so the way we're able to pioneer new ways to think about performance for computer vision NLP LLMs is probably the thing that I'm proudest about. John and his team publish papers all the time at Navs and other places. But I think it's really being able to define what performance means for basically any kind of model type and give people really powerful tools to understand that on an ongoing basis. >> John, secret sauce, how would you describe it? You got all the action happening all around you. >> Yeah, well I going to appreciate Adam talking me up like that. No, I. (all laughing) >> Furrier: Robs to you. >> I would also say a couple of other things here. So we have a very strong engineering team and so I think some early hires there really set the standard at a very high bar that we've maintained as we've grown. And I think that's really paid dividends as scalabilities become even more of a challenge in these spaces, right? And so that's not just scalability when it comes to LLMs, that's scalability when it comes to millions of inferences per day, that kind of thing as well in traditional ML models. And I think that's compared to potential competitors, that's really... Well, it's made us able to just operate more efficiently and pass that along to the client. >> Yeah, and I think the infancy comment is really important because it's the beginning. You really is a long journey ahead. A lot of change coming, like I said, it's a huge wave. So I'm sure you guys got a lot of plannings at the foundation even for your own company, so I appreciate the candid response there. Final question for you guys is, what should the top things be for a company in 2023? If I'm going to set the agenda and I'm a customer moving forward, putting the pedal to the metal, so to speak, what are the top things I should be prioritizing or I need to do to be successful with AI in 2023? >> Yeah, I think, so number one, as we talked about, we've been talking about this entire episode, the things are changing so quickly and the opportunities for business transformation and really disrupting different applications, different use cases, is almost, I don't think we've even fully comprehended how big it is. And so really digging in to your business and understanding where I can apply these new sets of foundation models is, that's a top priority. The interesting thing is I think there's another force at play, which is the macroeconomic conditions and a lot of places are, they're having to work harder to justify budgets. So in the past, couple years ago maybe, they had a blank check to spend on AI and AI development at a lot of large enterprises that was limited primarily by the amount of talent they could scoop up. Nowadays these expenditures are getting scrutinized more. And so one of the things that we really help our customers with is like really calculating the ROI on these things. And so if you have models out there performing and you have a new version that you can put out that lifts the performance by 3%, how many tens of millions of dollars does that mean in business benefit? Or if I want to go to get approval from the CFO to spend a few million dollars on this new project, how can I bake in from the beginning the tools to really show the ROI along the way? Because I think in these systems when done well for a software project, the ROI can be like pretty spectacular. Like we see over a hundred percent ROI in the first year on some of these projects. And so, I think in 2023, you just need to be able to show what you're getting for that spend. >> It's a needle moving moment. You see it all the time with some of these aha moments or like, whoa, blown away. John, I want to get your thoughts on this because one of the things that comes up a lot for companies that I talked to, that are on my second wave, I would say coming in, maybe not, maybe the front wave of adopters is talent and team building. You mentioned some of the hires you got were game changing for you guys and set the bar high. As you move the needle, new developers going to need to come in. What's your advice given that you've been a professor, you've seen students, I know a lot of computer science people want to shift, they might not be yet skilled in AI, but they're proficient in programming, is that's going to be another opportunity with open source when things are happening. How do you talk to that next level of talent that wants to come in to this market to supplement teams and be on teams, lead teams? Any advice you have for people who want to build their teams and people who are out there and want to be a coder in AI? >> Yeah, I've advice, and this actually works for what it would take to be a successful AI company in 2023 as well, which is, just don't be afraid to iterate really quickly with these tools. The space is still being explored on what they can be used for. A lot of the tasks that they're used for now right? like creating marketing content using a machine learning is not a new thing to do. It just works really well now. And so I'm excited to see what the next year brings in terms of folks from outside of core computer science who are, other engineers or physicists or chemists or whatever who are learning how to use these increasingly easy to use tools to leverage LLMs for tasks that I think none of us have really thought about before. So that's really, really exciting. And so toward that I would say iterate quickly. Build things on your own, build demos, show them the friends, host them online and you'll learn along the way and you'll have somebody to show for it. And also you'll help us explore that space. >> Guys, congratulations with Arthur. Great company, great picks and shovels opportunities out there for everybody. Iterate fast, get in quickly and don't be afraid to iterate. Great advice and thank you for coming on and being part of the AWS showcase, thanks. >> Yeah, thanks for having us on John. Always a pleasure. >> Yeah, great stuff. Adam Wenchel, John Dickerson with Arthur. Thanks for coming on theCUBE. I'm John Furrier, your host. Generative AI and AWS. Keep it right there for more action with theCUBE. Thanks for watching. (upbeat music)

(soft music) >> Hello, everyone. Welcome to theCUBE's presentation of the AWS Startup Showcase. AI and Machine Learning: Top Startups Building Foundational Model Infrastructure. This is season 3, episode 1 of the ongoing series covering the exciting stuff from the AWS ecosystem, talking about machine learning and AI. I'm your host, John Furrier and today we are excited to be joined by Luis Ceze who's the CEO of OctoML and Anna Connolly, VP of customer success and experience OctoML. Great to have you on again, Luis. Anna, thanks for coming on. Appreciate it. >> Thank you, John. It's great to be here. >> Thanks for having us. >> I love the company. We had a CUBE conversation about this. You guys are really addressing how to run foundational models faster for less. And this is like the key theme. But before we get into it, this is a hot trend, but let's explain what you guys do. Can you set the narrative of what the company's about, why it was founded, what's your North Star and your mission? >> Yeah, so John, our mission is to make AI sustainable and accessible for everyone. And what we offer customers is, you know, a way of taking their models into production in the most efficient way possible by automating the process of getting a model and optimizing it for a variety of hardware and making cost-effective. So better, faster, cheaper model deployment. >> You know, the big trend here is AI. Everyone's seeing the ChatGPT, kind of the shot heard around the world. The BingAI and this fiasco and the ongoing experimentation. People are into it, and I think the business impact is clear. I haven't seen this in all of my career in the technology industry of this kind of inflection point. And every senior leader I talk to is rethinking about how to rebuild their business with AI because now the large language models have come in, these foundational models are here, they can see value in their data. This is a 10 year journey in the big data world. Now it's impacting that, and everyone's rebuilding their company around this idea of being AI first 'cause they see ways to eliminate things and make things more efficient. And so now they telling 'em to go do it. And they're like, what do we do? So what do you guys think? Can you explain what is this wave of AI and why is it happening, why now, and what should people pay attention to? What does it mean to them? >> Yeah, I mean, it's pretty clear by now that AI can do amazing things that captures people's imaginations. And also now can show things that are really impactful in businesses, right? So what people have the opportunity to do today is to either train their own model that adds value to their business or find open models out there that can do very valuable things to them. So the next step really is how do you take that model and put it into production in a cost-effective way so that the business can actually get value out of it, right? >> Anna, what's your take? Because customers are there, you're there to make 'em successful, you got the new secret weapon for their business. >> Yeah, I think we just see a lot of companies struggle to get from a trained model into a model that is deployed in a cost-effective way that actually makes sense for the application they're building. I think that's a huge challenge we see today, kind of across the board across all of our customers. >> Well, I see this, everyone asking the same question. I have data, I want to get value out of it. I got to get these big models, I got to train it. What's it going to cost? So I think there's a reality of, okay, I got to do it. Then no one has any visibility on what it costs. When they get into it, this is going to break the bank. So I have to ask you guys, the cost of training these models is on everyone's mind. OctoML, your company's focus on the cost side of it as well as the efficiency side of running these models in production. Why are the production costs such a concern and where specifically are people looking at it and why did it get here? >> Yeah, so training costs get a lot of attention because normally a large number, but we shouldn't forget that it's a large, typically one time upfront cost that customers pay. But, you know, when the model is put into production, the cost grows directly with model usage and you actually want your model to be used because it's adding value, right? So, you know, the question that a customer faces is, you know, they have a model, they have a trained model and now what? So how much would it cost to run in production, right? And now without the big wave in generative AI, which rightfully is getting a lot of attention because of the amazing things that it can do. It's important for us to keep in mind that generative AI models like ChatGPT are huge, expensive energy hogs. They cost a lot to run, right? And given that model usage growth directly, model cost grows directly with usage, what you want to do is make sure that once you put a model into production, you have the best cost structure possible so that you're not surprised when it's gets popular, right? So let me give you an example. So if you have a model that costs, say 1 to $2 million to train, but then it costs about one to two cents per session to use it, right? So if you have a million active users, even if they use just once a day, it's 10 to $20,000 a day to operate that model in production. And that very, very quickly, you know, get beyond what you paid to train it. >> Anna, these aren't small numbers, and it's cost to train and cost to operate, it kind of reminds me of when the cloud came around and the data center versus cloud options. Like, wait a minute, one, it costs a ton of cash to deploy, and then running it. This is kind of a similar dynamic. What are you seeing? >> Yeah, absolutely. I think we are going to see increasingly the cost and production outpacing the costs and training by a lot. I mean, people talk about training costs now because that's what they're confronting now because people are so focused on getting models performant enough to even use in an application. And now that we have them and they're that capable, we're really going to start to see production costs go up a lot. >> Yeah, Luis, if you don't mind, I know this might be a little bit of a tangent, but, you know, training's super important. I get that. That's what people are doing now, but then there's the deployment side of production. Where do people get caught up and miss the boat or misconfigure? What's the gotcha? Where's the trip wire or so to speak? Where do people mess up on the cost side? What do they do? Is it they don't think about it, they tie it to proprietary hardware? What's the issue? >> Yeah, several things, right? So without getting really technical, which, you know, I might get into, you know, you have to understand relationship between performance, you know, both in terms of latency and throughput and cost, right? So reducing latency is important because you improve responsiveness of the model. But it's really important to keep in mind that it often leads diminishing returns. Below a certain latency, making it faster won't make a measurable difference in experience, but it's going to cost a lot more. So understanding that is important. Now, if you care more about throughputs, which is the time it takes for you to, you know, units per period of time, you care about time to solution, we should think about this throughput per dollar. And understand what you want is the highest throughput per dollar, which may come at the cost of higher latency, which you're not going to care about, right? So, and the reality here, John, is that, you know, humans and especially folks in this space want to have the latest and greatest hardware. And often they commit a lot of money to get access to them and have to commit upfront before they understand the needs that their models have, right? So common mistake here, one is not spending time to understand what you really need, and then two, over-committing and using more hardware than you actually need. And not giving yourself enough freedom to get your workload to move around to the more cost-effective choice, right? So this is just a metaphoric choice. And then another thing that's important here too is making a model run faster on the hardware directly translates to lower cost, right? So, but it takes a lot of engineers, you need to think of ways of producing very efficient versions of your model for the target hardware that you're going to use. >> Anna, what's the customer angle here? Because price performance has been around for a long time, people get that, but now latency and throughput, that's key because we're starting to see this in apps. I mean, there's an end user piece. I even seeing it on the infrastructure side where they're taking a heavy lifting away from operational costs. So you got, you know, application specific to the user and/or top of the stack, and then you got actually being used in operations where they want both. >> Yeah, absolutely. Maybe I can illustrate this with a quick story with the customer that we had recently been working with. So this customer is planning to run kind of a transformer based model for tech generation at super high scale on Nvidia T4 GPU, so kind of a commodity GPU. And the scale was so high that they would've been paying hundreds of thousands of dollars in cloud costs per year just to serve this model alone. You know, one of many models in their application stack. So we worked with this team to optimize our model and then benchmark across several possible targets. So that matching the hardware that Luis was just talking about, including the newer kind of Nvidia A10 GPUs. And what they found during this process was pretty interesting. First, the team was able to shave a quarter of their spend just by using better optimization techniques on the T4, the older hardware. But actually moving to a newer GPU would allow them to serve this model in a sub two milliseconds latency, so super fast, which was able to unlock an entirely new kind of user experience. So they were able to kind of change the value they're delivering in their application just because they were able to move to this new hardware easily. So they ultimately decided to plan their deployment on the more expensive A10 because of this, but because of the hardware specific optimizations that we helped them with, they managed to even, you know, bring costs down from what they had originally planned. And so if you extend this kind of example to everything that's happening with generative AI, I think the story we just talked about was super relevant, but the scale can be even higher, you know, it can be tenfold that. We were recently conducting kind of this internal study using GPT-J as a proxy to illustrate the experience of just a company trying to use one of these large language models with an example scenario of creating a chatbot to help job seekers prepare for interviews. So if you imagine kind of a conservative usage scenario where the model generates just 3000 words per user per day, which is, you know, pretty conservative for how people are interacting with these models. It costs 5 cents a session and if you're a company and your app goes viral, so from, you know, beginning of the year there's nobody, at the end of the year there's a million daily active active users in that year alone, going from zero to a million. You'll be spending about $6 million a year, which is pretty unmanageable. That's crazy, right? >> Yeah. >> For a company or a product that's just launching. So I think, you know, for us we see the real way to make these kind of advancements accessible and sustainable, as we said is to bring down cost to serve using these techniques. >> That's a great story and I think that illustrates this idea that deployment cost can vary from situation to situation, from model to model and that the efficiency is so strong with this new wave, it eliminates heavy lifting, creates more efficiency, automates intellect. I mean, this is the trend, this is radical, this is going to increase. So the cost could go from nominal to millions, literally, potentially. So, this is what customers are doing. Yeah, that's a great story. What makes sense on a financial, is there a cost of ownership? Is there a pattern for best practice for training? What do you guys advise cuz this is a lot of time and money involved in all potential, you know, good scenarios of upside. But you can get over your skis as they say, and be successful and be out of business if you don't manage it. I mean, that's what people are talking about, right? >> Yeah, absolutely. I think, you know, we see kind of three main vectors to reduce cost. I think one is make your deployment process easier overall, so that your engineering effort to even get your app running goes down. Two, would be get more from the compute you're already paying for, you're already paying, you know, for your instances in the cloud, but can you do more with that? And then three would be shop around for lower cost hardware to match your use case. So on the first one, I think making the deployment easier overall, there's a lot of manual work that goes into benchmarking, optimizing and packaging models for deployment. And because the performance of machine learning models can be really hardware dependent, you have to go through this process for each target you want to consider running your model on. And this is hard, you know, we see that every day. But for teams who want to incorporate some of these large language models into their applications, it might be desirable because licensing a model from a large vendor like OpenAI can leave you, you know, over provision, kind of paying for capabilities you don't need in your application or can lock you into them and you lose flexibility. So we have a customer whose team actually prepares models for deployment in a SaaS application that many of us use every day. And they told us recently that without kind of an automated benchmarking and experimentation platform, they were spending several days each to benchmark a single model on a single hardware type. So this is really, you know, manually intensive and then getting more from the compute you're already paying for. We do see customers who leave money on the table by running models that haven't been optimized specifically for the hardware target they're using, like Luis was mentioning. And for some teams they just don't have the time to go through an optimization process and for others they might lack kind of specialized expertise and this is something we can bring. And then on shopping around for different hardware types, we really see a huge variation in model performance across hardware, not just CPU vs. GPU, which is, you know, what people normally think of. But across CPU vendors themselves, high memory instances and across cloud providers even. So the best strategy here is for teams to really be able to, we say, look before you leap by running real world benchmarking and not just simulations or predictions to find the best software, hardware combination for their workload. >> Yeah. You guys sound like you have a very impressive customer base deploying large language models. Where would you categorize your current customer base? And as you look out, as you guys are growing, you have new customers coming in, take me through the progression. Take me through the profile of some of your customers you have now, size, are they hyperscalers, are they big app folks, are they kicking the tires? And then as people are out there scratching heads, I got to get in this game, what's their psychology like? Are they coming in with specific problems or do they have specific orientation point of view about what they want to do? Can you share some data around what you're seeing? >> Yeah, I think, you know, we have customers that kind of range across the spectrum of sophistication from teams that basically don't have MLOps expertise in their company at all. And so they're really looking for us to kind of give a full service, how should I do everything from, you know, optimization, find the hardware, prepare for deployment. And then we have teams that, you know, maybe already have their serving and hosting infrastructure up and ready and they already have models in production and they're really just looking to, you know, take the extra juice out of the hardware and just do really specific on that optimization piece. I think one place where we're doing a lot more work now is kind of in the developer tooling, you know, model selection space. And that's kind of an area that we're creating more tools for, particularly within the PyTorch ecosystem to bring kind of this power earlier in the development cycle so that as people are grabbing a model off the shelf, they can, you know, see how it might perform and use that to inform their development process. >> Luis, what's the big, I like this idea of picking the models because isn't that like going to the market and picking the best model for your data? It's like, you know, it's like, isn't there a certain approaches? What's your view on this? 'Cause this is where everyone, I think it's going to be a land rush for this and I want to get your thoughts. >> For sure, yeah. So, you know, I guess I'll start with saying the one main takeaway that we got from the GPT-J study is that, you know, having a different understanding of what your model's compute and memory requirements are, very quickly, early on helps with the much smarter AI model deployments, right? So, and in fact, you know, Anna just touched on this, but I want to, you know, make sure that it's clear that OctoML is putting that power into user's hands right now. So in partnership with AWS, we are launching this new PyTorch native profiler that allows you with a single, you know, one line, you know, code decorator allows you to see how your code runs on a variety of different hardware after accelerations. So it gives you very clear, you know, data on how you should think about your model deployments. And this ties back to choices of models. So like, if you have a set of choices that are equally good of models in terms of functionality and you want to understand after acceleration how are you going to deploy, how much they're going to cost or what are the options using a automated process of making a decision is really, really useful. And in fact, so I think these events can get early access to this by signing up for the Octopods, you know, this is exclusive group for insiders here, so you can go to OctoML.ai/pods to sign up. >> So that Octopod, is that a program? What is that, is that access to code? Is that a beta, what is that? Explain, take a minute and explain Octopod. >> I think the Octopod would be a group of people who is interested in experiencing this functionality. So it is the friends and users of OctoML that would be the Octopod. And then yes, after you sign up, we would provide you essentially the tool in code form for you to try out in your own. I mean, part of the benefit of this is that it happens in your own local environment and you're in control of everything kind of within the workflow that developers are already using to create and begin putting these models into their applications. So it would all be within your control. >> Got it. I think the big question I have for you is when do you, when does that one of your customers know they need to call you? What's their environment look like? What are they struggling with? What are the conversations they might be having on their side of the fence? If anyone's watching this, they're like, "Hey, you know what, I've got my team, we have a lot of data. Do we have our own language model or do I use someone else's?" There's a lot of this, I will say discovery going on around what to do, what path to take, what does that customer look like, if someone's listening, when do they know to call you guys, OctoML? >> Well, I mean the most obvious one is that you have a significant spend on AI/ML, come and talk to us, you know, putting AIML into production. So that's the clear one. In fact, just this morning I was talking to someone who is in life sciences space and is having, you know, 15 to $20 million a year cloud related to AI/ML deployment is a clear, it's a pretty clear match right there, right? So that's on the cost side. But I also want to emphasize something that Anna said earlier that, you know, the hardware and software complexity involved in putting model into production is really high. So we've been able to abstract that away, offering a clean automation flow enables one, to experiment early on, you know, how models would run and get them to production. And then two, once they are into production, gives you an automated flow to continuously updating your model and taking advantage of all this acceleration and ability to run the model on the right hardware. So anyways, let's say one then is cost, you know, you have significant cost and then two, you have an automation needs. And Anna please compliment that. >> Yeah, Anna you can please- >> Yeah, I think that's exactly right. Maybe the other time is when you are expecting a big scale up in serving your application, right? You're launching a new feature, you expect to get a lot of usage or, and you want to kind of anticipate maybe your CTO, your CIO, whoever pays your cloud bills is going to come after you, right? And so they want to know, you know, what's the return on putting this model essentially into my application stack? Am I going to, is the usage going to match what I'm paying for it? And then you can understand that. >> So you guys have a lot of the early adopters, they got big data teams, they're pushed in the production, they want to get a little QA, test the waters, understand, use your technology to figure it out. Is there any cases where people have gone into production, they have to pull it out? It's like the old lemon laws with your car, you buy a car and oh my god, it's not the way I wanted it. I mean, I can imagine the early people through the wall, so to speak, in the wave here are going to be bloody in the sense that they've gone in and tried stuff and get stuck with huge bills. Are you seeing that? Are people pulling stuff out of production and redeploying? Or I can imagine that if I had a bad deployment, I'd want to refactor that or actually replatform that. Do you see that too? >> Definitely after a sticker shock, yes, your customers will come and make sure that, you know, the sticker shock won't happen again. >> Yeah. >> But then there's another more thorough aspect here that I think we likely touched on, be worth elaborating a bit more is just how are you going to scale in a way that's feasible depending on the allocation that you get, right? So as we mentioned several times here, you know, model deployment is so hardware dependent and so complex that you tend to get a model for a hardware choice and then you want to scale that specific type of instance. But what if, when you want to scale because suddenly luckily got popular and, you know, you want to scale it up and then you don't have that instance anymore. So how do you live with whatever you have at that moment is something that we see customers needing as well. You know, so in fact, ideally what we want is customers to not think about what kind of specific instances they want. What they want is to know what their models need. Say, they know the SLA and then find a set of hybrid targets and instances that hit the SLA whenever they're also scaling, they're going to scale with more freedom, right? Instead of having to wait for AWS to give them more specific allocation for a specific instance. What if you could live with other types of hardware and scale up in a more free way, right? So that's another thing that we see customers, you know, like they need more freedom to be able to scale with whatever is available. >> Anna, you touched on this with the business model impact to that 6 million cost, if that goes out of control, there's a business model aspect and there's a technical operation aspect to the cost side too. You want to be mindful of riding the wave in a good way, but not getting over your skis. So that brings up the point around, you know, confidence, right? And teamwork. Because if you're in production, there's probably a team behind it. Talk about the team aspect of your customers. I mean, they're dedicated, they go put stuff into production, they're developers, there're data. What's in it for them? Are they getting better, are they in the beach, you know, reading the book. Are they, you know, are there easy street for them? What's the customer benefit to the teams? >> Yeah, absolutely. With just a few clicks of a button, you're in production, right? That's the dream. So yeah, I mean I think that, you know, we illustrated it before a little bit. I think the automated kind of benchmarking and optimization process, like when you think about the effort it takes to get that data by hand, which is what people are doing today, they just don't do it. So they're making decisions without the best information because it's, you know, there just isn't the bandwidth to get the information that they need to make the best decision and then know exactly how to deploy it. So I think it's actually bringing kind of a new insight and capability to these teams that they didn't have before. And then maybe another aspect on the team side is that it's making the hand-off of the models from the data science teams to the model deployment teams more seamless. So we have, you know, we have seen in the past that this kind of transition point is the place where there are a lot of hiccups, right? The data science team will give a model to the production team and it'll be too slow for the application or it'll be too expensive to run and it has to go back and be changed and kind of this loop. And so, you know, with the PyTorch profiler that Luis was talking about, and then also, you know, the other ways we do optimization that kind of prevents that hand-off problem from happening. >> Luis and Anna, you guys have a great company. Final couple minutes left. Talk about the company, the people there, what's the culture like, you know, if Intel has Moore's law, which is, you know, doubling the performance in few years, what's the culture like there? Is it, you know, more throughput, better pricing? Explain what's going on with the company and put a plug in. Luis, we'll start with you. >> Yeah, absolutely. I'm extremely proud of the team that we built here. You know, we have a people first culture, you know, very, very collaborative and folks, we all have a shared mission here of making AI more accessible and sustainable. We have a very diverse team in terms of backgrounds and life stories, you know, to do what we do here, we need a team that has expertise in software engineering, in machine learning, in computer architecture. Even though we don't build chips, we need to understand how they work, right? So, and then, you know, the fact that we have this, this very really, really varied set of backgrounds makes the environment, you know, it's say very exciting to learn more about, you know, assistance end-to-end. But also makes it for a very interesting, you know, work environment, right? So people have different backgrounds, different stories. Some of them went to grad school, others, you know, were in intelligence agencies and now are working here, you know. So we have a really interesting set of people and, you know, life is too short not to work with interesting humans. You know, that's something that I like to think about, you know. >> I'm sure your off-site meetings are a lot of fun, people talking about computer architectures, silicon advances, the next GPU, the big data models coming in. Anna, what's your take? What's the culture like? What's the company vibe and what are you guys looking to do? What's the customer success pattern? What's up? >> Yeah, absolutely. I mean, I, you know, second all of the great things that Luis just said about the team. I think one that I, an additional one that I'd really like to underscore is kind of this customer obsession, to use a term you all know well. And focus on the end users and really making the experiences that we're bringing to our user who are developers really, you know, useful and valuable for them. And so I think, you know, all of these tools that we're trying to put in the hands of users, the industry and the market is changing so rapidly that our products across the board, you know, all of the companies that, you know, are part of the showcase today, we're all evolving them so quickly and we can only do that kind of really hand in glove with our users. So that would be another thing I'd emphasize. >> I think the change dynamic, the power dynamics of this industry is just the beginning. I'm very bullish that this is going to be probably one of the biggest inflection points in history of the computer industry because of all the dynamics of the confluence of all the forces, which you mentioned some of them, I mean PC, you know, interoperability within internetworking and you got, you know, the web and then mobile. Now we have this, I mean, I wouldn't even put social media even in the close to this. Like, this is like, changes user experience, changes infrastructure. There's going to be massive accelerations in performance on the hardware side from AWS's of the world and cloud and you got the edge and more data. This is really what big data was going to look like. This is the beginning. Final question, what do you guys see going forward in the future? >> Well, it's undeniable that machine learning and AI models are becoming an integral part of an interesting application today, right? So, and the clear trends here are, you know, more and more competitional needs for these models because they're only getting more and more powerful. And then two, you know, seeing the complexity of the infrastructure where they run, you know, just considering the cloud, there's like a wide variety of choices there, right? So being able to live with that and making the most out of it in a way that does not require, you know, an impossible to find team is something that's pretty clear. So the need for automation, abstracting with the complexity is definitely here. And we are seeing this, you know, trends are that you also see models starting to move to the edge as well. So it's clear that we're seeing, we are going to live in a world where there's no large models living in the cloud. And then, you know, edge models that talk to these models in the cloud to form, you know, an end-to-end truly intelligent application. >> Anna? >> Yeah, I think, you know, our, Luis said it at the beginning. Our vision is to make AI sustainable and accessible. And I think as this technology just expands in every company and every team, that's going to happen kind of on its own. And we're here to help support that. And I think you can't do that without tools like those like OctoML. >> I think it's going to be an error of massive invention, creativity, a lot of the format heavy lifting is going to allow the talented people to automate their intellect. I mean, this is really kind of what we see going on. And Luis, thank you so much. Anna, thanks for coming on this segment. Thanks for coming on theCUBE and being part of the AWS Startup Showcase. I'm John Furrier, your host. Thanks for watching. (upbeat music)

(upbeat music) >> Hello everyone. Welcome to theCube's presentation of the "AWS Startup Showcase." The topic this episode is AI and machine learning, top startups building foundational model infrastructure. This is season three, episode one of the ongoing series covering exciting startups from the AWS ecosystem. And this time we're talking about AI and machine learning. I'm your host, John Furrier. I'm excited I'm joined today by Robert Nishihara, who's the co-founder and CEO of a hot startup called Anyscale. He's here to talk about Ray, the open source project, Anyscale's infrastructure for foundation as well. Robert, thank you for joining us today. >> Yeah, thanks so much as well. >> I've been following your company since the founding pre pandemic and you guys really had a great vision scaled up and in a perfect position for this big wave that we all see with ChatGPT and OpenAI that's gone mainstream. Finally, AI has broken out through the ropes and now gone mainstream, so I think you guys are really well positioned. I'm looking forward to to talking with you today. But before we get into it, introduce the core mission for Anyscale. Why do you guys exist? What is the North Star for Anyscale? >> Yeah, like you mentioned, there's a tremendous amount of excitement about AI right now. You know, I think a lot of us believe that AI can transform just every different industry. So one of the things that was clear to us when we started this company was that the amount of compute needed to do AI was just exploding. Like to actually succeed with AI, companies like OpenAI or Google or you know, these companies getting a lot of value from AI, were not just running these machine learning models on their laptops or on a single machine. They were scaling these applications across hundreds or thousands or more machines and GPUs and other resources in the Cloud. And so to actually succeed with AI, and this has been one of the biggest trends in computing, maybe the biggest trend in computing in, you know, in recent history, the amount of compute has been exploding. And so to actually succeed with that AI, to actually build these scalable applications and scale the AI applications, there's a tremendous software engineering lift to build the infrastructure to actually run these scalable applications. And that's very hard to do. So one of the reasons many AI projects and initiatives fail is that, or don't make it to production, is the need for this scale, the infrastructure lift, to actually make it happen. So our goal here with Anyscale and Ray, is to make that easy, is to make scalable computing easy. So that as a developer or as a business, if you want to do AI, if you want to get value out of AI, all you need to know is how to program on your laptop. Like, all you need to know is how to program in Python. And if you can do that, then you're good to go. Then you can do what companies like OpenAI or Google do and get value out of machine learning. >> That programming example of how easy it is with Python reminds me of the early days of Cloud, when infrastructure as code was talked about was, it was just code the infrastructure programmable. That's super important. That's what AI people wanted, first program AI. That's the new trend. And I want to understand, if you don't mind explaining, the relationship that Anyscale has to these foundational models and particular the large language models, also called LLMs, was seen with like OpenAI and ChatGPT. Before you get into the relationship that you have with them, can you explain why the hype around foundational models? Why are people going crazy over foundational models? What is it and why is it so important? >> Yeah, so foundational models and foundation models are incredibly important because they enable businesses and developers to get value out of machine learning, to use machine learning off the shelf with these large models that have been trained on tons of data and that are useful out of the box. And then, of course, you know, as a business or as a developer, you can take those foundational models and repurpose them or fine tune them or adapt them to your specific use case and what you want to achieve. But it's much easier to do that than to train them from scratch. And I think there are three, for people to actually use foundation models, there are three main types of workloads or problems that need to be solved. One is training these foundation models in the first place, like actually creating them. The second is fine tuning them and adapting them to your use case. And the third is serving them and actually deploying them. Okay, so Ray and Anyscale are used for all of these three different workloads. Companies like OpenAI or Cohere that train large language models. Or open source versions like GPTJ are done on top of Ray. There are many startups and other businesses that fine tune, that, you know, don't want to train the large underlying foundation models, but that do want to fine tune them, do want to adapt them to their purposes, and build products around them and serve them, those are also using Ray and Anyscale for that fine tuning and that serving. And so the reason that Ray and Anyscale are important here is that, you know, building and using foundation models requires a huge scale. It requires a lot of data. It requires a lot of compute, GPUs, TPUs, other resources. And to actually take advantage of that and actually build these scalable applications, there's a lot of infrastructure that needs to happen under the hood. And so you can either use Ray and Anyscale to take care of that and manage the infrastructure and solve those infrastructure problems. Or you can build the infrastructure and manage the infrastructure yourself, which you can do, but it's going to slow your team down. It's going to, you know, many of the businesses we work with simply don't want to be in the business of managing infrastructure and building infrastructure. They want to focus on product development and move faster. >> I know you got a keynote presentation we're going to go to in a second, but I think you hit on something I think is the real tipping point, doing it yourself, hard to do. These are things where opportunities are and the Cloud did that with data centers. Turned a data center and made it an API. The heavy lifting went away and went to the Cloud so people could be more creative and build their product. In this case, build their creativity. Is that kind of what's the big deal? Is that kind of a big deal happening that you guys are taking the learnings and making that available so people don't have to do that? >> That's exactly right. So today, if you want to succeed with AI, if you want to use AI in your business, infrastructure work is on the critical path for doing that. To do AI, you have to build infrastructure. You have to figure out how to scale your applications. That's going to change. We're going to get to the point, and you know, with Ray and Anyscale, we're going to remove the infrastructure from the critical path so that as a developer or as a business, all you need to focus on is your application logic, what you want the the program to do, what you want your application to do, how you want the AI to actually interface with the rest of your product. Now the way that will happen is that Ray and Anyscale will still, the infrastructure work will still happen. It'll just be under the hood and taken care of by Ray in Anyscale. And so I think something like this is really necessary for AI to reach its potential, for AI to have the impact and the reach that we think it will, you have to make it easier to do. >> And just for clarification to point out, if you don't mind explaining the relationship of Ray and Anyscale real quick just before we get into the presentation. >> So Ray is an open source project. We created it. We were at Berkeley doing machine learning. We started Ray so that, in order to provide an easy, a simple open source tool for building and running scalable applications. And Anyscale is the managed version of Ray, basically we will run Ray for you in the Cloud, provide a lot of tools around the developer experience and managing the infrastructure and providing more performance and superior infrastructure. >> Awesome. I know you got a presentation on Ray and Anyscale and you guys are positioning as the infrastructure for foundational models. So I'll let you take it away and then when you're done presenting, we'll come back, I'll probably grill you with a few questions and then we'll close it out so take it away. >> Robert: Sounds great. So I'll say a little bit about how companies are using Ray and Anyscale for foundation models. The first thing I want to mention is just why we're doing this in the first place. And the underlying observation, the underlying trend here, and this is a plot from OpenAI, is that the amount of compute needed to do machine learning has been exploding. It's been growing at something like 35 times every 18 months. This is absolutely enormous. And other people have written papers measuring this trend and you get different numbers. But the point is, no matter how you slice and dice it, it' a astronomical rate. Now if you compare that to something we're all familiar with, like Moore's Law, which says that, you know, the processor performance doubles every roughly 18 months, you can see that there's just a tremendous gap between the needs, the compute needs of machine learning applications, and what you can do with a single chip, right. So even if Moore's Law were continuing strong and you know, doing what it used to be doing, even if that were the case, there would still be a tremendous gap between what you can do with the chip and what you need in order to do machine learning. And so given this graph, what we've seen, and what has been clear to us since we started this company, is that doing AI requires scaling. There's no way around it. It's not a nice to have, it's really a requirement. And so that led us to start Ray, which is the open source project that we started to make it easy to build these scalable Python applications and scalable machine learning applications. And since we started the project, it's been adopted by a tremendous number of companies. Companies like OpenAI, which use Ray to train their large models like ChatGPT, companies like Uber, which run all of their deep learning and classical machine learning on top of Ray, companies like Shopify or Spotify or Instacart or Lyft or Netflix, ByteDance, which use Ray for their machine learning infrastructure. Companies like Ant Group, which makes Alipay, you know, they use Ray across the board for fraud detection, for online learning, for detecting money laundering, you know, for graph processing, stream processing. Companies like Amazon, you know, run Ray at a tremendous scale and just petabytes of data every single day. And so the project has seen just enormous adoption since, over the past few years. And one of the most exciting use cases is really providing the infrastructure for building training, fine tuning, and serving foundation models. So I'll say a little bit about, you know, here are some examples of companies using Ray for foundation models. Cohere trains large language models. OpenAI also trains large language models. You can think about the workloads required there are things like supervised pre-training, also reinforcement learning from human feedback. So this is not only the regular supervised learning, but actually more complex reinforcement learning workloads that take human input about what response to a particular question, you know is better than a certain other response. And incorporating that into the learning. There's open source versions as well, like GPTJ also built on top of Ray as well as projects like Alpa coming out of UC Berkeley. So these are some of the examples of exciting projects in organizations, training and creating these large language models and serving them using Ray. Okay, so what actually is Ray? Well, there are two layers to Ray. At the lowest level, there's the core Ray system. This is essentially low level primitives for building scalable Python applications. Things like taking a Python function or a Python class and executing them in the cluster setting. So Ray core is extremely flexible and you can build arbitrary scalable applications on top of Ray. So on top of Ray, on top of the core system, what really gives Ray a lot of its power is this ecosystem of scalable libraries. So on top of the core system you have libraries, scalable libraries for ingesting and pre-processing data, for training your models, for fine tuning those models, for hyper parameter tuning, for doing batch processing and batch inference, for doing model serving and deployment, right. And a lot of the Ray users, the reason they like Ray is that they want to run multiple workloads. They want to train and serve their models, right. They want to load their data and feed that into training. And Ray provides common infrastructure for all of these different workloads. So this is a little overview of what Ray, the different components of Ray. So why do people choose to go with Ray? I think there are three main reasons. The first is the unified nature. The fact that it is common infrastructure for scaling arbitrary workloads, from data ingest to pre-processing to training to inference and serving, right. This also includes the fact that it's future proof. AI is incredibly fast moving. And so many people, many companies that have built their own machine learning infrastructure and standardized on particular workflows for doing machine learning have found that their workflows are too rigid to enable new capabilities. If they want to do reinforcement learning, if they want to use graph neural networks, they don't have a way of doing that with their standard tooling. And so Ray, being future proof and being flexible and general gives them that ability. Another reason people choose Ray in Anyscale is the scalability. This is really our bread and butter. This is the reason, the whole point of Ray, you know, making it easy to go from your laptop to running on thousands of GPUs, making it easy to scale your development workloads and run them in production, making it easy to scale, you know, training to scale data ingest, pre-processing and so on. So scalability and performance, you know, are critical for doing machine learning and that is something that Ray provides out of the box. And lastly, Ray is an open ecosystem. You can run it anywhere. You can run it on any Cloud provider. Google, you know, Google Cloud, AWS, Asure. You can run it on your Kubernetes cluster. You can run it on your laptop. It's extremely portable. And not only that, it's framework agnostic. You can use Ray to scale arbitrary Python workloads. You can use it to scale and it integrates with libraries like TensorFlow or PyTorch or JAX or XG Boost or Hugging Face or PyTorch Lightning, right, or Scikit-learn or just your own arbitrary Python code. It's open source. And in addition to integrating with the rest of the machine learning ecosystem and these machine learning frameworks, you can use Ray along with all of the other tooling in the machine learning ecosystem. That's things like weights and biases or ML flow, right. Or you know, different data platforms like Databricks, you know, Delta Lake or Snowflake or tools for model monitoring for feature stores, all of these integrate with Ray. And that's, you know, Ray provides that kind of flexibility so that you can integrate it into the rest of your workflow. And then Anyscale is the scalable compute platform that's built on top, you know, that provides Ray. So Anyscale is a managed Ray service that runs in the Cloud. And what Anyscale does is it offers the best way to run Ray. And if you think about what you get with Anyscale, there are fundamentally two things. One is about moving faster, accelerating the time to market. And you get that by having the managed service so that as a developer you don't have to worry about managing infrastructure, you don't have to worry about configuring infrastructure. You also, it provides, you know, optimized developer workflows. Things like easily moving from development to production, things like having the observability tooling, the debug ability to actually easily diagnose what's going wrong in a distributed application. So things like the dashboards and the other other kinds of tooling for collaboration, for monitoring and so on. And then on top of that, so that's the first bucket, developer productivity, moving faster, faster experimentation and iteration. The second reason that people choose Anyscale is superior infrastructure. So this is things like, you know, cost deficiency, being able to easily take advantage of spot instances, being able to get higher GPU utilization, things like faster cluster startup times and auto scaling. Things like just overall better performance and faster scheduling. And so these are the kinds of things that Anyscale provides on top of Ray. It's the managed infrastructure. It's fast, it's like the developer productivity and velocity as well as performance. So this is what I wanted to share about Ray in Anyscale. >> John: Awesome. >> Provide that context. But John, I'm curious what you think. >> I love it. I love the, so first of all, it's a platform because that's the platform architecture right there. So just to clarify, this is an Anyscale platform, not- >> That's right. >> Tools. So you got tools in the platform. Okay, that's key. Love that managed service. Just curious, you mentioned Python multiple times, is that because of PyTorch and TensorFlow or Python's the most friendly with machine learning or it's because it's very common amongst all developers? >> That's a great question. Python is the language that people are using to do machine learning. So it's the natural starting point. Now, of course, Ray is actually designed in a language agnostic way and there are companies out there that use Ray to build scalable Java applications. But for the most part right now we're focused on Python and being the best way to build these scalable Python and machine learning applications. But, of course, down the road there always is that potential. >> So if you're slinging Python code out there and you're watching that, you're watching this video, get on Anyscale bus quickly. Also, I just, while you were giving the presentation, I couldn't help, since you mentioned OpenAI, which by the way, congratulations 'cause they've had great scale, I've noticed in their rapid growth 'cause they were the fastest company to the number of users than anyone in the history of the computer industry, so major successor, OpenAI and ChatGPT, huge fan. I'm not a skeptic at all. I think it's just the beginning, so congratulations. But I actually typed into ChatGPT, what are the top three benefits of Anyscale and came up with scalability, flexibility, and ease of use. Obviously, scalability is what you guys are called. >> That's pretty good. >> So that's what they came up with. So they nailed it. Did you have an inside prompt training, buy it there? Only kidding. (Robert laughs) >> Yeah, we hard coded that one. >> But that's the kind of thing that came up really, really quickly if I asked it to write a sales document, it probably will, but this is the future interface. This is why people are getting excited about the foundational models and the large language models because it's allowing the interface with the user, the consumer, to be more human, more natural. And this is clearly will be in every application in the future. >> Absolutely. This is how people are going to interface with software, how they're going to interface with products in the future. It's not just something, you know, not just a chat bot that you talk to. This is going to be how you get things done, right. How you use your web browser or how you use, you know, how you use Photoshop or how you use other products. Like you're not going to spend hours learning all the APIs and how to use them. You're going to talk to it and tell it what you want it to do. And of course, you know, if it doesn't understand it, it's going to ask clarifying questions. You're going to have a conversation and then it'll figure it out. >> This is going to be one of those things, we're going to look back at this time Robert and saying, "Yeah, from that company, that was the beginning of that wave." And just like AWS and Cloud Computing, the folks who got in early really were in position when say the pandemic came. So getting in early is a good thing and that's what everyone's talking about is getting in early and playing around, maybe replatforming or even picking one or few apps to refactor with some staff and managed services. So people are definitely jumping in. So I have to ask you the ROI cost question. You mentioned some of those, Moore's Law versus what's going on in the industry. When you look at that kind of scale, the first thing that jumps out at people is, "Okay, I love it. Let's go play around." But what's it going to cost me? Am I going to be tied to certain GPUs? What's the landscape look like from an operational standpoint, from the customer? Are they locked in and the benefit was flexibility, are you flexible to handle any Cloud? What is the customers, what are they looking at? Basically, that's my question. What's the customer looking at? >> Cost is super important here and many of the companies, I mean, companies are spending a huge amount on their Cloud computing, on AWS, and on doing AI, right. And I think a lot of the advantage of Anyscale, what we can provide here is not only better performance, but cost efficiency. Because if we can run something faster and more efficiently, it can also use less resources and you can lower your Cloud spending, right. We've seen companies go from, you know, 20% GPU utilization with their current setup and the current tools they're using to running on Anyscale and getting more like 95, you know, 100% GPU utilization. That's something like a five x improvement right there. So depending on the kind of application you're running, you know, it's a significant cost savings. We've seen companies that have, you know, processing petabytes of data every single day with Ray going from, you know, getting order of magnitude cost savings by switching from what they were previously doing to running their application on Ray. And when you have applications that are spending, you know, potentially $100 million a year and getting a 10 X cost savings is just absolutely enormous. So these are some of the kinds of- >> Data infrastructure is super important. Again, if the customer, if you're a prospect to this and thinking about going in here, just like the Cloud, you got infrastructure, you got the platform, you got SaaS, same kind of thing's going to go on in AI. So I want to get into that, you know, ROI discussion and some of the impact with your customers that are leveraging the platform. But first I hear you got a demo. >> Robert: Yeah, so let me show you, let me give you a quick run through here. So what I have open here is the Anyscale UI. I've started a little Anyscale Workspace. So Workspaces are the Anyscale concept for interactive developments, right. So here, imagine I'm just, you want to have a familiar experience like you're developing on your laptop. And here I have a terminal. It's not on my laptop. It's actually in the cloud running on Anyscale. And I'm just going to kick this off. This is going to train a large language model, so OPT. And it's doing this on 32 GPUs. We've got a cluster here with a bunch of CPU cores, bunch of memory. And as that's running, and by the way, if I wanted to run this on instead of 32 GPUs, 64, 128, this is just a one line change when I launch the Workspace. And what I can do is I can pull up VS code, right. Remember this is the interactive development experience. I can look at the actual code. Here it's using Ray train to train the torch model. We've got the training loop and we're saying that each worker gets access to one GPU and four CPU cores. And, of course, as I make the model larger, this is using deep speed, as I make the model larger, I could increase the number of GPUs that each worker gets access to, right. And how that is distributed across the cluster. And if I wanted to run on CPUs instead of GPUs or a different, you know, accelerator type, again, this is just a one line change. And here we're using Ray train to train the models, just taking my vanilla PyTorch model using Hugging Face and then scaling that across a bunch of GPUs. And, of course, if I want to look at the dashboard, I can go to the Ray dashboard. There are a bunch of different visualizations I can look at. I can look at the GPU utilization. I can look at, you know, the CPU utilization here where I think we're currently loading the model and running that actual application to start the training. And some of the things that are really convenient here about Anyscale, both I can get that interactive development experience with VS code. You know, I can look at the dashboards. I can monitor what's going on. It feels, I have a terminal, it feels like my laptop, but it's actually running on a large cluster. And I can, with however many GPUs or other resources that I want. And so it's really trying to combine the best of having the familiar experience of programming on your laptop, but with the benefits, you know, being able to take advantage of all the resources in the Cloud to scale. And it's like when, you know, you're talking about cost efficiency. One of the biggest reasons that people waste money, one of the silly reasons for wasting money is just forgetting to turn off your GPUs. And what you can do here is, of course, things will auto terminate if they're idle. But imagine you go to sleep, I have this big cluster. You can turn it off, shut off the cluster, come back tomorrow, restart the Workspace, and you know, your big cluster is back up and all of your code changes are still there. All of your local file edits. It's like you just closed your laptop and came back and opened it up again. And so this is the kind of experience we want to provide for our users. So that's what I wanted to share with you. >> Well, I think that whole, couple of things, lines of code change, single line of code change, that's game changing. And then the cost thing, I mean human error is a big deal. People pass out at their computer. They've been coding all night or they just forget about it. I mean, and then it's just like leaving the lights on or your water running in your house. It's just, at the scale that it is, the numbers will add up. That's a huge deal. So I think, you know, compute back in the old days, there's no compute. Okay, it's just compute sitting there idle. But you know, data cranking the models is doing, that's a big point. >> Another thing I want to add there about cost efficiency is that we make it really easy to use, if you're running on Anyscale, to use spot instances and these preemptable instances that can just be significantly cheaper than the on-demand instances. And so when we see our customers go from what they're doing before to using Anyscale and they go from not using these spot instances 'cause they don't have the infrastructure around it, the fault tolerance to handle the preemption and things like that, to being able to just check a box and use spot instances and save a bunch of money. >> You know, this was my whole, my feature article at Reinvent last year when I met with Adam Selipsky, this next gen Cloud is here. I mean, it's not auto scale, it's infrastructure scale. It's agility. It's flexibility. I think this is where the world needs to go. Almost what DevOps did for Cloud and what you were showing me that demo had this whole SRE vibe. And remember Google had site reliability engines to manage all those servers. This is kind of like an SRE vibe for data at scale. I mean, a similar kind of order of magnitude. I mean, I might be a little bit off base there, but how would you explain it? >> It's a nice analogy. I mean, what we are trying to do here is get to the point where developers don't think about infrastructure. Where developers only think about their application logic. And where businesses can do AI, can succeed with AI, and build these scalable applications, but they don't have to build, you know, an infrastructure team. They don't have to develop that expertise. They don't have to invest years in building their internal machine learning infrastructure. They can just focus on the Python code, on their application logic, and run the stuff out of the box. >> Awesome. Well, I appreciate the time. Before we wrap up here, give a plug for the company. I know you got a couple websites. Again, go, Ray's got its own website. You got Anyscale. You got an event coming up. Give a plug for the company looking to hire. Put a plug in for the company. >> Yeah, absolutely. Thank you. So first of all, you know, we think AI is really going to transform every industry and the opportunity is there, right. We can be the infrastructure that enables all of that to happen, that makes it easy for companies to succeed with AI, and get value out of AI. Now we have, if you're interested in learning more about Ray, Ray has been emerging as the standard way to build scalable applications. Our adoption has been exploding. I mentioned companies like OpenAI using Ray to train their models. But really across the board companies like Netflix and Cruise and Instacart and Lyft and Uber, you know, just among tech companies. It's across every industry. You know, gaming companies, agriculture, you know, farming, robotics, drug discovery, you know, FinTech, we see it across the board. And all of these companies can get value out of AI, can really use AI to improve their businesses. So if you're interested in learning more about Ray and Anyscale, we have our Ray Summit coming up in September. This is going to highlight a lot of the most impressive use cases and stories across the industry. And if your business, if you want to use LLMs, you want to train these LLMs, these large language models, you want to fine tune them with your data, you want to deploy them, serve them, and build applications and products around them, give us a call, talk to us. You know, we can really take the infrastructure piece, you know, off the critical path and make that easy for you. So that's what I would say. And, you know, like you mentioned, we're hiring across the board, you know, engineering, product, go-to-market, and it's an exciting time. >> Robert Nishihara, co-founder and CEO of Anyscale, congratulations on a great company you've built and continuing to iterate on and you got growth ahead of you, you got a tailwind. I mean, the AI wave is here. I think OpenAI and ChatGPT, a customer of yours, have really opened up the mainstream visibility into this new generation of applications, user interface, roll of data, large scale, how to make that programmable so we're going to need that infrastructure. So thanks for coming on this season three, episode one of the ongoing series of the hot startups. In this case, this episode is the top startups building foundational model infrastructure for AI and ML. I'm John Furrier, your host. Thanks for watching. (upbeat music)

(light airy music) >> Hello, everyone, welcome to theCUBE's presentation of the AWS Startup Showcase, AI and machine learning. "Top Startups Building Generative AI on AWS." This is season three, episode one of the ongoing series covering the exciting startups from the AWS ecosystem, talking about AI machine learning. We have three great guests Bratin Saha, VP, Vice President of Machine Learning and AI Services at Amazon Web Services. Tom Mason, the CTO of Stability AI, and Aidan Gomez, CEO and co-founder of Cohere. Two practitioners doing startups and AWS. Gentlemen, thank you for opening up this session, this episode. Thanks for coming on. >> Thank you. >> Thank you. >> Thank you. >> So the topic is hype versus reality. So I think we're all on the reality is great, hype is great, but the reality's here. I want to get into it. Generative AI's got all the momentum, it's going mainstream, it's kind of come out of the behind the ropes, it's now mainstream. We saw the success of ChatGPT, opens up everyone's eyes, but there's so much more going on. Let's jump in and get your early perspectives on what should people be talking about right now? What are you guys working on? We'll start with AWS. What's the big focus right now for you guys as you come into this market that's highly active, highly hyped up, but people see value right out of the gate? >> You know, we have been working on generative AI for some time. In fact, last year we released Code Whisperer, which is about using generative AI for software development and a number of customers are using it and getting real value out of it. So generative AI is now something that's mainstream that can be used by enterprise users. And we have also been partnering with a number of other companies. So, you know, stability.ai, we've been partnering with them a lot. We want to be partnering with other companies as well. In seeing how we do three things, you know, first is providing the most efficient infrastructure for generative AI. And that is where, you know, things like Trainium, things like Inferentia, things like SageMaker come in. And then next is the set of models and then the third is the kind of applications like Code Whisperer and so on. So, you know, it's early days yet, but clearly there's a lot of amazing capabilities that will come out and something that, you know, our customers are starting to pay a lot of attention to. >> Tom, talk about your company and what your focus is and why the Amazon Web Services relationship's important for you? >> So yeah, we're primarily committed to making incredible open source foundation models and obviously stable effusions been our kind of first big model there, which we trained all on AWS. We've been working with them over the last year and a half to develop, obviously a big cluster, and bring all that compute to training these models at scale, which has been a really successful partnership. And we're excited to take it further this year as we develop commercial strategy of the business and build out, you know, the ability for enterprise customers to come and get all the value from these models that we think they can get. So we're really excited about the future. We got hugely exciting pipeline for this year with new modalities and video models and wonderful things and trying to solve images for once and for all and get the kind of general value and value proposition correct for customers. So it's a really exciting time and very honored to be part of it. >> It's great to see some of your customers doing so well out there. Congratulations to your team. Appreciate that. Aidan, let's get into what you guys do. What does Cohere do? What are you excited about right now? >> Yeah, so Cohere builds large language models, which are the backbone of applications like ChatGPT and GPT-3. We're extremely focused on solving the issues with adoption for enterprise. So it's great that you can make a super flashy demo for consumers, but it takes a lot to actually get it into billion user products and large global enterprises. So about six months ago, we released our command models, which are some of the best that exist for large language models. And in December, we released our multilingual text understanding models and that's on over a hundred different languages and it's trained on, you know, authentic data directly from native speakers. And so we're super excited to continue pushing this into enterprise and solving those barriers for adoption, making this transformation a reality. >> Just real quick, while I got you there on the new products coming out. Where are we in the progress? People see some of the new stuff out there right now. There's so much more headroom. Can you just scope out in your mind what that looks like? Like from a headroom standpoint? Okay, we see ChatGPT. "Oh yeah, it writes my papers for me, does some homework for me." I mean okay, yawn, maybe people say that, (Aidan chuckles) people excited or people are blown away. I mean, it's helped theCUBE out, it helps me, you know, feed up a little bit from my write-ups but it's not always perfect. >> Yeah, at the moment it's like a writing assistant, right? And it's still super early in the technologies trajectory. I think it's fascinating and it's interesting but its impact is still really limited. I think in the next year, like within the next eight months, we're going to see some major changes. You've already seen the very first hints of that with stuff like Bing Chat, where you augment these dialogue models with an external knowledge base. So now the models can be kept up to date to the millisecond, right? Because they can search the web and they can see events that happened a millisecond ago. But that's still limited in the sense that when you ask the question, what can these models actually do? Well they can just write text back at you. That's the extent of what they can do. And so the real project, the real effort, that I think we're all working towards is actually taking action. So what happens when you give these models the ability to use tools, to use APIs? What can they do when they can actually affect change out in the real world, beyond just streaming text back at the user? I think that's the really exciting piece. >> Okay, so I wanted to tee that up early in the segment 'cause I want to get into the customer applications. We're seeing early adopters come in, using the technology because they have a lot of data, they have a lot of large language model opportunities and then there's a big fast follower wave coming behind it. I call that the people who are going to jump in the pool early and get into it. They might not be advanced. Can you guys share what customer applications are being used with large language and vision models today and how they're using it to transform on the early adopter side, and how is that a tell sign of what's to come? >> You know, one of the things we have been seeing both with the text models that Aidan talked about as well as the vision models that stability.ai does, Tom, is customers are really using it to change the way you interact with information. You know, one example of a customer that we have, is someone who's kind of using that to query customer conversations and ask questions like, you know, "What was the customer issue? How did we solve it?" And trying to get those kinds of insights that was previously much harder to do. And then of course software is a big area. You know, generating software, making that, you know, just deploying it in production. Those have been really big areas that we have seen customers start to do. You know, looking at documentation, like instead of you know, searching for stuff and so on, you know, you just have an interactive way, in which you can just look at the documentation for a product. You know, all of this goes to where we need to take the technology. One of which is, you know, the models have to be there but they have to work reliably in a production setting at scale, with privacy, with security, and you know, making sure all of this is happening, is going to be really key. That is what, you know, we at AWS are looking to do, which is work with partners like stability and others and in the open source and really take all of these and make them available at scale to customers, where they work reliably. >> Tom, Aidan, what's your thoughts on this? Where are customers landing on this first use cases or set of low-hanging fruit use cases or applications? >> Yeah, so I think like the first group of adopters that really found product market fit were the copywriting companies. So one great example of that is HyperWrite. Another one is Jasper. And so for Cohere, that's the tip of the iceberg, like there's a very long tail of usage from a bunch of different applications. HyperWrite is one of our customers, they help beat writer's block by drafting blog posts, emails, and marketing copy. We also have a global audio streaming platform, which is using us the power of search engine that can comb through podcast transcripts, in a bunch of different languages. Then a global apparel brand, which is using us to transform how they interact with their customers through a virtual assistant, two dozen global news outlets who are using us for news summarization. So really like, these large language models, they can be deployed all over the place into every single industry sector, language is everywhere. It's hard to think of any company on Earth that doesn't use language. So it's, very, very- >> We're doing it right now. We got the language coming in. >> Exactly. >> We'll transcribe this puppy. All right. Tom, on your side, what do you see the- >> Yeah, we're seeing some amazing applications of it and you know, I guess that's partly been, because of the growth in the open source community and some of these applications have come from there that are then triggering this secondary wave of innovation, which is coming a lot from, you know, controllability and explainability of the model. But we've got companies like, you know, Jasper, which Aidan mentioned, who are using stable diffusion for image generation in block creation, content creation. We've got Lensa, you know, which exploded, and is built on top of stable diffusion for fine tuning so people can bring themselves and their pets and you know, everything into the models. So we've now got fine tuned stable diffusion at scale, which is democratized, you know, that process, which is really fun to see your Lensa, you know, exploded. You know, I think it was the largest growing app in the App Store at one point. And lots of other examples like NightCafe and Lexica and Playground. So seeing lots of cool applications. >> So much applications, we'll probably be a customer for all you guys. We'll definitely talk after. But the challenges are there for people adopting, they want to get into what you guys see as the challenges that turn into opportunities. How do you see the customers adopting generative AI applications? For example, we have massive amounts of transcripts, timed up to all the videos. I don't even know what to do. Do I just, do I code my API there. So, everyone has this problem, every vertical has these use cases. What are the challenges for people getting into this and adopting these applications? Is it figuring out what to do first? Or is it a technical setup? Do they stand up stuff, they just go to Amazon? What do you guys see as the challenges? >> I think, you know, the first thing is coming up with where you think you're going to reimagine your customer experience by using generative AI. You know, we talked about Ada, and Tom talked about a number of these ones and you know, you pick up one or two of these, to get that robust. And then once you have them, you know, we have models and we'll have more models on AWS, these large language models that Aidan was talking about. Then you go in and start using these models and testing them out and seeing whether they fit in use case or not. In many situations, like you said, John, our customers want to say, "You know, I know you've trained these models on a lot of publicly available data, but I want to be able to customize it for my use cases. Because, you know, there's some knowledge that I have created and I want to be able to use that." And then in many cases, and I think Aidan mentioned this. You know, you need these models to be up to date. Like you can't have it staying. And in those cases, you augmented with a knowledge base, you know you have to make sure that these models are not hallucinating. And so you need to be able to do the right kind of responsible AI checks. So, you know, you start with a particular use case, and there are a lot of them. Then, you know, you can come to AWS, and then look at one of the many models we have and you know, we are going to have more models for other modalities as well. And then, you know, play around with the models. We have a playground kind of thing where you can test these models on some data and then you can probably, you will probably want to bring your own data, customize it to your own needs, do some of the testing to make sure that the model is giving the right output and then just deploy it. And you know, we have a lot of tools. >> Yeah. >> To make this easy for our customers. >> How should people think about large language models? Because do they think about it as something that they tap into with their IP or their data? Or is it a large language model that they apply into their system? Is the interface that way? What's the interaction look like? >> In many situations, you can use these models out of the box. But in typical, in most of the other situations, you will want to customize it with your own data or with your own expectations. So the typical use case would be, you know, these are models are exposed through APIs. So the typical use case would be, you know you're using these APIs a little bit for testing and getting familiar and then there will be an API that will allow you to train this model further on your data. So you use that AI, you know, make sure you augmented the knowledge base. So then you use those APIs to customize the model and then just deploy it in an application. You know, like Tom was mentioning, a number of companies that are using these models. So once you have it, then you know, you again, use an endpoint API and use it in an application. >> All right, I love the example. I want to ask Tom and Aidan, because like most my experience with Amazon Web Service in 2007, I would stand up in EC2, put my code on there, play around, if it didn't work out, I'd shut it down. Is that a similar dynamic we're going to see with the machine learning where developers just kind of log in and stand up infrastructure and play around and then have a cloud-like experience? >> So I can go first. So I mean, we obviously, with AWS working really closely with the SageMaker team, do fantastic platform there for ML training and inference. And you know, going back to your point earlier, you know, where the data is, is hugely important for companies. Many companies bringing their models to their data in AWS on-premise for them is hugely important. Having the models to be, you know, open sources, makes them explainable and transparent to the adopters of those models. So, you know, we are really excited to work with the SageMaker team over the coming year to bring companies to that platform and make the most of our models. >> Aidan, what's your take on developers? Do they just need to have a team in place, if we want to interface with you guys? Let's say, can they start learning? What do they got to do to set up? >> Yeah, so I think for Cohere, our product makes it much, much easier to people, for people to get started and start building, it solves a lot of the productionization problems. But of course with SageMaker, like Tom was saying, I think that lowers a barrier even further because it solves problems like data privacy. So I want to underline what Bratin was saying earlier around when you're fine tuning or when you're using these models, you don't want your data being incorporated into someone else's model. You don't want it being used for training elsewhere. And so the ability to solve for enterprises, that data privacy and that security guarantee has been hugely important for Cohere, and that's very easy to do through SageMaker. >> Yeah. >> But the barriers for using this technology are coming down super quickly. And so for developers, it's just becoming completely intuitive. I love this, there's this quote from Andrej Karpathy. He was saying like, "It really wasn't on my 2022 list of things to happen that English would become, you know, the most popular programming language." And so the barrier is coming down- >> Yeah. >> Super quickly and it's exciting to see. >> It's going to be awesome for all the companies here, and then we'll do more, we're probably going to see explosion of startups, already seeing that, the maps, ecosystem maps, the landscape maps are happening. So this is happening and I'm convinced it's not yesterday's chat bot, it's not yesterday's AI Ops. It's a whole another ballgame. So I have to ask you guys for the final question before we kick off the company's showcasing here. How do you guys gauge success of generative AI applications? Is there a lens to look through and say, okay, how do I see success? It could be just getting a win or is it a bigger picture? Bratin we'll start with you. How do you gauge success for generative AI? >> You know, ultimately it's about bringing business value to our customers. And making sure that those customers are able to reimagine their experiences by using generative AI. Now the way to get their ease, of course to deploy those models in a safe, effective manner, and ensuring that all of the robustness and the security guarantees and the privacy guarantees are all there. And we want to make sure that this transitions from something that's great demos to actual at scale products, which means making them work reliably all of the time not just some of the time. >> Tom, what's your gauge for success? >> Look, I think this, we're seeing a completely new form of ways to interact with data, to make data intelligent, and directly to bring in new revenue streams into business. So if businesses can use our models to leverage that and generate completely new revenue streams and ultimately bring incredible new value to their customers, then that's fantastic. And we hope we can power that revolution. >> Aidan, what's your take? >> Yeah, reiterating Bratin and Tom's point, I think that value in the enterprise and value in market is like a huge, you know, it's the goal that we're striving towards. I also think that, you know, the value to consumers and actual users and the transformation of the surface area of technology to create experiences like ChatGPT that are magical and it's the first time in human history we've been able to talk to something compelling that's not a human. I think that in itself is just extraordinary and so exciting to see. >> It really brings up a whole another category of markets. B2B, B2C, it's B2D, business to developer. Because I think this is kind of the big trend the consumers have to win. The developers coding the apps, it's a whole another sea change. Reminds me everyone use the "Moneyball" movie as example during the big data wave. Then you know, the value of data. There's a scene in "Moneyball" at the end, where Billy Beane's getting the offer from the Red Sox, then the owner says to the Red Sox, "If every team's not rebuilding their teams based upon your model, there'll be dinosaurs." I think that's the same with AI here. Every company will have to need to think about their business model and how they operate with AI. So it'll be a great run. >> Completely Agree >> It'll be a great run. >> Yeah. >> Aidan, Tom, thank you so much for sharing about your experiences at your companies and congratulations on your success and it's just the beginning. And Bratin, thanks for coming on representing AWS. And thank you, appreciate for what you do. Thank you. >> Thank you, John. Thank you, Aidan. >> Thank you John. >> Thanks so much. >> Okay, let's kick off season three, episode one. I'm John Furrier, your host. Thanks for watching. (light airy music)

>> Hello everyone. Welcome to theCUBE's coverage of International Women's Day. I'm John Furrier, host of theCUBE been profiling the leaders in the technology world, women in technology from developers to the boardroom, everything in between. We have two great guests promoting in from Malaysia. Nancy Wang is the general manager, also CUBE alumni from AWS Data Protection, and founder and board chair of Advancing Women in Tech, awit.org. And of course Kate Watts who's the executive director of Advancing Women in Tech.org. So it's awit.org. Nancy, Kate, thanks for coming all the way across remotely from Malaysia. >> Of course, we're coming to you as fast as our internet bandwidth will allow us. And you know, I'm just thrilled today that you get to see a whole nother aspect of my life, right? Because typically we talk about AWS, and here we're talking about a topic near and dear to my heart. >> Well, Nancy, I love the fact that you're spending a lot of time taking the empowerment to go out and help the industries and helping with the advancement of women in tech. Kate, the executive director it's a 501C3, it's nonprofit, dedicating to accelerating the careers of women in groups in tech. Can you talk about the organization? >> Yes, I can. So Advancing Women in Tech was founded in 2017 in order to fix some of the pathway problems that we're seeing on the rise to leadership in the industry. And so we specifically focus on supporting mid-level women in technical roles, get into higher positions. We do that in a few different ways through mentorship programs through building technical skills and by connecting people to a supportive community. So you have your peer network and then a vertical sort of relationships to help you navigate the next steps in your career. So to date we've served about 40,000 individuals globally and we're just looking to expand our reach and impact and be able to better support women in the industry. >> Nancy, talk about the creation, the origination story. How'd this all come together? Obviously the momentum, everyone in the industry's been focused on this for a long time. Where did AWIT come from? Advancing Women in Technology, that's the acronym. Advancing Women in Technology.org, where'd it come from? What's the origination story? >> Yeah, so AWIT really originated from this desire that I had, to Kate's point around, well if you look around right and you know, don't take my word for it, right? Look at stats, look at news reports, or just frankly go on your LinkedIn and see how many women in underrepresented groups are in senior technical leadership roles right out in the companies whose names we all know. And so that was my case back in 2016. And so when I first got the idea and back then I was actually at Google, just another large tech company in the valley, right? It was about how do we get more role models, how we get more, for example, women into leadership roles so they can bring up the next generation, right? And so this is actually part of a longer speech that I'm about to give on Wednesday and part of the US State Department speaker program. In fact, that's why Kate and I are here in Malaysia right now is working with over 200 women entrepreneurs from all over in Southeast Asia, including Malaysia Philippines, Vietnam, Borneo, you know, so many countries where having more women entrepreneurs can help raise the GDP right, and that fits within our overall mission of getting more women into top leadership roles in tech. >> You know, I was talking about Teresa Carlson she came on the program as well for this year this next season we're going to do. And she mentioned the decision between the US progress and international. And she's saying as much as it's still bad numbers, it's worse than outside the United States and needs to get better. Can you comment on the global aspect? You brought that up. I think it's super important to highlight that it's just not one area, it's a global evolution. >> Absolutely, so let me start, and I'd love to actually have Kate talk about our current programs and all of the international groups that we're working with. So as Teresa aptly mentioned there is so much work to be done not just outside the US and North Americas where typically tech nonprofits will focus, but rather if you think about the one to end model, right? For example when I was doing the product market fit workshop for the US State Department I had women dialing in from rice fields, right? So let me just pause there for a moment. They were holding their cell phones up near towers near trees just so that they can get a few minutes of time with me to do a workshop and how to accelerate their business. So if you don't call that the desire to propel oneself or accelerate oneself, not sure what is, right. And so it's really that passion that drove me to spend the next week and a half here working with local entrepreneurs working with policy makers so we can take advantage and really leverage that passion that people have, right? To accelerate more business globally. And so that's why, you know Kate will be leading our contingent with the United Nations Women Group, right? That is focused on women's economic empowerment because that's super important, right? One aspect can be sure, getting more directors, you know vice presidents into companies like Google and Amazon. But another is also how do you encourage more women around the world to start businesses, right? To reach economic and freedom independence, right? To overcome some of the maybe social barriers to becoming a leader in their own country. >> Yes, and if I think about our own programs and our model of being very intentional about supporting the learning development and skills of women and members of underrepresented groups we focused very much on providing global access to a number of our programs. For instance, our product management certification on Coursera or engineering management our upcoming women founders accelerator. We provide both access that you can get from anywhere. And then also very intentional programming that connects people into the networks to be able to further their networks and what they've learned through the skills online, so. >> Yeah, and something Kate just told me recently is these courses that Kate's mentioning, right? She was instrumental in working with the American Council on Education and so that our learners can actually get up to six college credits for taking these courses on product management engineering management, on cloud product management. And most recently we had our first organic one of our very first organic testimonials was from a woman's tech bootcamp in Nigeria, right? So if you think about the worldwide impact of these upskilling courses where frankly in the US we might take for granted right around the world as I mentioned, there are women dialing in from rice patties from other, you know, for example, outside the, you know corporate buildings in order to access this content. >> Can you think about the idea of, oh sorry, go ahead. >> Go ahead, no, go ahead Kate. >> I was going to say, if you can't see it, you can't become it. And so we are very intentional about ensuring that we have we're spotlighting the expertise of women and we are broadcasting that everywhere so that anybody coming up can gain the skills and the networks to be able to succeed in this industry. >> We'll make sure we get those links so we can promote them. Obviously we feel the same way getting the word out. I think a couple things I'd like to ask you guys cause I think you hit a great point. One is the economic advantage the numbers prove that diverse teams perform better number one, that's clear. So good point there. But I want to get your thoughts on the entrepreneurial equation. You mentioned founders and startups and there's also different makeups in different countries. It's not like the big corporations sometimes it's smaller business in certain areas the different cultures have different business sizes and business types. How do you guys see that factoring in outside the United States, say the big tech companies? Okay, yeah. The easy lower the access to get in education than stay with them, in other countries is it the same or is it more diverse in terms of business? >> So what really actually got us started with the US State Department was around our work with women founders. And I love for Kate to actually share her experience working with AWS startups in that capacity. But frankly, you know, we looked at the content and the mentor programs that were providing women who wanted to be executives, you know, quickly realize a lot of those same skills such as finding customers, right? Scaling your product and building channels can also apply to women founders, not just executives. And so early supporters of our efforts from firms such as Moderna up in Seattle, Emergence Ventures, Decibel Ventures in, you know, the Bay Area and a few others that we're working with right now. Right, they believed in the mission and really helped us scale out what is now our existing platform and offerings for women founders. >> Those are great firms by the way. And they also are very founder friendly and also understand the global workforce. I mean, that's a whole nother dimension. Okay, what's your reaction to all that? >> Yes, we have been very intentional about taking the product expertise and the learnings of women and in our network, we first worked with AWS startups to support the development of the curriculum for the recent accelerator for women founders that was held last spring. And so we're able to support 25 founders and also brought in the expertise of about 20 or 30 women from Advancing Women in Tech to be able to be the lead instructors and mentors for that. And so we have really realized that with this network and this individual sort of focus on product expertise building strong teams, we can take that information and bring it to folks everywhere. And so there is very much the intentionality of allowing founders allowing individuals to take the lessons and bring it to their individual circumstances and the cultures in which they are operating. But the product sense is a skill that we can support the development of and we're proud to do so. >> That's awesome. Nancy, I want to ask you some never really talk about data storage and AWS cloud greatness and goodness, here's different and you also work full-time at AWS and you're the founder or the chairman of this great organization. How do you balance both and do you get, they're getting behind you on this, Amazon is getting behind you on this. >> Well, as I say it's always easier to negotiate on the way in. But jokes aside, I have to say the leadership has been tremendously supportive. If you think about, for example, my leaders Wayne Duso who's also been on the show multiple times, Bill Vaas who's also been on the show multiple times, you know they're both founders and also operators entrepreneurs at heart. So they understand that it is important, right? For all of us, it's really incumbent on all of us who are in positions to do so, to create a pathway for more people to be in leadership roles for more people to be successful entrepreneurs. So, no, I mean if you just looked at LinkedIn they're always uploading my vote so they reach to more audiences. And frankly they're rooting for us back home in the US while we're in Malaysia this week. >> That's awesome. And I think that's a good culture to have that empowerment and I think that's very healthy. What's next for you guys? What's on the agenda? Take us through the activities. I know that you got a ton of things happening. You got your event out there, which is why you're out there. There's a bunch of other activities. I think you guys call it the Advancing Women in Tech week. >> Yes, this week we are having a week of programming that you can check out at Advancing Women in Tech.org. That is spotlighting the expertise of a number of women in our space. So it is three days of programming Tuesday, Wednesday and Thursday if you are in the US so the seventh through the ninth, but available globally. We are also going to be in New York next week for the event at the UN and are looking to continue to support our mentorship programs and also our work supporting women founders throughout the year. >> All right. I have to ask you guys if you don't mind get a little market data so you can share with us here at theCUBE. What are you hearing this year that's different in the conversation space around the topics, the interests? Obviously I've seen massive amounts of global acceleration around conversations, more video, things like this more stories are scaling, a lot more LinkedIn activity. It just seems like it's a lot different this year. Can you guys share any kind of current trends you're seeing relative to the conversations and topics being discussed across the the community? >> Well, I think from a needle moving perspective, right? I think due to the efforts of wonderful organizations including the Q for spotlighting all of these awesome women, right? Trailblazing women and the nonprofits the government entities that we work with there's definitely more emphasis on creating access and creating pathways. So that's probably one thing that you're seeing is more women, more investors posting about their activities. Number two, from a global trend perspective, right? The rise of women in security. I noticed that on your agenda today, you had Lena Smart who's a good friend of mine chief information security officer at MongoDB, right? She and I are actually quite involved in helping founders especially early stage founders in the security space. And so globally from a pure technical perspective, right? There's right more increasing regulations around data privacy, data sovereignty, right? For example, India's in a few weeks about to get their first data protection regulation there locally. So all of that is giving rise to yet another wave of opportunity and we want women founders uniquely positioned to take advantage of that opportunity. >> I love it. Kate, reaction to that? I mean founders, more pathways it sounds like a neural network, it sounds like AI enabled. >> Yes, and speaking of AI, with the rise of that we are also hearing from many community members the importance of continuing to build their skills upskill learn to be able to keep up with the latest trends. There's a lot of people wondering what does this mean for my own career? And so they're turning to organizations like Advancing Women in Tech to find communities to both learn the latest information, but also build their networks so that they are able to move forward regardless of what the industry does. >> I love the work you guys are doing. It's so impressive. I think the economic angle is new it's more amplified this year. It's always kind of been there and continues to be. What do you guys hope for by next year this time what do you hope to see different from a needle moving perspective, to use your word Nancy, for next year? What's the visual output in your mind? >> I want to see real effort made towards 50-50 representation in all tech leadership roles. And I'd like to see that happen by 2050. >> Kate, anything on your end? >> I love that. I'm going to go a little bit more touchy-feely. I want everybody in our space to understand that the skills that they build and that the networks they have carry with them regardless of wherever they go. And so to be able to really lean in and learn and continue to develop the career that you want to have. So whether that be at a large organization or within your own business, that you've got the potential to move forward on that within you. >> Nancy, Kate, thank you so much for your contribution. I'll give you the final word. Put a plug in for the organization. What are you guys looking for? Any kind of PSA you want to share with the folks watching? >> Absolutely, so if you're in a position to be a mentor, join as a mentor, right? Help elevate and accelerate the next generation of women leaders. If you're an investor help us invest in more women started companies, right? Women founded startups and lastly, if you are women looking to accelerate your career, come join our community. We have resources, we have mentors and who we have investors who are willing to come in on the ground floor and help you accelerate your business. >> Great work. Thank you so much for participating in our International Women's Day 23 program and we'd look to keep this going quarterly. We'll see you next year, next time. Thanks for coming on. Appreciate it. >> Thanks so much John. >> Thank you. >> Okay, women leaders here. >> Nancy: Thanks for having us >> All over the world, coming together for a great celebration but really highlighting the accomplishments, the pathways the investment, the mentoring, everything in between. It's theCUBE. Bring as much as we can. I'm John Furrier, your host. Thanks for watching.

(upbeat music) >> Hello and welcome to theCube's coverage of International Women's Day. I'm John Furrier, your host of "theCUBE." We've got great two remote guests coming into our Palo Alto Studios, some tech athletes, as we say, people that've been in the trenches, years of experience, Lena Smart, CISO at MongoDB, Cube alumni, and Tara Hernandez, VP of Developer Productivity at MongoDB as well. Thanks for coming in to this program and supporting our efforts today. Thanks so much. >> Thanks for having us. >> Yeah, everyone talk about the journey in tech, where it all started. Before we get there, talk about what you guys are doing at MongoDB specifically. MongoDB is kind of gone the next level as a platform. You have your own ecosystem, lot of developers, very technical crowd, but it's changing the business transformation. What do you guys do at Mongo? We'll start with you, Lena. >> So I'm the CISO, so all security goes through me. I like to say, well, I don't like to say, I'm described as the ones throat to choke. So anything to do with security basically starts and ends with me. We do have a fantastic Cloud engineering security team and a product security team, and they don't report directly to me, but obviously we have very close relationships. I like to keep that kind of church and state separate and I know I've spoken about that before. And we just recently set up a physical security team with an amazing gentleman who left the FBI and he came to join us after 26 years for the agency. So, really starting to look at the physical aspects of what we offer as well. >> I interviewed a CISO the other day and she said, "Every day is day zero for me." Kind of goofing on the Amazon Day one thing, but Tara, go ahead. Tara, go ahead. What's your role there, developer productivity? What are you focusing on? >> Sure. Developer productivity is kind of the latest description for things that we've described over the years as, you know, DevOps oriented engineering or platform engineering or build and release engineering development infrastructure. It's all part and parcel, which is how do we actually get our code from developer to customer, you know, and all the mechanics that go into that. It's been something I discovered from my first job way back in the early '90s at Borland. And the art has just evolved enormously ever since, so. >> Yeah, this is a very great conversation both of you guys, right in the middle of all the action and data infrastructures changing, exploding, and involving big time AI and data tsunami and security never stops. Well, let's get into, we'll talk about that later, but let's get into what motivated you guys to pursue a career in tech and what were some of the challenges that you faced along the way? >> I'll go first. The fact of the matter was I intended to be a double major in history and literature when I went off to university, but I was informed that I had to do a math or a science degree or else the university would not be paid for. At the time, UC Santa Cruz had a policy that called Open Access Computing. This is, you know, the late '80s, early '90s. And anybody at the university could get an email account and that was unusual at the time if you were, those of us who remember, you used to have to pay for that CompuServe or AOL or, there's another one, I forget what it was called, but if a student at Santa Cruz could have an email account. And because of that email account, I met people who were computer science majors and I'm like, "Okay, I'll try that." That seems good. And it was a little bit of a struggle for me, a lot I won't lie, but I can't complain with how it ended up. And certainly once I found my niche, which was development infrastructure, I found my true love and I've been doing it for almost 30 years now. >> Awesome. Great story. Can't wait to ask a few questions on that. We'll go back to that late '80s, early '90s. Lena, your journey, how you got into it. >> So slightly different start. I did not go to university. I had to leave school when I was 16, got a job, had to help support my family. Worked a bunch of various jobs till I was about 21 and then computers became more, I think, I wouldn't say they were ubiquitous, but they were certainly out there. And I'd also been saving up every penny I could earn to buy my own computer and bought an Amstrad 1640, 20 meg hard drive. It rocked. And kind of took that apart, put it back together again, and thought that could be money in this. And so basically just teaching myself about computers any job that I got. 'Cause most of my jobs were like clerical work and secretary at that point. But any job that had a computer in front of that, I would make it my business to go find the guy who did computing 'cause it was always a guy. And I would say, you know, I want to learn how these work. Let, you know, show me. And, you know, I would take my lunch hour and after work and anytime I could with these people and they were very kind with their time and I just kept learning, so yep. >> Yeah, those early days remind me of the inflection point we're going through now. This major C change coming. Back then, if you had a computer, you had to kind of be your own internal engineer to fix things. Remember back on the systems revolution, late '80s, Tara, when, you know, your career started, those were major inflection points. Now we're seeing a similar wave right now, security, infrastructure. It feels like it's going to a whole nother level. At Mongo, you guys certainly see this as well, with this AI surge coming in. A lot more action is coming in. And so there's a lot of parallels between these inflection points. How do you guys see this next wave of change? Obviously, the AI stuff's blowing everyone away. Oh, new user interface. It's been called the browser moment, the mobile iPhone moment, kind of for this generation. There's a lot of people out there who are watching that are young in their careers, what's your take on this? How would you talk to those folks around how important this wave is? >> It, you know, it's funny, I've been having this conversation quite a bit recently in part because, you know, to me AI in a lot of ways is very similar to, you know, back in the '90s when we were talking about bringing in the worldwide web to the forefront of the world, right. And we tended to think in terms of all the optimistic benefits that would come of it. You know, free passing of information, availability to anyone, anywhere. You just needed an internet connection, which back then of course meant a modem. >> John: Not everyone had though. >> Exactly. But what we found in the subsequent years is that human beings are what they are and we bring ourselves to whatever platforms that are there, right. And so, you know, as much as it was amazing to have this freely available HTML based internet experience, it also meant that the negatives came to the forefront quite quickly. And there were ramifications of that. And so to me, when I look at AI, we're already seeing the ramifications to that. Yes, are there these amazing, optimistic, wonderful things that can be done? Yes. >> Yeah. >> But we're also human and the bad stuff's going to come out too. And how do we- >> Yeah. >> How do we as an industry, as a community, you know, understand and mitigate those ramifications so that we can benefit more from the positive than the negative. So it is interesting that it comes kind of full circle in really interesting ways. >> Yeah. The underbelly takes place first, gets it in the early adopter mode. Normally industries with, you know, money involved arbitrage, no standards. But we've seen this movie before. Is there hope, Lena, that we can have a more secure environment? >> I would hope so. (Lena laughs) Although depressingly, we've been in this well for 30 years now and we're, at the end of the day, still telling people not to click links on emails. So yeah, that kind of still keeps me awake at night a wee bit. The whole thing about AI, I mean, it's, obviously I am not an expert by any stretch of the imagination in AI. I did read (indistinct) book recently about AI and that was kind of interesting. And I'm just trying to teach myself as much as I can about it to the extent of even buying the "Dummies Guide to AI." Just because, it's actually not a dummies guide. It's actually fairly interesting, but I'm always thinking about it from a security standpoint. So it's kind of my worst nightmare and the best thing that could ever happen in the same dream. You know, you've got this technology where I can ask it a question and you know, it spits out generally a reasonable answer. And my team are working on with Mark Porter our CTO and his team on almost like an incubation of AI link. What would it look like from MongoDB? What's the legal ramifications? 'Cause there will be legal ramifications even though it's the wild, wild west just now, I think. Regulation's going to catch up to us pretty quickly, I would think. >> John: Yeah, yeah. >> And so I think, you know, as long as companies have a seat at the table and governments perhaps don't become too dictatorial over this, then hopefully we'll be in a good place. But we'll see. I think it's a really interest, there's that curse, we're living in interesting times. I think that's where we are. >> It's interesting just to stay on this tech trend for a minute. The standards bodies are different now. Back in the old days there were, you know, IEEE standards, ITF standards. >> Tara: TPC. >> The developers are the new standard. I mean, now you're seeing open source completely different where it was in the '90s to here beginning, that was gen one, some say gen two, but I say gen one, now we're exploding with open source. You have kind of developers setting the standards. If developers like it in droves, it becomes defacto, which then kind of rolls into implementation. >> Yeah, I mean I think if you don't have developer input, and this is why I love working with Tara and her team so much is 'cause they get it. If we don't have input from developers, it's not going to get used. There's going to be ways of of working around it, especially when it comes to security. If they don't, you know, if you're a developer and you're sat at your screen and you don't want to do that particular thing, you're going to find a way around it. You're a smart person. >> Yeah. >> So. >> Developers on the front lines now versus, even back in the '90s, they're like, "Okay, consider the dev's, got a QA team." Everything was Waterfall, now it's Cloud, and developers are on the front lines of everything. Tara, I mean, this is where the standards are being met. What's your reaction to that? >> Well, I think it's outstanding. I mean, you know, like I was at Netscape and part of the crowd that released the browser as open source and we founded mozilla.org, right. And that was, you know, in many ways kind of the birth of the modern open source movement beyond what we used to have, what was basically free software foundation was sort of the only game in town. And I think it is so incredibly valuable. I want to emphasize, you know, and pile onto what Lena was saying, it's not just that the developers are having input on a sort of company by company basis. Open source to me is like a checks and balance, where it allows us as a broader community to be able to agree on and enforce certain standards in order to try and keep the technology platforms as accessible as possible. I think Kubernetes is a great example of that, right. If we didn't have Kubernetes, that would've really changed the nature of how we think about container orchestration. But even before that, Linux, right. Linux allowed us as an industry to end the Unix Wars and as someone who was on the front lines of that as well and having to support 42 different operating systems with our product, you know, that was a huge win. And it allowed us to stop arguing about operating systems and start arguing about software or not arguing, but developing it in positive ways. So with, you know, with Kubernetes, with container orchestration, we all agree, okay, that's just how we're going to orchestrate. Now we can build up this huge ecosystem, everybody gets taken along, right. And now it changes the game for what we're defining as business differentials, right. And so when we talk about crypto, that's a little bit harder, but certainly with AI, right, you know, what are the checks and balances that as an industry and as the developers around this, that we can in, you know, enforce to make sure that no one company or no one body is able to overly control how these things are managed, how it's defined. And I think that is only for the benefit in the industry as a whole, particularly when we think about the only other option is it gets regulated in ways that do not involve the people who actually know the details of what they're talking about. >> Regulated and or thrown away or bankrupt or- >> Driven underground. >> Yeah. >> Which would be even worse actually. >> Yeah, that's a really interesting, the checks and balances. I love that call out. And I was just talking with another interview part of the series around women being represented in the 51% ratio. Software is for everybody. So that we believe that open source movement around the collective intelligence of the participants in the industry and independent of gender, this is going to be the next wave. You're starting to see these videos really have impact because there are a lot more leaders now at the table in companies developing software systems and with AI, the aperture increases for applications. And this is the new dynamic. What's your guys view on this dynamic? How does this go forward in a positive way? Is there a certain trajectory you see? For women in the industry? >> I mean, I think some of the states are trying to, again, from the government angle, some of the states are trying to force women into the boardroom, for example, California, which can be no bad thing, but I don't know, sometimes I feel a bit iffy about all this kind of forced- >> John: Yeah. >> You know, making, I don't even know how to say it properly so you can cut this part of the interview. (John laughs) >> Tara: Well, and I think that they're >> I'll say it's not organic. >> No, and I think they're already pulling it out, right. It's already been challenged so they're in the process- >> Well, this is the open source angle, Tara, you are getting at it. The change agent is open, right? So to me, the history of the proven model is openness drives transparency drives progress. >> No, it's- >> If you believe that to be true, this could have another impact. >> Yeah, it's so interesting, right. Because if you look at McKinsey Consulting or Boston Consulting or some of the other, I'm blocking on all of the names. There has been a decade or more of research that shows that a non homogeneous employee base, be it gender or ethnicity or whatever, generates more revenue, right? There's dollar signs that can be attached to this, but it's not enough for all companies to want to invest in that way. And it's not enough for all, you know, venture firms or investment firms to grant that seed money or do those seed rounds. I think it's getting better very slowly, but socialization is a much harder thing to overcome over time. Particularly, when you're not just talking about one country like the United States in our case, but around the world. You know, tech centers now exist all over the world, including places that even 10 years ago we might not have expected like Nairobi, right. Which I think is amazing, but you have to factor in the cultural implications of that as well, right. So yes, the openness is important and we have, it's important that we have those voices, but I don't think it's a panacea solution, right. It's just one more piece. I think honestly that one of the most important opportunities has been with Cloud computing and Cloud's been around for a while. So why would I say that? It's because if you think about like everybody holds up the Steve Jobs, Steve Wozniak, back in the '70s, or Sergey and Larry for Google, you know, you had to have access to enough credit card limit to go to Fry's and buy your servers and then access to somebody like Susan Wojcicki to borrow the garage or whatever. But there was still a certain amount of upfrontness that you had to be able to commit to, whereas now, and we've, I think, seen a really good evidence of this being able to lease server resources by the second and have development platforms that you can do on your phone. I mean, for a while I think Africa, that the majority of development happened on mobile devices because there wasn't a sufficient supply chain of laptops yet. And that's no longer true now as far as I know. But like the power that that enables for people who would otherwise be underrepresented in our industry instantly opens it up, right? And so to me that's I think probably the biggest opportunity that we've seen from an industry on how to make more availability in underrepresented representation for entrepreneurship. >> Yeah. >> Something like AI, I think that's actually going to take us backwards if we're not careful. >> Yeah. >> Because of we're reinforcing that socialization. >> Well, also the bias. A lot of people commenting on the biases of the large language inherently built in are also problem. Lena, I want you to weigh on this too, because I think the skills question comes up here and I've been advocating that you don't need the pedigree, college pedigree, to get into a certain jobs, you mentioned Cloud computing. I mean, it's been around for you think a long time, but not really, really think about it. The ability to level up, okay, if you're going to join something new and half the jobs in cybersecurity are created in the past year, right? So, you have this what used to be a barrier, your degree, your pedigree, your certification would take years, would be a blocker. Now that's gone. >> Lena: Yeah, it's the opposite. >> That's, in fact, psychology. >> I think so, but the people who I, by and large, who I interview for jobs, they have, I think security people and also I work with our compliance folks and I can't forget them, but let's talk about security just now. I've always found a particular kind of mindset with security folks. We're very curious, not very good at following rules a lot of the time, and we'd love to teach others. I mean, that's one of the big things stem from the start of my career. People were always interested in teaching and I was interested in learning. So it was perfect. And I think also having, you know, strong women leaders at MongoDB allows other underrepresented groups to actually apply to the company 'cause they see that we're kind of talking the talk. And that's been important. I think it's really important. You know, you've got Tara and I on here today. There's obviously other senior women at MongoDB that you can talk to as well. There's a bunch of us. There's not a whole ton of us, but there's a bunch of us. And it's good. It's definitely growing. I've been there for four years now and I've seen a growth in women in senior leadership positions. And I think having that kind of track record of getting really good quality underrepresented candidates to not just interview, but come and join us, it's seen. And it's seen in the industry and people take notice and they're like, "Oh, okay, well if that person's working, you know, if Tara Hernandez is working there, I'm going to apply for that." And that in itself I think can really, you know, reap the rewards. But it's getting started. It's like how do you get your first strong female into that position or your first strong underrepresented person into that position? It's hard. I get it. If it was easy, we would've sold already. >> It's like anything. I want to see people like me, my friends in there. Am I going to be alone? Am I going to be of a group? It's a group psychology. Why wouldn't? So getting it out there is key. Is there skills that you think that people should pay attention to? One's come up as curiosity, learning. What are some of the best practices for folks trying to get into the tech field or that's in the tech field and advancing through? What advice are you guys- >> I mean, yeah, definitely, what I say to my team is within my budget, we try and give every at least one training course a year. And there's so much free stuff out there as well. But, you know, keep learning. And even if it's not right in your wheelhouse, don't pick about it. Don't, you know, take a look at what else could be out there that could interest you and then go for it. You know, what does it take you few minutes each night to read a book on something that might change your entire career? You know, be enthusiastic about the opportunities out there. And there's so many opportunities in security. Just so many. >> Tara, what's your advice for folks out there? Tons of stuff to taste, taste test, try things. >> Absolutely. I mean, I always say, you know, my primary qualifications for people, I'm looking for them to be smart and motivated, right. Because the industry changes so quickly. What we're doing now versus what we did even last year versus five years ago, you know, is completely different though themes are certainly the same. You know, we still have to code and we still have to compile that code or package the code and ship the code so, you know, how well can we adapt to these new things instead of creating floppy disks, which was my first job. Five and a quarters, even. The big ones. >> That's old school, OG. There it is. Well done. >> And now it's, you know, containers, you know, (indistinct) image containers. And so, you know, I've gotten a lot of really great success hiring boot campers, you know, career transitioners. Because they bring a lot experience in addition to the technical skills. I think the most important thing is to experiment and figuring out what do you like, because, you know, maybe you are really into security or maybe you're really into like deep level coding and you want to go back, you know, try to go to school to get a degree where you would actually want that level of learning. Or maybe you're a front end engineer, you want to be full stacked. Like there's so many different things, data science, right. Maybe you want to go learn R right. You know, I think it's like figure out what you like because once you find that, that in turn is going to energize you 'cause you're going to feel motivated. I think the worst thing you could do is try to force yourself to learn something that you really could not care less about. That's just the worst. You're going in handicapped. >> Yeah and there's choices now versus when we were breaking into the business. It was like, okay, you software engineer. They call it software engineering, that's all it was. You were that or you were in sales. Like, you know, some sort of systems engineer or sales and now it's,- >> I had never heard of my job when I was in school, right. I didn't even know it was a possibility. But there's so many different types of technical roles, you know, absolutely. >> It's so exciting. I wish I was young again. >> One of the- >> Me too. (Lena laughs) >> I don't. I like the age I am. So one of the things that I did to kind of harness that curiosity is we've set up a security champions programs. About 120, I guess, volunteers globally. And these are people from all different backgrounds and all genders, diversity groups, underrepresented groups, we feel are now represented within this champions program. And people basically give up about an hour or two of their time each week, with their supervisors permission, and we basically teach them different things about security. And we've now had seven full-time people move from different areas within MongoDB into my team as a result of that program. So, you know, monetarily and time, yeah, saved us both. But also we're showing people that there is a path, you know, if you start off in Tara's team, for example, doing X, you join the champions program, you're like, "You know, I'd really like to get into red teaming. That would be so cool." If it fits, then we make that happen. And that has been really important for me, especially to give, you know, the women in the underrepresented groups within MongoDB just that window into something they might never have seen otherwise. >> That's a great common fit is fit matters. Also that getting access to what you fit is also access to either mentoring or sponsorship or some sort of, at least some navigation. Like what's out there and not being afraid to like, you know, just ask. >> Yeah, we just actually kicked off our big mentor program last week, so I'm the executive sponsor of that. I know Tara is part of it, which is fantastic. >> We'll put a plug in for it. Go ahead. >> Yeah, no, it's amazing. There's, gosh, I don't even know the numbers anymore, but there's a lot of people involved in this and so much so that we've had to set up mentoring groups rather than one-on-one. And I think it was 45% of the mentors are actually male, which is quite incredible for a program called Mentor Her. And then what we want to do in the future is actually create a program called Mentor Them so that it's not, you know, not just on the female and so that we can live other groups represented and, you know, kind of break down those groups a wee bit more and have some more granularity in the offering. >> Tara, talk about mentoring and sponsorship. Open source has been there for a long time. People help each other. It's community-oriented. What's your view of how to work with mentors and sponsors if someone's moving through ranks? >> You know, one of the things that was really interesting, unfortunately, in some of the earliest open source communities is there was a lot of pervasive misogyny to be perfectly honest. >> Yeah. >> And one of the important adaptations that we made as an open source community was the idea, an introduction of code of conducts. And so when I'm talking to women who are thinking about expanding their skills, I encourage them to join open source communities to have opportunity, even if they're not getting paid for it, you know, to develop their skills to work with people to get those code reviews, right. I'm like, "Whatever you join, make sure they have a code of conduct and a good leadership team. It's very important." And there are plenty, right. And then that idea has come into, you know, conferences now. So now conferences have codes of contact, if there are any good, and maybe not all of them, but most of them, right. And the ideas of expanding that idea of intentional healthy culture. >> John: Yeah. >> As a business goal and business differentiator. I mean, I won't lie, when I was recruited to come to MongoDB, the culture that I was able to discern through talking to people, in addition to seeing that there was actually women in senior leadership roles like Lena, like Kayla Nelson, that was a huge win. And so it just builds on momentum. And so now, you know, those of us who are in that are now representing. And so that kind of reinforces, but it's all ties together, right. As the open source world goes, particularly for a company like MongoDB, which has an open source product, you know, and our community builds. You know, it's a good thing to be mindful of for us, how we interact with the community and you know, because that could also become an opportunity for recruiting. >> John: Yeah. >> Right. So we, in addition to people who might become advocates on Mongo's behalf in their own company as a solution for themselves, so. >> You guys had great successful company and great leadership there. I mean, I can't tell you how many times someone's told me "MongoDB doesn't scale. It's going to be dead next year." I mean, I was going back 10 years. It's like, just keeps getting better and better. You guys do a great job. So it's so fun to see the success of developers. Really appreciate you guys coming on the program. Final question, what are you guys excited about to end the segment? We'll give you guys the last word. Lena will start with you and Tara, you can wrap us up. What are you excited about? >> I'm excited to see what this year brings. I think with ChatGPT and its copycats, I think it'll be a very interesting year when it comes to AI and always in the lookout for the authentic deep fakes that we see coming out. So just trying to make people aware that this is a real thing. It's not just pretend. And then of course, our old friend ransomware, let's see where that's going to go. >> John: Yeah. >> And let's see where we get to and just genuine hygiene and housekeeping when it comes to security. >> Excellent. Tara. >> Ah, well for us, you know, we're always constantly trying to up our game from a security perspective in the software development life cycle. But also, you know, what can we do? You know, one interesting application of AI that maybe Google doesn't like to talk about is it is really cool as an addendum to search and you know, how we might incorporate that as far as our learning environment and developer productivity, and how can we enable our developers to be more efficient, productive in their day-to-day work. So, I don't know, there's all kinds of opportunities that we're looking at for how we might improve that process here at MongoDB and then maybe be able to share it with the world. One of the things I love about working at MongoDB is we get to use our own products, right. And so being able to have this interesting document database in order to put information and then maybe apply some sort of AI to get it out again, is something that we may well be looking at, if not this year, then certainly in the coming year. >> Awesome. Lena Smart, the chief information security officer. Tara Hernandez, vice president developer of productivity from MongoDB. Thank you so much for sharing here on International Women's Day. We're going to do this quarterly every year. We're going to do it and then we're going to do quarterly updates. Thank you so much for being part of this program. >> Thank you. >> Thanks for having us. >> Okay, this is theCube's coverage of International Women's Day. I'm John Furrier, your host. Thanks for watching. (upbeat music)

(upbeat music) >> Hi, everyone. Welcome to theCUBE's coverage of International Women's Day. I'm John Furrier, host of theCUBE. As part of International Women's Day, we're featuring some of the leading women in business technology from developer to all types of titles and to the executive level. And one topic that's really important is called Getting a Seat at the Table, board makeup, having representation at corporate boards, private and public companies. It's been a big push. And former technology operating executive and corporate board member, she's a board machine Sue Barsamian, formerly with HPE, Hewlett Packard. Sue, great to see you. CUBE alumni, distinguished CUBE alumni. Thank you for coming on. >> Yes, I'm very proud of my CUBE alumni title. >> I'm sure it opens a lot of doors for you. (Sue laughing) We're psyched to have you on. This is a really important topic, and I want to get into the whole, as women advance up, and they're sitting on the boards, they can implement policy and there's governance. Obviously public companies have very strict oversight, and not strict, but like formal. Private boards have to operate, be nimble. They don't have to share all their results. But still, boards play an important role in the success of scaled up companies. So super important, that representation there is key. >> Yes. >> I want to get into that, but first, before we get started, how did you get into tech? How did it all start for you? >> Yeah, long time ago, I was an electrical engineering major. Came out in 1981 when, you know, opportunities for engineering, if you were kind, I went to Kansas State as an undergrad, and basically in those days you went to Texas and did semiconductors. You went to Atlanta and did communication satellites. You went to Boston or you went to Silicon Valley. And for me, that wasn't too hard a choice. I ended up going west and really, I guess what, embarked on a 40 year career in Silicon Valley and absolutely loved it. Largely software, but some time on the hardware side. Started out in networking, but largely software. And then, you know, four years ago transitioned to my next chapter, which is the corporate board director. And again, focused on technology software and cybersecurity boards. >> For the folks watching, we'll cut through another segment we can probably do about your operating career, but you rose through the ranks and became a senior operating executive at the biggest companies in the world. Hewlett Packard Enterprise, Hewlett Packard Enterprise and others. Very great career, okay. And so now you're kind of like, put that on pause, and you're moving on to the next chapter, which is being a board director. What inspired you to be a board director for multiple public companies and multiple private companies? Well, how many companies are you on? But what's the inspiration? What's the inspiration? First tell me how many board ships you're on, board seats you're on, and then what inspired you to become a board director? >> Yeah, so I'm on three public, and you are limited in terms of the number of publics that you can do to four. So I'm on three public, and I'm on four private from a tech perspective. And those range from, you know, a $4 billion in revenue public company down to a 35 person private company. So I've got the whole range. >> So you're like freelancing, I mean, what is it like? It's a full-time job, obviously. It's a lot of work involved. >> Yeah, yeah, it's. >> John: Why are you doing it? >> Well, you know, so I retired from being an operating executive after 37 years. And, but I loved, I mean, it's tough, right? It's tough these days, particularly with all the pressures out there in the market, not to mention the pandemic, et cetera. But I loved it. I loved working. I loved having a career, and I was ready to back off on, I would say the stresses of quarterly results and the stresses of international travel. You have so much of it. But I wasn't ready to back off from being involved and engaged and continuing to learn new things. I think this is why you come to tech, and for me, why I went to the valley to begin with was really that energy and that excitement, and it's like it's constantly reinventing itself. And I felt like that wasn't over for me. And I thought because I hadn't done boards before I retired from operating roles, I thought, you know, that would fill the bill. And it's honestly, it has exceeded expectations. >> In a good way. You feel good about where you're at and. >> Yeah. >> What you went in, what was the expectation going in and what surprised you? And were there people along the way that kind of gave you some pointers or don't do this, stay away from this. Take us through your experiences. >> Yeah, honestly, there is an amazing network of technology board directors, you know, in the US and specifically in the Valley. And we are all incredibly supportive. We have groups where we get together as board directors, and we talk about topics, and we share best practices and stories, and so I underestimated that, right? I thought I was going to, I thought I was going to enter this chapter where I would be largely giving back after 37 years. You've learned a little bit, right? What I underestimated was just the power of continuing to learn and being surrounded by so many amazing people. When, you know, when you do, you know, multiple boards, your learnings are just multiplied, right? Because you see not just one model, but you see many models. You see not just one problem, but many problems. Not just one opportunity, but many opportunities. And I underestimated how great that would be for me from a learning perspective and then your ability to share from one board to the other board because all of my boards are companies who are also quite close to each other, the executives collaborate. So that has turned out to be really exciting for me. >> So you had the stressful job. You rose to the top of the ranks, quarterly shot clock earnings, and it's hard charging. It's like, it's like, you know, being an athlete, as we say tech athlete. You're a tech athlete. Now you're taking that to the next level, which is now you're juggling multiple operational kind of things, but not with super pressure. But there's still a lot of responsibility. I know there's one board, you got compensation committee, I mean there's work involved. It's not like you're clipping coupons and having pizza. >> Yeah, no, it's real work. Believe me, it's real work. But I don't know how long it took me to not, to stop waking up and looking at my phone and thinking somebody was going to be dropping their forecast, right? Just that pressure of the number, and as a board member, obviously you are there to support and help guide the company and you feel, you know, you feel the pressure and the responsibility of what that role entails, but it's not the same as the frontline pressure every quarter. It's different. And so I did the first type. I loved it, you know. I'm loving this second type. >> You know, the retirement, it's always a cliche these days, but it's not really like what people think it is. It's not like getting a boat, going fishing or whatever. It's doing whatever you want to do, that's what retirement is. And you've chose to stay active. Your brain's being tested, and you're working it, having fun without all the stress. But it's enough, it's like going the gym. You're not hardcore workout, but you're working out with the brain. >> Yeah, no, for sure. It's just a different, it's just a different model. But the, you know, the level of conversations, the level of decisions, all of that is quite high. Which again, I like, yeah. >> Again, you really can't talk about some of the fun questions I want to ask, like what's the valuations like? How's the market, your headwinds? Is there tailwinds? >> Yes, yes, yes. It's an amazing, it's an amazing market right now with, as you know, counter indicators everywhere, right? Something's up, something's down, you know. Consumer spending's up, therefore interest rates go up and, you know, employment's down. And so or unemployment's down. And so it's hard. Actually, I really empathize with, you know, the, and have a great deal of respect for the CEOs and leadership teams of my board companies because, you know, I kind of retired from operating role, and then everybody else had to deal with running a company during a pandemic and then running a company through the great resignation, and then running a company through a downturn. You know, those are all tough things, and I have a ton of respect for any operating executive who's navigating through this and leading a company right now. >> I'd love to get your take on the board conversations at the end if we have more time, what the mood is, but I want to ask you about one more thing real quick before we go to the next topic is you're a retired operating executive. You have multiple boards, so you've got your hands full. I noticed there's a lot of amazing leaders, other female tech athletes joining boards, but they also have full-time jobs. >> Yeah. >> And so what's your advice? Cause I know there's a lot of networking, a lot of sharing going on. There's kind of a balance between how much you can contribute on the board versus doing the day job, but there's a real need for more women on boards, so yet there's a lot going on boards. What's the current state of the union if you will, state of the market relative to people in their careers and the stresses? >> Yeah. >> Cause you left one and jumped in all in there. >> Yeah. >> Some can't do that. They can't be on five boards, but they're on a few. What's the? >> Well, and you know, and if you're an operating executive, you wouldn't be on five boards, right? You would be on one or two. And so I spend a lot of time now bringing along the next wave of women and helping them both in their career but also to get a seat at the table on a board. And I'm very vocal about telling people not to do it the way I do it. There's no reason for it to be sequential. You can, you know, I thought I was so busy and was traveling all the time, and yes, all of that was true, but, and maybe I should say, you know, you can still fit in a board. And so, and what I see now is that your learnings are so exponential with outside perspective that I believe I would've been an even better operating executive had I done it earlier. I know I would've been an even better operating executive had I done it earlier. And so my advice is don't do it the way I did it. You know, it's worked out fine for me, but hindsight's 2020, I would. >> If you can go back and do a mulligan or a redo, what would you do? >> Yeah, I would get on a board earlier, full stop, yeah. >> Board, singular, plural? >> Well, I really, I don't think as an operating executive you can do, you could do one, maybe two. I wouldn't go beyond that, and I think that's fine. >> Yeah, totally makes sense. Okay, I got to ask you about your career. I know technical, you came in at that time in the market, I remember when I broke into the business, very male dominated, and then now it's much better. When you went through the ranks as a technical person, I know you had some blockers and definitely some, probably some people like, well, you know. We've seen that. How did you handle that? What were some of the key pivot points in your journey? And we've had a lot of women tell their stories here on theCUBE, candidly, like, hey, I was going to tell that professor, I'm going to sit in the front row. I'm going to, I'm getting two degrees, you know, robotics and aerospace. So, but they were challenged, even with the aspiration to do tech. I'm not saying that was something that you had, but like have you had experience like that, that you overcome? What were those key points and how did you handle them and how does that help people today? >> Yeah, you know, I have to say, you know, and not discounting that obviously this has been a journey for women, and there are a lot of things to overcome both in the workforce and also just balancing life honestly. And they're all real. There's also a story of incredible support, and you know, I'm the type of person where if somebody blocked me or didn't like me, I tended to just, you know, think it was me and like work harder and get around them, and I'm sure that some of that was potentially gender related. I didn't interpret it that way at the time. And I was lucky to have amazing mentors, many, many, many of whom were men, you know, because they were in the positions of power, and they made a huge difference on my career, huge. And I also had amazing female mentors, Meg Whitman, Ann Livermore at HPE, who you know well. So I had both, but you know, when I look back on the people who made a difference, there are as many men on the list as there are women. >> Yeah, and that's a learning there. Create those coalitions, not just one or the other. >> Yeah, yeah, yeah, absolutely. >> Well, I got to ask you about the, well, you brought up the pandemic. This has come up on some interviews this year, a little bit last year on the International Women's Day, but this year it's resonating, and I would never ask in an interview. I saw an interview once where a host asked a woman, how do you balance it all? And I was just like, no one asked men that. And so it's like, but with remote work, it's come up now the word empathy around people knowing each other's personal situation. In other words, when remote work happened, everybody went home. So we all got a glimpse of the backdrop. You got, you can see what their personal life was on Facebook. We were just commenting before we came on camera about that. So remote work really kind of opened up this personal side of everybody, men and women. >> Yeah. >> So I think this brings this new empathy kind of vibe or authentic self people call it. Is remote work an opportunity or a threat for advancement of women in tech? >> It's a much debated topic. I look at it as an opportunity for many of the reasons that you just said. First of all, let me say that when I was an operating executive and would try to create an environment on my team that was family supportive, I would do that equally for young or, you know, early to mid-career women as I did for early to mid-career men. And the reason is I needed those men, you know, chances are they had a working spouse at home, right? I needed them to be able to share the load. It's just as important to the women that companies give, you know, the partner, male or female, the partner support and the ability to share the love, right? So to me it's not just a woman thing. It's women and men, and I always tried to create the environment where it was okay to go to your soccer game. I knew you would be online later in the evening when the kids were in bed, and that was fine. And I think the pandemic has democratized that and made that, you know, made that kind of an everyday occurrence. >> Yeah the baby walks in. They're in the zoom call. The dog comes in. The leaf blower going on the outside the window. I've seen it all on theCUBE. >> Yeah, and people don't try to pretend anymore that like, you know, the house is clean, the dog's behaved, you know, I mean it's just, it's just real, and it's authentic, and I think that's healthy. >> Yeah. >> I do, you know, I also love, I also love the office, and you know, I've got a 31 year old and a soon to be 27 year old daughter, two daughters. And you know, they love going into the office, and I think about when I was their age, how just charged up I would get from being in the office. I also see how great it is for them to have a couple of days a week at home because you can get a few things done in between Zoom calls that you don't have to end up piling onto the weekend, and, you know, so I think it's a really healthy, I think it's a really healthy mix now. Most tech companies are not mandating five days in. Most tech companies are at two to three days in. I think that's a, I think that's a really good combination. >> It's interesting how people are changing their culture to get together more as groups and even events. I mean, while I got you, I might as well ask you, what's the board conversations around, you know, the old conferences? You know, before the pandemic, every company had like a user conference. Right, now it's like, well, do we really need to have that? Maybe we do smaller, and we do digital. Have you seen how companies are handling the in-person? Because there's where the relationships are really formed face-to-face, but not everyone's going to be going. But now certain it's clearly back to face-to-face. We're seeing that with theCUBE as you know. >> Yeah, yeah. >> But the numbers aren't coming back, and the numbers aren't that high, but the stakeholders. >> Yeah. >> And the numbers are actually higher if you count digital. >> Yeah, absolutely. But you know, also on digital there's fatigue from 100% digital, right? It's a hybrid. People don't want to be 100% digital anymore, but they also don't want to go back to the days when everybody got on a plane for every meeting, every call, every sales call. You know, I'm seeing a mix on user conferences. I would say two-thirds of my companies are back, but not at the expense level that they were on user conferences. We spend a lot of time getting updates on, cause nobody has put, interestingly enough, nobody has put T&E, travel and expense back to pre-pandemic levels. Nobody, so everybody's pulled back on number of trips. You know, marketing events are being very scrutinized, but I think very effective. We're doing a lot of, and, you know, these were part of the old model as well, like some things, some things just recycle, but you know, there's a lot of CIO and customer round tables in regional cities. You know, those are quite effective right now because people want some face-to-face, but they don't necessarily want to get on a plane and go to Las Vegas in order to do it. I mean, some of them are, you know, there are a lot of things back in Las Vegas. >> And think about the meetings that when you were an operating executive. You got to go to the sales kickoff, you got to go to this, go to that. There were mandatory face-to-faces that you had to go to, but there was a lot of travel that you probably could have done on Zoom. >> Oh, a lot, I mean. >> And then the productivity to the family impact too. Again, think about again, we're talking about the family and people's personal lives, right? So, you know, got to meet a customer. All right. Salesperson wants you to get in front of a customer, got to fly to New York, take a red eye, come on back. Like, I mean, that's gone. >> Yeah, and oh, by the way, the customer doesn't necessarily want to be in the office that day, so, you know, they may or may not be happy about that. So again, it's and not or, right? It's a mix. And I think it's great to see people back to some face-to-face. It's great to see marketing and events back to some face-to-face. It's also great to see that it hasn't gone back to the level it was. I think that's a really healthy dynamic. >> Well, I'll tell you that from our experience while we're on the topic, we'll move back to the International Women's Day is that the productivity of digital, this program we're doing is going to be streamed. We couldn't do this face-to-face because we had to have everyone fly to an event. We're going to do hundreds of stories that we couldn't have done. We're doing it remote. Because it's better to get the content than not have it. I mean it's offline, so, but it's not about getting people to the event and watch the screen for seven hours. It's pick your interview, and then engage. >> Yeah. >> So it's self-service. So we're seeing a lot, the new user experience kind of direct to consumer, and so I think there will be an, I think there's going to be a digital first class citizen with events, so that that matches up with the kind of experience, but the offline version. Face-to-face optimized for relationships, and that's where the recruiting gets done. That's where, you know, people can build these relationships with each other. >> Yeah, and it can be asynchronous. I think that's a real value proposition. It's a great point. >> Okay, I want to get, I want to get into the technology side of the education and re-skilling and those things. I remember in the 80s, computer science was software engineering. You learned like nine languages. You took some double E courses, one or two, and all the other kind of gut classes in school. Engineering, you had the four class disciplines and some offshoots of specialization. Now it's incredible the diversity of tracks in all engineering programs and computer science and outside of those departments. >> Yeah. >> Can you speak to the importance of STEM and the diversity in the technology industry and how this brings opportunity to lower the bar to get in and how people can stay in and grow and keep leveling up? >> Yeah, well look, we're constantly working on how to, how to help the incoming funnel. But then, you know, at a university level, I'm on the foundation board of Kansas State where I got my engineering degree. I was also Chairman of the National Action Council for Minorities in Engineering, which was all about diversity in STEM and how do you keep that pipeline going because honestly the US needs more tech resources than we have. And if you don't tap into the diversity of our entire workforce, we won't be able to fill that need. And so we focused a lot on both the funnel, right, that starts at the middle school level, particularly for girls, getting them in, you know, the situation of hands-on comfort level with coding, with robot building, you know, whatever gives them that confidence. And then keeping that going all the way into, you know, university program, and making sure that they don't attrit out, right? And so there's a number of initiatives, whether it's mentoring and support groups and financial aid to make sure that underrepresented minorities, women and other minorities, you know, get through the funnel and stay, you know, stay in. >> Got it. Now let me ask you, you said, I have two daughters. You have a family of girls too. Is there a vibe difference between the new generation and what's the trends that you're seeing in this next early wave? I mean, not maybe, I don't know how this is in middle school, but like as people start getting into their adult lives, college and beyond what's the current point of view, posture, makeup of the talent coming in? >> Yeah, yeah. >> Certain orientations, do you see any patterns? What's your observation? >> Yeah, it's interesting. So if I look at electrical engineering, my major, it's, and if I look at Kansas State, which spends a lot of time on this, and I think does a great job, but the diversity of that as a major has not changed dramatically since I was there in the early 80s. Where it has changed very significantly is computer science. There are many, many university and college programs around the country where, you know, it's 50/50 in computer science from a gender mix perspective, which is huge progress. Huge progress. And so, and to me that's, you know, I think CS is a fantastic degree for tech, regardless of what function you actually end up doing in these companies. I mean, I was an electrical engineer. I never did core electrical engineering work. I went right into sales and marketing and general management roles. So I think, I think a bunch of, you know, diverse CS graduates is a really, really good sign. And you know, we need to continue to push on that, but progress has been made. I think the, you know, it kind of goes back to the thing we were just talking about, which is the attrition of those, let's just talk about women, right? The attrition of those women once they got past early career and into mid-career then was a concern, right? And that goes back to, you know, just the inability to, you know, get it all done. And that I am hopeful is going to be better served now. >> Well, Sue, it's great to have you on. I know you're super busy. I appreciate you taking the time and contributing to our program on corporate board membership and some of your story and observations and opinions and analysis. Always great to have you and call you a contributor for theCUBE. You can jump on on one more board, be one of our board contributors for our analysts. (Sue laughing) >> I'm at capacity. (both laughing) >> Final, final word. What's the big seat at the table issue that's going well and areas that need to be improved? >> So I'll speak for my boards because they have made great progress in efficiency. You know, obviously with interest rates going up and the mix between growth and profitability changing in terms of what investors are looking for. Many, many companies have had to do a hard pivot from grow at all costs to healthy balance of growth and profit. And I'm very pleased with how my companies have made that pivot. And I think that is going to make much better companies as a result. I think diversity is something that has not been solved at the corporate level, and we need to keep working it. >> Awesome. Thank you for coming on theCUBE. CUBE alumni now contributor, on multiple boards, full-time job. Love the new challenge and chapter you're on, Sue. We'll be following, and we'll check in for more updates. And thank you for being a contributor on this program this year and this episode. We're going to be doing more of these quarterly, so we're going to move beyond once a year. >> That's great. (cross talking) It's always good to see you, John. >> Thank you. >> Thanks very much. >> Okay. >> Sue: Talk to you later. >> This is theCUBE coverage of IWD, International Women's Day 2023. I'm John Furrier, your host. Thanks for watching. (upbeat music)

(gentle music) >> Hello everyone. Welcome to this CUBE conversation here in Palo Alto, California. I'm John Furrier, host of theCUBE. We got a great remote guest coming in. Joseph Nelson, co-founder and CEO of RoboFlow hot startup in AI, computer vision. Really interesting topic in this wave of AI next gen hitting. Joseph, thanks for coming on this CUBE conversation. >> Thanks for having me. >> Yeah, I love the startup tsunami that's happening here in this wave. RoboFlow, you're in the middle of it. Exciting opportunities, you guys are in the cutting edge. I think computer vision's been talked about more as just as much as the large language models and these foundational models are merging. You're in the middle of it. What's it like right now as a startup and growing in this new wave hitting? >> It's kind of funny, it's, you know, I kind of describe it like sometimes you're in a garden of gnomes. It's like we feel like we've got this giant headstart with hundreds of thousands of people building with computer vision, training their own models, but that's a fraction of what it's going to be in six months, 12 months, 24 months. So, as you described it, a wave is a good way to think about it. And the wave is still building before it gets to its full size. So it's a ton of fun. >> Yeah, I think it's one of the most exciting areas in computer science. I wish I was in my twenties again, because I would be all over this. It's the intersection, there's so many disciplines, right? It's not just tech computer science, it's computer science, it's systems, it's software, it's data. There's so much aperture of things going on around your world. So, I mean, you got to be batting all the students away kind of trying to get hired in there, probably. I can only imagine you're hiring regiment. I'll ask that later, but first talk about what the company is that you're doing. How it's positioned, what's the market you're going after, and what's the origination story? How did you guys get here? How did you just say, hey, want to do this? What was the origination story? What do you do and how did you start the company? >> Yeah, yeah. I'll give you the what we do today and then I'll shift into the origin. RoboFlow builds tools for making the world programmable. Like anything that you see should be read write access if you think about it with a programmer's mind or legible. And computer vision is a technology that enables software to be added to these real world objects that we see. And so any sort of interface, any sort of object, any sort of scene, we can interact with it, we can make it more efficient, we can make it more entertaining by adding the ability for the tools that we use and the software that we write to understand those objects. And at RoboFlow, we've empowered a little over a hundred thousand developers, including those in half the Fortune 100 so far in that mission. Whether that's Walmart understanding the retail in their stores, Cardinal Health understanding the ways that they're helping their patients, or even electric vehicle manufacturers ensuring that they're making the right stuff at the right time. As you mentioned, it's early. Like I think maybe computer vision has touched one, maybe 2% of the whole economy and it'll be like everything in a very short period of time. And so we're focused on enabling that transformation. I think it's it, as far as I think about it, I've been fortunate to start companies before, start, sell these sorts of things. This is the last company I ever wanted to start and I think it will be, should we do it right, the world's largest in riding the wave of bringing together the disparate pieces of that technology. >> What was the motivating point of the formation? Was it, you know, you guys were hanging around? Was there some catalyst? What was the moment where it all kind of came together for you? >> You know what's funny is my co-founder, Brad and I, we were making computer vision apps for making board games more fun to play. So in 2017, Apple released AR kit, augmented reality kit for building augmented reality applications. And Brad and I are both sort of like hacker persona types. We feel like we don't really understand the technology until we build something with it and so we decided that we should make an app that if you point your phone at a Sudoku puzzle, it understands the state of the board and then it kind of magically fills in that experience with all the digits in real time, which totally ruins the game of Sudoku to be clear. But it also just creates this like aha moment of like, oh wow, like the ability for our pocket devices to understand and see the world as good or better than we can is possible. And so, you know, we actually did that as I mentioned in 2017, and the app went viral. It was, you know, top of some subreddits, top of Injure, Reddit, the hacker community as well as Product Hunt really liked it. So it actually won Product Hunt AR app of the year, which was the same year that the Tesla model three won the product of the year. So we joked that we share an award with Elon our shared (indistinct) But frankly, so that was 2017. RoboFlow wasn't incorporated as a business until 2019. And so, you know, when we made Magic Sudoku, I was running a different company at the time, Brad was running a different company at the time, and we kind of just put it out there and were excited by how many people liked it. And we assumed that other curious developers would see this inevitable future of, oh wow, you know. This is much more than just a pedestrian point your phone at a board game. This is everything can be seen and understood and rewritten in a different way. Things like, you know, maybe your fridge. Knowing what ingredients you have and suggesting recipes or auto ordering for you, or we were talking about some retail use cases of automated checkout. Like anything can be seen and observed and we presume that that would kick off a Cambrian explosion of applications. It didn't. So you fast forward to 2019, we said, well we might as well be the guys to start to tackle this sort of problem. And because of our success with board games before, we returned to making more board game solving applications. So we made one that solves Boggle, you know, the four by four word game, we made one that solves chess, you point your phone at a chess board and it understands the state of the board and then can make move recommendations. And each additional board game that we added, we realized that the tooling was really immature. The process of collecting images, knowing which images are actually going to be useful for improving model performance, training those models, deploying those models. And if we really wanted to make the world programmable, developers waiting for us to make an app for their thing of interest is a lot less efficient, less impactful than taking our tool chain and releasing that externally. And so, that's what RoboFlow became. RoboFlow became the internal tools that we used to make these game changing applications readily available. And as you know, when you give developers new tools, they create new billion dollar industries, let alone all sorts of fun hobbyist projects along the way. >> I love that story. Curious, inventive, little radical. Let's break the rules, see how we can push the envelope on the board games. That's how companies get started. It's a great story. I got to ask you, okay, what happens next? Now, okay, you realize this new tooling, but this is like how companies get built. Like they solve their own problem that they had 'cause they realized there's one, but then there has to be a market for it. So you actually guys knew that this was coming around the corner. So okay, you got your hacker mentality, you did that thing, you got the award and now you're like, okay, wow. Were you guys conscious of the wave coming? Was it one of those things where you said, look, if we do this, we solve our own problem, this will be big for everybody. Did you have that moment? Was that in 2019 or was that more of like, it kind of was obvious to you guys? >> Absolutely. I mean Brad puts this pretty effectively where he describes how we lived through the initial internet revolution, but we were kind of too young to really recognize and comprehend what was happening at the time. And then mobile happened and we were working on different companies that were not in the mobile space. And computer vision feels like the wave that we've caught. Like, this is a technology and capability that rewrites how we interact with the world, how everyone will interact with the world. And so we feel we've been kind of lucky this time, right place, right time of every enterprise will have the ability to improve their operations with computer vision. And so we've been very cognizant of the fact that computer vision is one of those groundbreaking technologies that every company will have as a part of their products and services and offerings, and we can provide the tooling to accelerate that future. >> Yeah, and the developer angle, by the way, I love that because I think, you know, as we've been saying in theCUBE all the time, developer's the new defacto standard bodies because what they adopt is pure, you know, meritocracy. And they pick the best. If it's sell service and it's good and it's got open source community around it, its all in. And they'll vote. They'll vote with their code and that is clear. Now I got to ask you, as you look at the market, we were just having this conversation on theCUBE in Barcelona at recent Mobile World Congress, now called MWC, around 5G versus wifi. And the debate was specifically computer vision, like facial recognition. We were talking about how the Cleveland Browns were using facial recognition for people coming into the stadium they were using it for ships in international ports. So the question was 5G versus wifi. My question is what infrastructure or what are the areas that need to be in place to make computer vision work? If you have developers building apps, apps got to run on stuff. So how do you sort that out in your mind? What's your reaction to that? >> A lot of the times when we see applications that need to run in real time and on video, they'll actually run at the edge without internet. And so a lot of our users will actually take their models and run it in a fully offline environment. Now to act on that information, you'll often need to have internet signal at some point 'cause you'll need to know how many people were in the stadium or what shipping crates are in my port at this point in time. You'll need to relay that information somewhere else, which will require connectivity. But actually using the model and creating the insights at the edge does not require internet. I mean we have users that deploy models on underwater submarines just as much as in outer space actually. And those are not very friendly environments to internet, let alone 5g. And so what you do is you use an edge device, like an Nvidia Jetson is common, mobile devices are common. Intel has some strong edge devices, the Movidius family of chips for example. And you use that compute that runs completely offline in real time to process those signals. Now again, what you do with those signals may require connectivity and that becomes a question of the problem you're solving of how soon you need to relay that information to another place. >> So, that's an architectural issue on the infrastructure. If you're a tactical edge war fighter for instance, you might want to have highly available and maybe high availability. I mean, these are words that mean something. You got storage, but it's not at the edge in real time. But you can trickle it back and pull it down. That's management. So that's more of a business by business decision or environment, right? >> That's right, that's right. Yeah. So I mean we can talk through some specifics. So for example, the RoboFlow actually powers the broadcaster that does the tennis ball tracking at Wimbledon. That runs completely at the edge in real time in, you know, technically to track the tennis ball and point the camera, you actually don't need internet. Now they do have internet of course to do the broadcasting and relay the signal and feeds and these sorts of things. And so that's a case where you have both edge deployment of running the model and high availability act on that model. We have other instances where customers will run their models on drones and the drone will go and do a flight and it'll say, you know, this many residential homes are in this given area, or this many cargo containers are in this given shipping yard. Or maybe we saw these environmental considerations of soil erosion along this riverbank. The model in that case can run on the drone during flight without internet, but then you only need internet once the drone lands and you're going to act on that information because for example, if you're doing like a study of soil erosion, you don't need to be real time. You just need to be able to process and make use of that information once the drone finishes its flight. >> Well I can imagine a zillion use cases. I heard of a use case interview at a company that does computer vision to help people see if anyone's jumping the fence on their company. Like, they know what a body looks like climbing a fence and they can spot it. Pretty easy use case compared to probably some of the other things, but this is the horizontal use cases, its so many use cases. So how do you guys talk to the marketplace when you say, hey, we have generative AI for commuter vision. You might know language models that's completely different animal because vision's like the world, right? So you got a lot more to do. What's the difference? How do you explain that to customers? What can I build and what's their reaction? >> Because we're such a developer centric company, developers are usually creative and show you the ways that they want to take advantage of new technologies. I mean, we've had people use things for identifying conveyor belt debris, doing gas leak detection, measuring the size of fish, airplane maintenance. We even had someone that like a hobby use case where they did like a specific sushi identifier. I dunno if you know this, but there's a specific type of whitefish that if you grew up in the western hemisphere and you eat it in the eastern hemisphere, you get very sick. And so there was someone that made an app that tells you if you happen to have that fish in the sushi that you're eating. But security camera analysis, transportation flows, plant disease detection, really, you know, smarter cities. We have people that are doing curb management identifying, and a lot of these use cases, the fantastic thing about building tools for developers is they're a creative bunch and they have these ideas that if you and I sat down for 15 minutes and said, let's guess every way computer vision can be used, we would need weeks to list all the example use cases. >> We'd miss everything. >> And we'd miss. And so having the community show us the ways that they're using computer vision is impactful. Now that said, there are of course commercial industries that have discovered the value and been able to be out of the gate. And that's where we have the Fortune 100 customers, like we do. Like the retail customers in the Walmart sector, healthcare providers like Medtronic, or vehicle manufacturers like Rivian who all have very difficult either supply chain, quality assurance, in stock, out of stock, anti-theft protection considerations that require successfully making sense of the real world. >> Let me ask you a question. This is maybe a little bit in the weeds, but it's more developer focused. What are some of the developer profiles that you're seeing right now in terms of low-hanging fruit applications? And can you talk about the academic impact? Because I imagine if I was in school right now, I'd be all over it. Are you seeing Master's thesis' being worked on with some of your stuff? Is the uptake in both areas of younger pre-graduates? And then inside the workforce, What are some of the devs like? Can you share just either what their makeup is, what they work on, give a little insight into the devs you're working with. >> Leading developers that want to be on state-of-the-art technology build with RoboFlow because they know they can use the best in class open source. They know that they can get the most out of their data. They know that they can deploy extremely quickly. That's true among students as you mentioned, just as much as as industries. So we welcome students and I mean, we have research grants that will regularly support for people to publish. I mean we actually have a channel inside our internal slack where every day, more student publications that cite building with RoboFlow pop up. And so, that helps inspire some of the use cases. Now what's interesting is that the use case is relatively, you know, useful or applicable for the business or the student. In other words, if a student does a thesis on how to do, we'll say like shingle damage detection from satellite imagery and they're just doing that as a master's thesis, in fact most insurance businesses would be interested in that sort of application. So, that's kind of how we see uptick and adoption both among researchers who want to be on the cutting edge and publish, both with RoboFlow and making use of open source tools in tandem with the tool that we provide, just as much as industry. And you know, I'm a big believer in the philosophy that kind of like what the hackers are doing nights and weekends, the Fortune 500 are doing in a pretty short order period of time and we're experiencing that transition. Computer vision used to be, you know, kind of like a PhD, multi-year investment endeavor. And now with some of the tooling that we're working on in open source technologies and the compute that's available, these science fiction ideas are possible in an afternoon. And so you have this idea of maybe doing asset management or the aerial observation of your shingles or things like this. You have a few hundred images and you can de-risk whether that's possible for your business today. So there's pretty broad-based adoption among both researchers that want to be on the state of the art, as much as companies that want to reduce the time to value. >> You know, Joseph, you guys and your partner have got a great front row seat, ground floor, presented creation wave here. I'm seeing a pattern emerging from all my conversations on theCUBE with founders that are successful, like yourselves, that there's two kind of real things going on. You got the enterprises grabbing the products and retrofitting into their legacy and rebuilding their business. And then you have startups coming out of the woodwork. Young, seeing greenfield or pick a specific niche or focus and making that the signature lever to move the market. >> That's right. >> So can you share your thoughts on the startup scene, other founders out there and talk about that? And then I have a couple questions for like the enterprises, the old school, the existing legacy. Little slower, but the startups are moving fast. What are some of the things you're seeing as startups are emerging in this field? >> I think you make a great point that independent of RoboFlow, very successful, especially developer focused businesses, kind of have three customer types. You have the startups and maybe like series A, series B startups that you're building a product as fast as you can to keep up with them, and they're really moving just as fast as as you are and pulling the product out at you for things that they need. The second segment that you have might be, call it SMB but not enterprise, who are able to purchase and aren't, you know, as fast of moving, but are stable and getting value and able to get to production. And then the third type is enterprise, and that's where you have typically larger contract value sizes, slower moving in terms of adoption and feedback for your product. And I think what you see is that successful companies balance having those three customer personas because you have the small startups, small fast moving upstarts that are discerning buyers who know the market and elect to build on tooling that is best in class. And so you basically kind of pass the smell test of companies who are quite discerning in their purchases, plus are moving so quick they're pulling their product out of you. Concurrently, you have a product that's enterprise ready to service the scalability, availability, and trust of enterprise buyers. And that's ultimately where a lot of companies will see tremendous commercial success. I mean I remember seeing the Twilio IPO, Uber being like a full 20% of their revenue, right? And so there's this very common pattern where you have the ability to find some of those upstarts that you make bets on, like the next Ubers of the world, the smaller companies that continue to get developed with the product and then the enterprise whom allows you to really fund the commercial success of the business, and validate the size of the opportunity in market that's being creative. >> It's interesting, there's so many things happening there. It's like, in a way it's a new category, but it's not a new category. It becomes a new category because of the capabilities, right? So, it's really interesting, 'cause that's what you're talking about is a category, creating. >> I think developer tools. So people often talk about B to B and B to C businesses. I think developer tools are in some ways a third way. I mean ultimately they're B to B, you're selling to other businesses and that's where your revenue's coming from. However, you look kind of like a B to C company in the ways that you measure product adoption and kind of go to market. In other words, you know, we're often tracking the leading indicators of commercial success in the form of usage, adoption, retention. Really consumer app, traditionally based metrics of how to know you're building the right stuff, and that's what product led growth companies do. And then you ultimately have commercial traction in a B to B way. And I think that that actually kind of looks like a third thing, right? Like you can do these sort of funny zany marketing examples that you might see historically from consumer businesses, but yet you ultimately make your money from the enterprise who has these de-risked high value problems you can solve for them. And I selfishly think that that's the best of both worlds because I don't have to be like Evan Spiegel, guessing the next consumer trend or maybe creating the next consumer trend and catching lightning in a bottle over and over again on the consumer side. But I still get to have fun in our marketing and make sort of fun, like we're launching the world's largest game of rock paper scissors being played with computer vision, right? Like that's sort of like a fun thing you can do, but then you can concurrently have the commercial validation and customers telling you the things that they need to be built for them next to solve commercial pain points for them. So I really do think that you're right by calling this a new category and it really is the best of both worlds. >> It's a great call out, it's a great call out. In fact, I always juggle with the VC. I'm like, it's so easy. Your job is so easy to pick the winners. What are you talking about its so easy? I go, just watch what the developers jump on. And it's not about who started, it could be someone in the dorm room to the boardroom person. You don't know because that B to C, the C, it's B to D you know? You know it's developer 'cause that's a human right? That's a consumer of the tool which influences the business that never was there before. So I think this direct business model evolution, whether it's media going direct or going direct to the developers rather than going to a gatekeeper, this is the reality. >> That's right. >> Well I got to ask you while we got some time left to describe, I want to get into this topic of multi-modality, okay? And can you describe what that means in computer vision? And what's the state of the growth of that portion of this piece? >> Multi modality refers to using multiple traditionally siloed problem types, meaning text, image, video, audio. So you could treat an audio problem as only processing audio signal. That is not multimodal, but you could use the audio signal at the same time as a video feed. Now you're talking about multi modality. In computer vision, multi modality is predominantly happening with images and text. And one of the biggest releases in this space is actually two years old now, was clip, contrastive language image pre-training, which took 400 million image text pairs and basically instead of previously when you do classification, you basically map every single image to a single class, right? Like here's a bunch of images of chairs, here's a bunch of images of dogs. What clip did is used, you can think about it like, the class for an image being the Instagram caption for the image. So it's not one single thing. And by training on understanding the corpora, you basically see which words, which concepts are associated with which pixels. And this opens up the aperture for the types of problems and generalizability of models. So what does this mean? This means that you can get to value more quickly from an existing trained model, or at least validate that what you want to tackle with a computer vision, you can get there more quickly. It also opens up the, I mean. Clip has been the bedrock of some of the generative image techniques that have come to bear, just as much as some of the LLMs. And increasingly we're going to see more and more of multi modality being a theme simply because at its core, you're including more context into what you're trying to understand about the world. I mean, in its most basic sense, you could ask yourself, if I have an image, can I know more about that image with just the pixels? Or if I have the image and the sound of when that image was captured or it had someone describe what they see in that image when the image was captured, which one's going to be able to get you more signal? And so multi modality helps expand the ability for us to understand signal processing. >> Awesome. And can you just real quick, define clip for the folks that don't know what that means? >> Yeah. Clip is a model architecture, it's an acronym for contrastive language image pre-training and like, you know, model architectures that have come before it captures the almost like, models are kind of like brands. So I guess it's a brand of a model where you've done these 400 million image text pairs to match up which visual concepts are associated with which text concepts. And there have been new releases of clip, just at bigger sizes of bigger encoding's, of longer strings of texture, or larger image windows. But it's been a really exciting advancement that OpenAI released in January, 2021. >> All right, well great stuff. We got a couple minutes left. Just I want to get into more of a company-specific question around culture. All startups have, you know, some sort of cultural vibe. You know, Intel has Moore's law doubles every whatever, six months. What's your culture like at RoboFlow? I mean, if you had to describe that culture, obviously love the hacking story, you and your partner with the games going number one on Product Hunt next to Elon and Tesla and then hey, we should start a company two years later. That's kind of like a curious, inventing, building, hard charging, but laid back. That's my take. How would you describe the culture? >> I think that you're right. The culture that we have is one of shipping, making things. So every week each team shares what they did for our customers on a weekly basis. And we have such a strong emphasis on being better week over week that those sorts of things compound. So one big emphasis in our culture is getting things done, shipping, doing things for our customers. The second is we're an incredibly transparent place to work. For example, how we think about giving decisions, where we're progressing against our goals, what problems are biggest and most important for the company is all open information for those that are inside the company to know and progress against. The third thing that I'd use to describe our culture is one that thrives with autonomy. So RoboFlow has a number of individuals who have founded companies before, some of which have sold their businesses for a hundred million plus upon exit. And the way that we've been able to attract talent like that is because the problems that we're tackling are so immense, yet individuals are able to charge at it with the way that they think is best. And this is what pairs well with transparency. If you have a strong sense of what the company's goals are, how we're progressing against it, and you have this ownership mentality of what can I do to change or drive progress against that given outcome, then you create a really healthy pairing of, okay cool, here's where the company's progressing. Here's where things are going really well, here's the places that we most need to improve and work on. And if you're inside that company as someone who has a preponderance to be a self-starter and even a history of building entire functions or companies yourself, then you're going to be a place where you can really thrive. You have the inputs of the things where we need to work on to progress the company's goals. And you have the background of someone that is just necessarily a fast moving and ambitious type of individual. So I think the best way to describe it is a transparent place with autonomy and an emphasis on getting things done. >> Getting shit done as they say. Getting stuff done. Great stuff. Hey, final question. Put a plug out there for the company. What are you going to hire? What's your pipeline look like for people? What jobs are open? I'm sure you got hiring all around. Give a quick plug for the company what you're looking for. >> I appreciate you asking. Basically you're either building the product or helping customers be successful with the product. So in the building product category, we have platform engineering roles, machine learning engineering roles, and we're solving some of the hardest and most impactful problems of bringing such a groundbreaking technology to the masses. And so it's a great place to be where you can kind of be your own user as an engineer. And then if you're enabling people to be successful with the products, I mean you're working in a place where there's already such a strong community around it and you can help shape, foster, cultivate, activate, and drive commercial success in that community. So those are roles that tend themselves to being those that build the product for developer advocacy, those that are account executives that are enabling our customers to realize commercial success, and even hybrid roles like we call it field engineering, where you are a technical resource to drive success within customer accounts. And so all this is listed on roboflow.com/careers. And one thing that I actually kind of want to mention John that's kind of novel about the thing that's working at RoboFlow. So there's been a lot of discussion around remote companies and there's been a lot of discussion around in-person companies and do you need to be in the office? And one thing that we've kind of recognized is you can actually chart a third way. You can create a third way which we call satellite, which basically means people can work from where they most like to work and there's clusters of people, regular onsite's. And at RoboFlow everyone gets, for example, $2,500 a year that they can use to spend on visiting coworkers. And so what's sort of organically happened is team numbers have started to pull together these resources and rent out like, lavish Airbnbs for like a week and then everyone kind of like descends in and works together for a week and makes and creates things. And we call this lighthouses because you know, a lighthouse kind of brings ships into harbor and we have an emphasis on shipping. >> Yeah, quality people that are creative and doers and builders. You give 'em some cash and let the self-governing begin, you know? And like, creativity goes through the roof. It's a great story. I think that sums up the culture right there, Joseph. Thanks for sharing that and thanks for this great conversation. I really appreciate it and it's very inspiring. Thanks for coming on. >> Yeah, thanks for having me, John. >> Joseph Nelson, co-founder and CEO of RoboFlow. Hot company, great culture in the right place in a hot area, computer vision. This is going to explode in value. The edge is exploding. More use cases, more development, and developers are driving the change. Check out RoboFlow. This is theCUBE. I'm John Furrier, your host. Thanks for watching. (gentle music)

(bright upbeat music) >> Hello and welcome to this Cube Conversation. I'm John Furrier, host of theCUBE. We've got a great conversation featuring Arthur AI. I'm your host. I'm excited to have Adam Wenchel who's the Co-Founder and CEO. Thanks for joining us today, appreciate it. >> Yeah, thanks for having me on, John, looking forward to the conversation. >> I got to say, it's been an exciting world in AI or artificial intelligence. Just an explosion of interest kind of in the mainstream with the language models, which people don't really get, but they're seeing the benefits of some of the hype around OpenAI. Which kind of wakes everyone up to, "Oh, I get it now." And then of course the pessimism comes in, all the skeptics are out there. But this breakthrough in generative AI field is just awesome, it's really a shift, it's a wave. We've been calling it probably the biggest inflection point, then the others combined of what this can do from a surge standpoint, applications. I mean, all aspects of what we used to know is the computing industry, software industry, hardware, is completely going to get turbo. So we're totally obviously bullish on this thing. So, this is really interesting. So my first question is, I got to ask you, what's you guys taking? 'Cause you've been doing this, you're in it, and now all of a sudden you're at the beach where the big waves are. What's the explosion of interest is there? What are you seeing right now? >> Yeah, I mean, it's amazing, so for starters, I've been in AI for over 20 years and just seeing this amount of excitement and the growth, and like you said, the inflection point we've hit in the last six months has just been amazing. And, you know, what we're seeing is like people are getting applications into production using LLMs. I mean, really all this excitement just started a few months ago, with ChatGPT and other breakthroughs and the amount of activity and the amount of new systems that we're seeing hitting production already so soon after that is just unlike anything we've ever seen. So it's pretty awesome. And, you know, these language models are just, they could be applied in so many different business contexts and that it's just the amount of value that's being created is again, like unprecedented compared to anything. >> Adam, you know, you've been in this for a while, so it's an interesting point you're bringing up, and this is a good point. I was talking with my friend John Markoff, former New York Times journalist and he was talking about, there's been a lot of work been done on ethics. So there's been, it's not like it's new. It's like been, there's a lot of stuff that's been baking over many, many years and, you know, decades. So now everyone wakes up in the season, so I think that is a key point I want to get into some of your observations. But before we get into it, I want you to explain for the folks watching, just so we can kind of get a definition on the record. What's an LLM, what's a foundational model and what's generative ai? Can you just quickly explain the three things there? >> Yeah, absolutely. So an LLM or a large language model, it's just a large, they would imply a large language model that's been trained on a huge amount of data typically pulled from the internet. And it's a general purpose language model that can be built on top for all sorts of different things, that includes traditional NLP tasks like document classification and sentiment understanding. But the thing that's gotten people really excited is it's used for generative tasks. So, you know, asking it to summarize documents or asking it to answer questions. And these aren't new techniques, they've been around for a while, but what's changed is just this new class of models that's based on new architectures. They're just so much more capable that they've gone from sort of science projects to something that's actually incredibly useful in the real world. And there's a number of companies that are making them accessible to everyone so that you can build on top of them. So that's the other big thing is, this kind of access to these models that can power generative tasks has been democratized in the last few months and it's just opening up all these new possibilities. And then the third one you mentioned foundation models is sort of a broader term for the category that includes LLMs, but it's not just language models that are included. So we've actually seen this for a while in the computer vision world. So people have been building on top of computer vision models, pre-trained computer vision models for a while for image classification, object detection, that's something we've had customers doing for three or four years already. And so, you know, like you said, there are antecedents to like, everything that's happened, it's not entirely new, but it does feel like a step change. >> Yeah, I did ask ChatGPT to give me a riveting introduction to you and it gave me an interesting read. If we have time, I'll read it. It's kind of, it's fun, you get a kick out of it. "Ladies and gentlemen, today we're a privileged "to have Adam Wenchel, Founder of Arthur who's going to talk "about the exciting world of artificial intelligence." And then it goes on with some really riveting sentences. So if we have time, I'll share that, it's kind of funny. It was good. >> Okay. >> So anyway, this is what people see and this is why I think it's exciting 'cause I think people are going to start refactoring what they do. And I've been saying this on theCUBE now for about a couple months is that, you know, there's a scene in "Moneyball" where Billy Beane sits down with the Red Sox owner and the Red Sox owner says, "If people aren't rebuilding their teams on your model, "they're going to be dinosaurs." And it reminds me of what's happening right now. And I think everyone that I talk to in the business sphere is looking at this and they're connecting the dots and just saying, if we don't rebuild our business with this new wave, they're going to be out of business because there's so much efficiency, there's so much automation, not like DevOps automation, but like the generative tasks that will free up the intellect of people. Like just the simple things like do an intro or do this for me, write some code, write a countermeasure to a hack. I mean, this is kind of what people are doing. And you mentioned computer vision, again, another huge field where 5G things are coming on, it's going to accelerate. What do you say to people when they kind of are leaning towards that, I need to rethink my business? >> Yeah, it's 100% accurate and what's been amazing to watch the last few months is the speed at which, and the urgency that companies like Microsoft and Google or others are actually racing to, to do that rethinking of their business. And you know, those teams, those companies which are large and haven't always been the fastest moving companies are working around the clock. And the pace at which they're rolling out LLMs across their suite of products is just phenomenal to watch. And it's not just the big, the large tech companies as well, I mean, we're seeing the number of startups, like we get, every week a couple of new startups get in touch with us for help with their LLMs and you know, there's just a huge amount of venture capital flowing into it right now because everyone realizes the opportunities for transforming like legal and healthcare and content creation in all these different areas is just wide open. And so there's a massive gold rush going on right now, which is amazing. >> And the cloud scale, obviously horizontal scalability of the cloud brings us to another level. We've been seeing data infrastructure since the Hadoop days where big data was coined. Now you're seeing this kind of take fruit, now you have vertical specialization where data shines, large language models all of a set up perfectly for kind of this piece. And you know, as you mentioned, you've been doing it for a long time. Let's take a step back and I want to get into how you started the company, what drove you to start it? Because you know, as an entrepreneur you're probably saw this opportunity before other people like, "Hey, this is finally it, it's here." Can you share the origination story of what you guys came up with, how you started it, what was the motivation and take us through that origination story. >> Yeah, absolutely. So as I mentioned, I've been doing AI for many years. I started my career at DARPA, but it wasn't really until 2015, 2016, my previous company was acquired by Capital One. Then I started working there and shortly after I joined, I was asked to start their AI team and scale it up. And for the first time I was actually doing it, had production models that we were working with, that was at scale, right? And so there was hundreds of millions of dollars of business revenue and certainly a big group of customers who were impacted by the way these models acted. And so it got me hyper-aware of these issues of when you get models into production, it, you know. So I think people who are earlier in the AI maturity look at that as a finish line, but it's really just the beginning and there's this constant drive to make them better, make sure they're not degrading, make sure you can explain what they're doing, if they're impacting people, making sure they're not biased. And so at that time, there really weren't any tools to exist to do this, there wasn't open source, there wasn't anything. And so after a few years there, I really started talking to other people in the industry and there was a really clear theme that this needed to be addressed. And so, I joined with my Co-Founder John Dickerson, who was on the faculty in University of Maryland and he'd been doing a lot of research in these areas. And so we ended up joining up together and starting Arthur. >> Awesome. Well, let's get into what you guys do. Can you explain the value proposition? What are people using you for now? Where's the action? What's the customers look like? What do prospects look like? Obviously you mentioned production, this has been the theme. It's not like people woke up one day and said, "Hey, I'm going to put stuff into production." This has kind of been happening. There's been companies that have been doing this at scale and then yet there's a whole follower model coming on mainstream enterprise and businesses. So there's kind of the early adopters are there now in production. What do you guys do? I mean, 'cause I think about just driving the car off the lot is not, you got to manage operations. I mean, that's a big thing. So what do you guys do? Talk about the value proposition and how you guys make money? >> Yeah, so what we do is, listen, when you go to validate ahead of deploying these models in production, starts at that point, right? So you want to make sure that if you're going to be upgrading a model, if you're going to replacing one that's currently in production, that you've proven that it's going to perform well, that it's going to be perform ethically and that you can explain what it's doing. And then when you launch it into production, traditionally data scientists would spend 25, 30% of their time just manually checking in on their model day-to-day babysitting as we call it, just to make sure that the data hasn't drifted, the model performance hasn't degraded, that a programmer did make a change in an upstream data system. You know, there's all sorts of reasons why the world changes and that can have a real adverse effect on these models. And so what we do is bring the same kind of automation that you have for other kinds of, let's say infrastructure monitoring, application monitoring, we bring that to your AI systems. And that way if there ever is an issue, it's not like weeks or months till you find it and you find it before it has an effect on your P&L and your balance sheet, which is too often before they had tools like Arthur, that was the way they were detected. >> You know, I was talking to Swami at Amazon who I've known for a long time for 13 years and been on theCUBE multiple times and you know, I watched Amazon try to pick up that sting with stage maker about six years ago and so much has happened since then. And he and I were talking about this wave, and I kind of brought up this analogy to how when cloud started, it was, Hey, I don't need a data center. 'Cause when I did my startup that time when Amazon, one of my startups at that time, my choice was put a box in the colo, get all the configuration before I could write over the line of code. So the cloud became the benefit for that and you can stand up stuff quickly and then it grew from there. Here it's kind of the same dynamic, you don't want to have to provision a large language model or do all this heavy lifting. So that seeing companies coming out there saying, you can get started faster, there's like a new way to get it going. So it's kind of like the same vibe of limiting that heavy lifting. >> Absolutely. >> How do you look at that because this seems to be a wave that's going to be coming in and how do you guys help companies who are going to move quickly and start developing? >> Yeah, so I think in the race to this kind of gold rush mentality, race to get these models into production, there's starting to see more sort of examples and evidence that there are a lot of risks that go along with it. Either your model says things, your system says things that are just wrong, you know, whether it's hallucination or just making things up, there's lots of examples. If you go on Twitter and the news, you can read about those, as well as sort of times when there could be toxic content coming out of things like that. And so there's a lot of risks there that you need to think about and be thoughtful about when you're deploying these systems. But you know, you need to balance that with the business imperative of getting these things into production and really transforming your business. And so that's where we help people, we say go ahead, put them in production, but just make sure you have the right guardrails in place so that you can do it in a smart way that's going to reflect well on you and your company. >> Let's frame the challenge for the companies now that you have, obviously there's the people who doing large scale production and then you have companies maybe like as small as us who have large linguistic databases or transcripts for example, right? So what are customers doing and why are they deploying AI right now? And is it a speed game, is it a cost game? Why have some companies been able to deploy AI at such faster rates than others? And what's a best practice to onboard new customers? >> Yeah, absolutely. So I mean, we're seeing across a bunch of different verticals, there are leaders who have really kind of started to solve this puzzle about getting AI models into production quickly and being able to iterate on them quickly. And I think those are the ones that realize that imperative that you mentioned earlier about how transformational this technology is. And you know, a lot of times, even like the CEOs or the boards are very personally kind of driving this sense of urgency around it. And so, you know, that creates a lot of movement, right? And so those companies have put in place really smart infrastructure and rails so that people can, data scientists aren't encumbered by having to like hunt down data, get access to it. They're not encumbered by having to stand up new platforms every time they want to deploy an AI system, but that stuff is already in place. There's a really nice ecosystem of products out there, including Arthur, that you can tap into. Compared to five or six years ago when I was building at a top 10 US bank, at that point you really had to build almost everything yourself and that's not the case now. And so it's really nice to have things like, you know, you mentioned AWS SageMaker and a whole host of other tools that can really accelerate things. >> What's your profile customer? Is it someone who already has a team or can people who are learning just dial into the service? What's the persona? What's the pitch, if you will, how do you align with that customer value proposition? Do people have to be built out with a team and in play or is it pre-production or can you start with people who are just getting going? >> Yeah, people do start using it pre-production for validation, but I think a lot of our customers do have a team going and they're starting to put, either close to putting something into production or about to, it's everything from large enterprises that have really sort of complicated, they have dozens of models running all over doing all sorts of use cases to tech startups that are very focused on a single problem, but that's like the lifeblood of the company and so they need to guarantee that it works well. And you know, we make it really easy to get started, especially if you're using one of the common model development platforms, you can just kind of turn key, get going and make sure that you have a nice feedback loop. So then when your models are out there, it's pointing out, areas where it's performing well, areas where it's performing less well, giving you that feedback so that you can make improvements, whether it's in training data or futurization work or algorithm selection. There's a number of, you know, depending on the symptoms, there's a number of things you can do to increase performance over time and we help guide people on that journey. >> So Adam, I have to ask, since you have such a great customer base and they're smart and they got teams and you're on the front end, I mean, early adopters is kind of an overused word, but they're killing it. They're putting stuff in the production's, not like it's a test, it's not like it's early. So as the next wave comes of fast followers, how do you see that coming online? What's your vision for that? How do you see companies that are like just waking up out of the frozen, you know, freeze of like old IT to like, okay, they got cloud, but they're not yet there. What do you see in the market? I see you're in the front end now with the top people really nailing AI and working hard. What's the- >> Yeah, I think a lot of these tools are becoming, or every year they get easier, more accessible, easier to use. And so, you know, even for that kind of like, as the market broadens, it takes less and less of a lift to put these systems in place. And the thing is, every business is unique, they have their own kind of data and so you can use these foundation models which have just been trained on generic data. They're a great starting point, a great accelerant, but then, in most cases you're either going to want to create a model or fine tune a model using data that's really kind of comes from your particular customers, the people you serve and so that it really reflects that and takes that into account. And so I do think that these, like the size of that market is expanding and its broadening as these tools just become easier to use and also the knowledge about how to build these systems becomes more widespread. >> Talk about your customer base you have now, what's the makeup, what size are they? Give a taste a little bit of a customer base you got there, what's they look like? I'll say Capital One, we know very well while you were at there, they were large scale, lot of data from fraud detection to all kinds of cool stuff. What do your customers now look like? >> Yeah, so we have a variety, but I would say one area we're really strong, we have several of the top 10 US banks, that's not surprising, that's a strength for us, but we also have Fortune 100 customers in healthcare, in manufacturing, in retail, in semiconductor and electronics. So what we find is like in any sort of these major verticals, there's typically, you know, one, two, three kind of companies that are really leading the charge and are the ones that, you know, in our opinion, those are the ones that for the next multiple decades are going to be the leaders, the ones that really kind of lead the charge on this AI transformation. And so we're very fortunate to be working with some of those. And then we have a number of startups as well who we love working with just because they're really pushing the boundaries technologically and so they provide great feedback and make sure that we're continuing to innovate and staying abreast of everything that's going on. >> You know, these early markups, even when the hyperscalers were coming online, they had to build everything themselves. That's the new, they're like the alphas out there building it. This is going to be a big wave again as that fast follower comes in. And so when you look at the scale, what advice would you give folks out there right now who want to tee it up and what's your secret sauce that will help them get there? >> Yeah, I think that the secret to teeing it up is just dive in and start like the, I think these are, there's not really a secret. I think it's amazing how accessible these are. I mean, there's all sorts of ways to access LLMs either via either API access or downloadable in some cases. And so, you know, go ahead and get started. And then our secret sauce really is the way that we provide that performance analysis of what's going on, right? So we can tell you in a very actionable way, like, hey, here's where your model is doing good things, here's where it's doing bad things. Here's something you want to take a look at, here's some potential remedies for it. We can help guide you through that. And that way when you're putting it out there, A, you're avoiding a lot of the common pitfalls that people see and B, you're able to really kind of make it better in a much faster way with that tight feedback loop. >> It's interesting, we've been kind of riffing on this supercloud idea because it was just different name than multicloud and you see apps like Snowflake built on top of AWS without even spending any CapEx, you just ride that cloud wave. This next AI, super AI wave is coming. I don't want to call AIOps because I think there's a different distinction. If you, MLOps and AIOps seem a little bit old, almost a few years back, how do you view that because everyone's is like, "Is this AIOps?" And like, "No, not kind of, but not really." How would you, you know, when someone says, just shoots off the hip, "Hey Adam, aren't you doing AIOps?" Do you say, yes we are, do you say, yes, but we do differently because it's doesn't seem like it's the same old AIOps. What's your- >> Yeah, it's a good question. AIOps has been a term that was co-opted for other things and MLOps also has people have used it for different meanings. So I like the term just AI infrastructure, I think it kind of like describes it really well and succinctly. >> But you guys are doing the ops. I mean that's the kind of ironic thing, it's like the next level, it's like NextGen ops, but it's not, you don't want to be put in that bucket. >> Yeah, no, it's very operationally focused platform that we have, I mean, it fires alerts, people can action off them. If you're familiar with like the way people run security operations centers or network operations centers, we do that for data science, right? So think of it as a DSOC, a Data Science Operations Center where all your models, you might have hundreds of models running across your organization, you may have five, but as problems are detected, alerts can be fired and you can actually work the case, make sure they're resolved, escalate them as necessary. And so there is a very strong operational aspect to it, you're right. >> You know, one of the things I think is interesting is, is that, if you don't mind commenting on it, is that the aspect of scale is huge and it feels like that was made up and now you have scale and production. What's your reaction to that when people say, how does scale impact this? >> Yeah, scale is huge for some of, you know, I think, I think look, the highest leverage business areas to apply these to, are generally going to be the ones at the biggest scale, right? And I think that's one of the advantages we have. Several of us come from enterprise backgrounds and we're used to doing things enterprise grade at scale and so, you know, we're seeing more and more companies, I think they started out deploying AI and sort of, you know, important but not necessarily like the crown jewel area of their business, but now they're deploying AI right in the heart of things and yeah, the scale that some of our companies are operating at is pretty impressive. >> John: Well, super exciting, great to have you on and congratulations. I got a final question for you, just random. What are you most excited about right now? Because I mean, you got to be pretty pumped right now with the way the world is going and again, I think this is just the beginning. What's your personal view? How do you feel right now? >> Yeah, the thing I'm really excited about for the next couple years now, you touched on it a little bit earlier, but is a sort of convergence of AI and AI systems with sort of turning into AI native businesses. And so, as you sort of do more, get good further along this transformation curve with AI, it turns out that like the better the performance of your AI systems, the better the performance of your business. Because these models are really starting to underpin all these key areas that cumulatively drive your P&L. And so one of the things that we work a lot with our customers is to do is just understand, you know, take these really esoteric data science notions and performance and tie them to all their business KPIs so that way you really are, it's kind of like the operating system for running your AI native business. And we're starting to see more and more companies get farther along that maturity curve and starting to think that way, which is really exciting. >> I love the AI native. I haven't heard any startup yet say AI first, although we kind of use the term, but I guarantee that's going to come in all the pitch decks, we're an AI first company, it's going to be great run. Adam, congratulations on your success to you and the team. Hey, if we do a few more interviews, we'll get the linguistics down. We can have bots just interact with you directly and ask you, have an interview directly. >> That sounds good, I'm going to go hang out on the beach, right? So, sounds good. >> Thanks for coming on, really appreciate the conversation. Super exciting, really important area and you guys doing great work. Thanks for coming on. >> Adam: Yeah, thanks John. >> Again, this is Cube Conversation. I'm John Furrier here in Palo Alto, AI going next gen. This is legit, this is going to a whole nother level that's going to open up huge opportunities for startups, that's going to use opportunities for investors and the value to the users and the experience will come in, in ways I think no one will ever see. So keep an eye out for more coverage on siliconangle.com and theCUBE.net, thanks for watching. (bright upbeat music)

>> Narrator: TheCUBE's live coverage is made possible by funding from Dell Technologies. Creating technologies that drive human progress. (light airy music) >> Hi, everybody, welcome back to the Fira in Barcelona. My name is Dave Vellante. I'm here with my co-host Dave Nicholson. Lisa Martin is in the house. John Furrier is pounding the news from our Palo Alto studio. We are super excited to be talking about cloud at the edge, what that means. Charles Tsai is here. He's the Senior Director of product management at Dell Technologies and Peter Fetterolf is the Chief Technology Officer at ACG Business Analytics, a firm that goes deep into the TCO and the telco space, among other things. Gents, welcome to theCUBE. Thanks for coming on. Thank you. >> Good to be here. >> Yeah, good to be here. >> So I've been in search all week of the elusive next wave of monetization for the telcos. We know they make great money on connectivity, they're really good at that. But they're all talking about how they can't let this happen again. Meaning we can't let the over the top vendors yet again, basically steal our cookies. So we're going to not mess it up this time. We're going to win in the monetization. Charles, where are those monetization opportunities? Obviously at the edge, the telco cloud at the edge. What is that all about and where's the money? >> Well, Dave, I think from a Dell's perspective, what we want to be able to enable operators is a solution that enable them to roll out services much quicker, right? We know there's a lot of innovation around IoT, MEG and so on and so forth, but they continue to rely on traditional technology and way of operations is going to take them years to enable new services. So what Dell is doing is now, creating the entire vertical stack from the hardware through CAST and automation that enable them, not only to push out services very quickly, but operating them using cloud principles. >> So it's when you say the entire vertical stack, it's the integrated hardware components with like, for example, Red Hat on top- >> Right. >> Or a Wind River? >> That's correct. >> Okay, and then open API, so the developers can create workloads, I presume data companies. We just had a data conversation 'cause that was part of the original stack- >> That's correct. >> So through an open ecosystem, you can actually sort of recreate that value, correct? >> That's correct. >> Okay. >> So one thing Dell is doing, is we are offering an infrastructure block where we are taking over the overhead of certifying every release coming from the Red Hat or the Wind River of the world, right? We want telcos to spend their resources on what is going to generate them revenue. Not the overhead of creating this cloud stack. >> Dave, I remember when we went through this in the enterprise and you had companies like, you know, IBM with the AS400 and the mainframe saying it's easier to manage, which it was, but it's still, you know, it was subsumed by the open systems trend. >> Yeah, yeah. And I think that's an important thing to probe on, is this idea of what is, what exactly does it mean to be cloud at the edge in the telecom space? Because it's a much used term. >> Yeah. >> When we talk about cloud and edge, in sort of generalized IT, but what specifically does it mean? >> Yeah, so when we talk about telco cloud, first of all it's kind of different from what you're thinking about public cloud today. And there's a couple differences. One, if you look at the big hyperscaler public cloud today, they tend to be centralized in huge data centers. Okay, telco cloud, there are big data centers, but then there's also regional data centers. There are edge data centers, which are your typical like access central offices that have turned data centers, and then now even cell sites are becoming mini data centers. So it's distributed. I mean like you could have like, even in a country like say Germany, you'd have 30,000 soul sites, each one of them being a data center. So it's a very different model. Now the other thing I want to go back to the question of monetization, okay? So how do you do monetization? The only way to do that, is to be able to offer new services, like Charles said. How do you offer new services? You have to have an open ecosystem that's going to be very, very flexible. And if we look at where telcos are coming from today, they tend to be very inflexible 'cause they're all kind of single vendor solutions. And even as we've moved to virtualization, you know, if you look at packet core for instance, a lot of them are these vertical stacks of say a Nokia or Ericson or Huawei where you know, you can't really put any other vendors or any other solutions into that. So basically the idea is this kind of horizontal architecture, right? Where now across, not just my central data centers, but across my edge data centers, which would be traditionally my access COs, as well as my cell sites. I have an open environment. And we're kind of starting with, you know, packet core obviously with, and UPFs being distributed, but now open ran or virtual ran, where I can have CUs and DUs and I can split CUs, they could be at the soul site, they could be in edge data centers. But then moving forward, we're going to have like MEG, which are, you know, which are new kinds of services, you know, could be, you know, remote cars it could be gaming, it could be the Metaverse. And these are going to be a multi-vendor environment. So one of the things you need to do is you need to have you know, this cloud layer, and that's what Charles was talking about with the infrastructure blocks is helping the service providers do that, but they still own their infrastructure. >> Yeah, so it's still not clear to me how the service providers win that game but we can maybe come back to that because I want to dig into TCO a little bit. >> Sure. >> Because I have a lot of friends at Dell. I don't have a lot of friends at HPE. I've always been critical when they take an X86 server put a name on it that implies edge and they throw it over the fence to the edge, that's not going to work, okay? We're now seeing, you know we were just at the Dell booth yesterday, you did the booth crawl, which was awesome. Purpose-built servers for this environment. >> Charles: That's right. >> So there's two factors here that I want to explore in TCO. One is, how those next gen servers compare to the previous gen, especially in terms of power consumption but other factors and then how these sort of open ran, open ecosystem stacks compared to proprietary stacks. Peter, can you help us understand those? >> Yeah, sure. And Charles can comment on this as well. But I mean there, there's a couple areas. One is just moving the next generation. So especially on the Intel side, moving from Ice Lake to the Sapphire Rapids is a big deal, especially when it comes to the DU. And you know, with the radios, right? There's the radio unit, the RU, and then there's the DU the distributed unit, and the CU. The DU is really like part of the radio, but it's virtualized. When we moved from Ice lake to Sapphire Rapids, which is third generation intel to fourth generation intel, we're literally almost doubling the performance in the DU. And that's really important 'cause it means like almost half the number of servers and we're talking like 30, 40, 50,000 servers in some cases. So, you know, being able to divide that by two, that's really big, right? In terms of not only the the cost but all the TCO and the OpEx. Now another area that's really important, when I was talking moving from these vertical silos to the horizontal, the issue with the vertical silos is, you can't place any other workloads into those silos. So it's kind of inefficient, right? Whereas when we have the horizontal architecture, now you can place workloads wherever you want, which basically also means less servers but also more flexibility, more service agility. And then, you know, I think Charles can comment more, specifically on the XR8000, some things Dell's doing, 'cause it's really exciting relative to- >> Sure. >> What's happening in there. >> So, you know, when we start looking at putting compute at the edge, right? We recognize the first thing we have to do is understand the environment we are going into. So we spend with a lot of time with telcos going to the south side, going to the edge data center, looking at operation, how do the engineer today deal with maintenance replacement at those locations? Then based on understanding the operation constraints at those sites, we create innovation and take a traditional server, remodel it to make sure that we minimize the disruption to the operations, right? Just because we are helping them going from appliances to open compute, we do not want to disrupt what is have been a very efficient operation on the remote sites. So we created a lot of new ideas and develop them on general compute, where we believe we can save a lot of headache and disruptions and still provide the same level of availability, resiliency, and redundancy on an open compute platform. >> So when we talk about open, we don't mean generic? Fair? See what I mean? >> Open is more from the software workload perspective, right? A Dell server can run any type of workload that customer intend. >> But it's engineered for this? >> Environment. >> Environment. >> That's correct. >> And so what are some of the environmental issues that are dealt with in the telecom space that are different than the average data center? >> The most basic one, is in most of the traditional cell tower, they are deployed within cabinets instead of racks. So they are depth constraints that you just have no access to the rear of the chassis. So that means on a server, is everything you need to access, need to be in the front, nothing should be in the back. Then you need to consider how labor union come into play, right? There's a lot of constraint on who can go to a cell tower and touch power, who can go there and touch compute, right? So we minimize all that disruption through a modular design and make it very efficient. >> So when we took a look at XR8000, literally right here, sitting on the desk. >> Uh-huh. >> Took it apart, don't panic, just pulled out some sleds and things. >> Right, right. >> One of the interesting demonstrations was how it compared to the size of a shoe. Now apparently you hired someone at Dell specifically because they wear a size 14 shoe, (Charles laughs) so it was even more dramatic. >> That's right. >> But when you see it, and I would suggest that viewers go back and take a look at that segment, specifically on the hardware. You can see exactly what you just referenced. This idea that everything is accessible from the front. Yeah. >> So I want to dig in a couple things. So I want to push back a little bit on what you were saying about the horizontal 'cause there's the benefit, if you've got the horizontal infrastructure, you can run a lot more workloads. But I compare it to the enterprise 'cause I, that was the argument, I've made that argument with converged infrastructure versus say an Oracle vertical stack, but it turned out that actually Oracle ran Oracle better, okay? Is there an analog in telco or is this new open architecture going to be able to not only service the wide range of emerging apps but also be as resilient as the proprietary infrastructure? >> Yeah and you know, before I answer that, I also want to say that we've been writing a number of white papers. So we have actually three white papers we've just done with Dell looking at infrastructure blocks and looking at vertical versus horizontal and also looking at moving from the previous generation hardware to the next generation hardware. So all those details, you can find the white papers, and you can find them either in the Dell website or at the ACG research website >> ACGresearch.com? >> ACG research. Yeah, if you just search ACG research, you'll find- >> Yeah. >> Lots of white papers on TCO. So you know, what I want to say, relative to the vertical versus horizontal. Yeah, obviously in the vertical side, some of those things will run well, I mean it won't have issues. However, that being said, as we move to cloud native, you know, it's very high performance, okay? In terms of the stack, whether it be a Red Hat or a VMware or other cloud layers, that's really become much more mature. It now it's all CNF base, which is really containerized, very high performance. And so I don't think really performance is an issue. However, my feeling is that, if you want to offer new services and generate new revenue, you're not going to do it in vertical stacks, period. You're going to be able to do a packet core, you'll be able to do a ran over here. But now what if I want to offer a gaming service? What if I want to do metaverse? What if I want to do, you have to have an environment that's a multi-vendor environment that supports an ecosystem. Even in the RAN, when we look at the RIC, and the xApps and the rApps, these are multi-vendor environments that's going to create a lot of flexibility and you can't do that if you're restricted to, I can only have one vendor running on this hardware. >> Yeah, we're seeing these vendors work together and create RICs. That's obviously a key point, but what I'm hearing is that there may be trade offs, but the incremental value is going to overwhelm that. Second question I have, Peter is, TCO, I've been hearing a lot about 30%, you know, where's that 30% come from? Is it Op, is it from an OpEx standpoint? Is it labor, is it power? Is it, you mentioned, you know, cutting the number of servers in half. If I can unpack the granularity of that TCO, where's the benefit coming from? >> Yeah, the answer is yes. (Peter and Charles laugh) >> Okay, we'll do. >> Yeah, so- >> One side that, in terms of, where is the big bang for the bucks? >> So I mean, so you really need to look at the white paper to see details, but definitely power, definitely labor, definitely reducing the number of servers, you know, reducing the CapEx. The other thing is, is as you move to this really next generation horizontal telco cloud, there's the whole automation and orchestration, that is a key component as well. And it's enabled by what Dell is doing. It's enabled by the, because the thing is you're not going to have end-to-end automation if you have all this legacy stuff there or if you have these vertical stacks where you can't integrate. I mean you can automate that part and then you have separate automation here, you separate. you need to have integrated automation and orchestration across the whole thing. >> One other point I would add also, right, on the hardware perspective, right? With the customized hardware, what we allow operator to do is, take out the existing appliance and push a edge optimized server without reworking the entire infrastructure. There is a significant saving where you don't have to rethink about what is my power infrastructure, right? What is my security infrastructure? The server is designed to leverage the existing, what is already there. >> How should telco, Charles, plan for this transformation? Are there specific best practices that you would recommend in terms of the operational model? >> Great question. I think first thing is do an inventory of what you have. Understand what your constraints are and then come to Dell, we will love to consult with you, based on our experience on the best practices. We know how to minimize additional changes. We know how to help your support engineer, understand how to shift appliance based operation to a cloud-based operation. >> Is that a service you offer? Is that a pre-sales freebie? What is maybe both? >> It's both. >> Yeah. >> It's both. >> Yeah. >> Guys- >> Just really quickly. >> We're going to wrap. >> The, yeah. Dave loves the TCO discussion. I'm always thinking in terms of, well how do you measure TCO when you're comparing something where you can't do something to an environment where you're going to be able to do something new? And I know that that's always the challenge in any kind of emerging market where things are changing, any? >> Well, I mean we also look at, not only TCO, but we look at overall business case. So there's basically service at GLD and revenue and then there's faster time to revenues. Well, and actually ACG, we actually have a platform called the BAE or Business Analytics Engine that's a very sophisticated simulation cloud-based platform, where we can actually look at revenue month by month. And we look at what's the impact of accelerating revenue by three months. By four months. >> So you're looking into- >> By six months- >> So you're forward looking. You're just not consistently- >> So we're not just looking at TCO, we're looking at the overall business case benefit. >> Yeah, exactly right. There's the TCO, which is the hard dollars. >> Right. >> CFO wants to see that, he or she needs to see that. But you got to, you can convince that individual, that there's a business case around it. >> Peter: Yeah. >> And then you're going to sign up for that number. >> Peter: Yeah. >> And they're going to be held to it. That's the story the world wants. >> At the end of the day, telcos have to be offered new services 'cause look at all the money that's been spent. >> Dave: Yeah, that's right. >> On investment on 5G and everything else. >> 0.5 trillion over the next seven years. All right, guys, we got to go. Sorry to cut you off. >> Okay, thank you very much. >> But we're wall to wall here. All right, thanks so much for coming on. >> Dave: Fantastic. >> All right, Dave Vellante, for Dave Nicholson. Lisa Martin's in the house. John Furrier in Palo Alto Studios. Keep it right there. MWC 23 live from the Fira in Barcelona. (light airy music)

(upbeat music) >> Narrator: theCUBE's live coverage is made possible by funding from Dell Technologies: creating technologies that drive human progress. (upbeat music) >> Welcome back to the Fira in Barcelona. You're watching theCUBE's Coverage day two of MWC 23. Check out SiliconANGLE.com for all the news, John Furrier in our Palo Alto studio, breaking that down. But we're here live Dave Vellante, Dave Nicholson and Lisa Martin. We're really excited. We're going to talk qubits. Vanessa Diaz is here. She's CEO of LuxQuanta And Antonio Acin is a professor of ICFO. Folks, welcome to theCUBE. We're going to talk quantum. Really excited about that. >> Vanessa: Thank you guys. >> What does quantum have to do with the network? Tell us. >> Right, so we are actually leaving the second quantum revolution. So the first one actually happened quite a few years ago. It enabled very much the communications that we have today. So in this second quantum revolution, if in the first one we learn about some very basic properties of quantum physics now our scientific community is able to actually work with the systems and ask them to do things. So quantum technologies mean right now, three main pillars, no areas of exploration. The first one is quantum computing. Everybody knows about that. Antonio knows a lot about that too so he can explain further. And it's about computers that now can do wonder. So the ability of of these computers to compute is amazing. So they'll be able to do amazing things. The other pillar is quantum communications but in fact it's slightly older than quantum computer, nobody knows that. And we are the ones that are coming to actually counteract the superpowers of quantum computers. And last but not least quantum sensing, that's the the application of again, quantum physics to measure things that were impossible to measure in with such level of quality, of precision than before. So that's very much where we are right now. >> Okay, so I think I missed the first wave of quantum computing Because, okay, but my, our understanding is ones and zeros, they can be both and the qubits aren't that stable, et cetera. But where are we today, Antonio in terms of actually being able to apply quantum computing? I'm inferring from what Vanessa said that we've actually already applied it but has it been more educational or is there actual work going on with quantum? >> Well, at the moment, I mean, typical question is like whether we have a quantum computer or not. I think we do have some quantum computers, some machines that are able to deal with these quantum bits. But of course, this first generation of quantum computers, they have noise, they're imperfect, they don't have many qubits. So we have to understand what we can do with these quantum computers today. Okay, this is science, but also technology working together to solve relevant problems. So at this moment is not clear what we can do with present quantum computers but we also know what we can do with a perfect quantum computer without noise with many quantum bits, with many qubits. And for instance, then we can solve problems that are out of reach for our classical computers. So the typical example is the problem of factorization that is very connected to what Vanessa does in her company. So we have identified problems that can be solved more efficiently with a quantum computer, with a very good quantum computer. People are working to have this very good quantum computer. At the moment, we have some imperfect quantum computers, we have to understand what we can do with these imperfect machines. >> Okay. So for the first wave was, okay, we have it working for a little while so we see the potential. Okay, and we have enough evidence almost like a little experiment. And now it's apply it to actually do some real work. >> Yeah, so now there is interest by companies so because they see a potential there. So they are investing and they're working together with scientists. We have to identify use cases, problems of relevance for all of us. And then once you identify a problem where a quantum computer can help you, try to solve it with existing machines and see if you can get an advantage. So now the community is really obsessed with getting a quantum advantage. So we really hope that we will get a quantum advantage. This, we know we will get it. We eventually have a very good quantum computer. But we want to have it now. And we're working on that. We have some results, there were I would say a bit academic situation in which a quantum advantage was proven. But to be honest with you on a really practical problem, this has not happened yet. But I believe the day that this happens and I mean it will be really a game changing. >> So you mentioned the word efficiency and you talked about the quantum advantage. Is the quantum advantage a qualitative advantage in that it is fundamentally different? Or is it simply a question of greater efficiency, so therefore a quantitative advantage? The example in the world we're used to, think about a card system where you're writing information on a card and putting it into a filing cabinet and then you want to retrieve it. Well, the information's all there, you can retrieve it. Computer system accelerates that process. It's not doing something that is fundamentally different unless you accept that the speed with which these things can be done gives it a separate quality. So how would you characterize that quantum versus non quantum? Is it just so much horse power changes the game or is it fundamentally different? >> Okay, so from a fundamental perspective, quantum physics is qualitatively different from classical physics. I mean, this year the Nobel Prize was given to three experimentalists who made experiments that proved that quantum physics is qualitatively different from classical physics. This is established, I mean, there have been experiments proving that. Now when we discuss about quantum computation, it's more a quantitative difference. So we have problems that you can solve, in principle you can solve with the classical computers but maybe the amount of time you need to solve them is we are talking about centuries and not with your laptop even with a classic super computer, these machines that are huge, where you have a building full of computers there are some problems for which computers take centuries to solve them. So you can say that it's quantitative, but in practice you may even say that it's impossible in practice and it will remain impossible. And now these problems become feasible with a quantum computer. So it's quantitative but almost qualitative I would say. >> Before we get into the problems, 'cause I want to understand some of those examples, but Vanessa, so your role at LuxQuanta is you're applying quantum in the communication sector for security purposes, correct? >> Vanessa: Correct. >> Because everybody talks about how quantum's going to ruin our lives in terms of taking all our passwords and figuring everything out. But can quantum help us defend against quantum and is that what you do? >> That's what we do. So one of the things that Antonio's explaining so our quantum computer will be able to solve in a reasonable amount of time something that today is impossible to solve unless you leave a laptop or super computer working for years. So one of those things is cryptography. So at the end, when use send a message and you want to preserve its confidentiality what you do is you destroy it but following certain rules which means they're using some kind of key and therefore you can send it through a public network which is the case for every communication that we have, we go through the internet and then the receiver is going to be able to reassemble it because they have that private key and nobody else has. So that private key is actually made of computational problems or mathematical problems that are very, very hard. We're talking about 40 years time for a super computer today to be able to hack it. However, we do not have the guarantee that there is already very smart mind that already have potentially the capacity also of a quantum computer even with enough, no millions, but maybe just a few qubits, it's enough to actually hack this cryptography. And there is also the fear that somebody could actually waiting for quantum computing to finally reach out this amazing capacity we harvesting now which means capturing all this confidential information storage in it. So when we are ready to have the power to unlock it and hack it and see what's behind. So we are talking about information as delicate as governmental, citizens information related to health for example, you name it. So what we do is we build a key to encrypt the information but it's not relying on a mathematical problem it's relying on the laws of quantum physics. So I'm going to have a channel that I'm going to pump photons there, light particles of light. And that quantum channel, because of the laws of physics is going to allow to detect somebody trying to sneak in and seeing the key that I'm establishing. If that happens, I will not create a key if it's clean and nobody was there, I'll give you a super key that nobody today or in the future, regardless of their computational power, will be able to hack. >> So it's like super zero trust. >> Super zero trust. >> Okay so but quantum can solve really challenging mathematical problems. If you had a quantum computer could you be a Bitcoin billionaire? >> Not that I know. I think people are, okay, now you move me a bit of my comfort zone. Because I know people have working on that. I don't think there is a lot of progress at least not that I am aware of. Okay, but I mean, in principle you have to understand that our society is based on information and computation. Computers are a key element in our society. And if you have a machine that computes better but much better than our existing machines, this gives you an advantage for many things. I mean, progress is locked by many computational problems we cannot solve. We can want to have better materials better medicines, better drugs. I mean this, you have to solve hard computational problems. If you have machine that gives you machine learning, big data. I mean, if you have a machine that gives you an advantage there, this may be a really real change. I'm not saying that we know how to do these things with a quantum computer. But if we understand how this machine that has been proven more powerful in some context can be adapted to some other context. I mean having a much better computer machine is an advantage. >> When? When are we going to have, you said we don't really have it today, we want it today. Are we five years away, 10 years away? Who's working on this? >> There are already quantum computers are there. It's just that the capacity that they have of right now is the order of a few hundred qubits. So people are, there are already companies harvesting, they're actually the companies that make these computers they're already putting them. People can access to them through the cloud and they can actually run certain algorithms that have been tailor made or translated to the language of a quantum computer to see how that performs there. So some people are already working with them. There is billions of investment across the world being put on different flavors of technologies that can reach to that quantum supremacy that we are talking about. The question though that you're asking is Q day it sounds like doomsday, you know, Q day. So depending on who you talk to, they will give you a different estimation. So some people say, well, 2030 for example but perhaps we could even think that it could be a more aggressive date, maybe 2027. So it is yet to be the final, let's say not that hard deadline but I think that the risk, that it can actually bring is big enough for us to pay attention to this and start preparing for it. So the end times of cryptography that's what quantum is doing is we have a system here that can actually prevent all your communications from being hacked. So if you think also about Q day and you go all the way back. So whatever tools you need to protect yourself from it, you need to deploy them, you need to see how they fit in your organization, evaluate the benefits, learn about it. So that, how close in time does that bring us? Because I believe that the time to start thinking about this is now. >> And it's likely it'll be some type of hybrid that will get us there, hybrid between existing applications. 'Cause you have to rewrite or write new applications and that's going to take some time. But it sounds like you feel like this decade we will see Q day. What probability would you give that? Is it better than 50/50? By 2030 we'll see Q day. >> But I'm optimistic by nature. So yes, I think it's much higher than 50. >> Like how much higher? >> 80, I would say yes. I'm pretty confident. I mean, but what I want to say also usually when I think there is a message here so you have your laptop, okay, in the past I had a Spectrum This is very small computer, it was more or less the same size but this machine is much more powerful. Why? Because we put information on smaller scales. So we always put information in smaller and smaller scale. This is why here you have for the same size, you have much more information because you put on smaller scales. So if you go small and small and small, you'll find the quantum word. So this is unavoidable. So our information devices are going to meet the quantum world and they're going to exploit it. I'm fully convinced about this, maybe not for the quantum computer we're imagining now but they will find it and they will use quantum effects. And also for cryptography, for me, this is unavoidable. >> And you brought the point there are several companies working on that. I mean, I can get quantum computers on in the cloud and Amazon and other suppliers. IBM of course is. >> The underlying technology, there are competing versions of how you actually create these qubits. pins of electrons and all sorts of different things. Does it need to be super cooled or not? >> Vanessa: There we go. >> At a fundamental stage we'd be getting ground. But what is, what does ChatGPT look like when it can leverage the quantum realm? >> Well, okay. >> I Mean are we all out of jobs at that point? Should we all just be planning for? >> No. >> Not you. >> I think all of us real estate in Portugal, should we all be looking? >> No, actually, I mean in machine learning there are some hopes about quantum competition because usually you have to deal with lots of data. And we know that in quantum physics you have a concept that is called superposition. So we, there are some hopes not in concrete yet but we have some hopes that these superpositions may allow you to explore this big data in a more efficient way. One has to if this can be confirmed. But one of the hopes creating this lots of qubits in this superpositions that you will have better artificial intelligence machines but, okay, this is quite science fiction what I'm saying now. >> At this point and when you say superposition, that's in contrast to the ones and zeros that we're used to. So when someone says it could be a one or zero or a one and a zero, that's referencing the concept of superposition. And so if this is great for encryption, doesn't that necessarily mean that bad actors can leverage it in a way that is now unhackable? >> I mean our technologies, again it's impossible to hack because it is the laws of physics what are allowing me to detect an intruder. So that's the beauty of it. It's not something that you're going to have to replace in the future because there will be a triple quantum computer, it is not going to affect us in any way but definitely the more capacity, computational capacity that we see out there in quantum computers in particular but in any other technologies in general, I mean, when we were coming to talk to you guys, Antonio and I, he was the one saying we do not know whether somebody has reached some relevant computational power already with the technologies that we have. And they've been able to hack already current cryptography and then they're not telling us. So it's a bit of, the message is a little bit like a paranoid message, but if you think about security that the amount of millions that means for a private institution know when there is a data breach, we see it every day. And also the amount of information that is relevant for the wellbeing of a country. Can you really put a reasonable amount of paranoid to that? Because I believe that it's worth exploring whatever tool is going to prevent you from putting any of those piece of information at risk. >> Super interesting topic guys. I know you're got to run. Thanks for stopping by theCUBE, it was great to have you on. >> Thank you guys. >> All right, so this is the SiliconANGLE theCUBE's coverage of Mobile World Congress, MWC now 23. We're live at the Fira Check out silicon SiliconANGLE.com and theCUBE.net for all the videos. Be right back, right after this short break. (relaxing music)