(Gentle music) >> Hello and welcome to the Cube here in Palo Alto, California. I'm John Furrier here, joined by Platform nine, Amelia Bell the Chief Revenue Officer, really digging into the conversation around Kubernetes Cloud native and the journey this next generation cloud. Amelia, thanks for coming in and joining me today. >> Thank you, thank you. Great pleasure to be here. >> So, CRO, chief Revenue Officer. So you're mainly in charge of serving the customers, making sure they're they're happy with the solution you guys have. >> That's right. >> And this market must be pretty exciting. >> Oh, it's very exciting and we are seeing a lot of new use cases coming up all the time. So part of my job is to obtain new customers but then of course, service our existing customers and then there's a constant evolution. Nothing is standing still right now. >> We've had all your co-founders on, on the show here and we've kind of talked about the trends and where you guys have come from, where you guys are going now. And it's interesting, if you look at the cloud native market, the scale is still huge. You seeing now this next wave of AI coming on, which I call that's the real web three in my mind in terms of like the next experiences really still points to data infrastructure scale. These next gen apps are coming. And so that's being built on the previous generation of DevSecOps. >> Right >> And so a lot of enterprises are having to grow up really, really fast >> Right. >> And figure out, okay, I got to have scale I got large scale data, I got horizontal scalability I got to apply machine learning now the new software engineering practice. And then, oh, by the way I got the Kubernetes clusters I got to manage >> Right. >> I got what's containers weather, the security problems. This is a really complicated but important area of build out right now in the marketplace. >> Right. What are you seeing? >> So it's, it's really important that the infrastructure is not the hindrance in these cases. And we, one of our customers is in fact a large AI company and we, I met with them yesterday and asked them, you know, why are you giving that to us? You've got really smart engineers. They can run and create the infrastructure, you know in a custom way that you want it. And they said, we've got to be core to our business. There's plenty of work to do just on delivering the AI capabilities, and there's plenty of work to do. We can't get bogged down in the infrastructure. We don't want to have people running the engine we want them driving the car. We want them creating value on top of that. so they can't have the infrastructure being the bottleneck for them. >> It's interesting, the AI companies, that's their value proposition to their customers is that they don't want the technical talent. >> Right. >> Working on, you know, non-differentiated heavy lifting things. >> Right. >> And automate those and scale it up. Can you talk about the problem that you guys are solving? Because there's a lot going on here. >> Yeah. >> You can look at all aspects of the DevOps scale. There's a lot of little problems, some big problems. What are you guys focusing on? What's the bullseye for Platform known? >> Okay, so the bullseye is that Kubernetes infrastructure is really hard, right? It's really hard to create and run. So we introduce a time to market efficiency, let's get this up and running and let's get you into production and and producing results for your customers fast. But at the same time, let's reduce your cost and complexity and increase reliability. So, >> And what are some of the things that they're having problems with that are breaking? Is it more of updates on code? Is it size of the, I mean clusters they have, what what is it more operational? What are the, what are some of the things that are that kind of get them to call you guys up? What's the main thing? >> It's the operations. It's all operations. So what, what happens is that if you have a look at Kubernetes platform it's made up of many, many components. And that's where it gets complex. It's not just Kubernetes. There's load balances, networking, there's observability. All these things have to operate together. And all the piece parts have to be upgraded and maintained. The integrations need to work, you need to have probes into the system to predict where problems can be coming. So the operational part of it is complex. So you need to be observing not only your clusters in the health of the clusters and the nodes and so on but the health of the platform itself. >> We're going to get Peter Frey in on here after I talk about some of the technical issues on deployments. But what's the, what's the big decision for the customer? Because there's kind of, there's two schools of thought. One is, I'm going to build my own and have my team build it or I'm going to go with a partner >> Right. >> Say platform nine, what's the trade offs there? Because it seems to me that, that there's a there's a certain area of where it's core competency but I can outsource it or partner with it and, and work with platform nine versus trying to take it all on internally >> Right. >> Of which requires more costs. So there's a, there's a line where you kind of like figure out that customers have to figure out that, that piece >> Right >> What do, what's your view on that? Because I'm hearing that more people are saying, hey I want to, I want to focus my people on solutions. The app side, not so much the ops >> Right. >> What's the trade off? How do you talk about? >> It's a really interesting question because most companies think they have two options. It's either a DIY option and they love that engineers love playing with the new and on the latest. And then they think the other option is going to cloud, public cloud and have it semi managed by them. And you get very different out of those. So in the DIY you get flexibility coz you get to choose your infrastructure but then you've got all the complexities of the DIY piece. You've got to not only choose all your components but you've got to keep them working. Now if you go to public cloud option, you lose flexibility because a lot of those choices are made for you but you gain agility because quite frankly it's really easy to spin up clusters. So what we are, is that in the middle we bring the agility and the flexibility because we bring the control plane that allows you to spin up clusters and and lifecycle manage them very quickly. So the agility's there but you can do it on the infrastructure of your choice. And in the DIY culture, one of the hardest things to do actually is to convince them they don't have to do it themselves. They can focus on higher value activities, which are more focused on delivering outcomes to their customers. >> So you provide the solution that allows them to feel like they're billing it themselves. >> Correct. >> And get these scale and speed and the efficiencies of the op side. So it's kind of the best of both worlds. It's not a full outsource. >> Right, right. >> You're bringing them in to make their jobs easier >> Right, That's right. So they get choices. >> Yeah. >> We, we, they get choices on how they build it and then we run and operate it for them. But they, they have all the observability. The benefit is that if we are managing their operations and most of our customers choose the managed operations piece of it, then they don't. If something goes wrong, we fix that and they, they they get told, oh, by the way, you had a problem. We've dealt with it. But in the other model is they've got to create all that observability themselves and they've got to get ahead of the issues themselves, and then they've got to raise tickets to whoever they need to raise tickets to. Whereas we have things like auto ticket generation and so on where, look, just drive the car let us worry about the engine and all of that. Let us deal with that. And you can choose whatever you want about the engine but let us manage it for you. So >> What do you, what do you say to folks out there that are may have a need for platform nine? What's the signals inside their company that they should be calling you guys up and, and leaning in with platform nine? >> Right. >> Is it more sprawl on on clusters? Is it more errors? Is it more tickets? Is it more hassle? What are some of the signs? If someone's watching this say, hey I have, I have an issue with this. >> I would say, if there's operational inefficiencies you can't get things to market fast enough because you are building this and it's just taking too long you're spending way too much time operationally on the infrastructure, then you are, you are not using your resources where they should best be used. And, and that is delivering services to the customer. >> Ed me Hora on for International Women's Day. And she was talking about how they love to solve complex problems on the engineering team at Platform nine. It's going to get pretty complex with the edge emerging >> Indeed >> and cloud native on-premises distributed computing. >> Indeed. >> essentially is what it is. That's kind of the core DNA of the team. >> Yeah. >> What, how does that translate to the customers? Because IT seems to be, okay, I have virtual machines were great, now I got to scale up and and convert over a transform to containers, Kubernetes >> Right. >> And then large scale app, app applications. >> Right, so when it comes to Edge it gets complex pretty fast because it's highly distributed. So how do you have standardization and governance across all the different edge locations? So what we bring into play is an ability to, um, at each edge, location eh, provision from bare metal up all the way up to the application. So let's say you have thousands of stores and you want to modernize those stores, you know rather than having a server being sent somewhere to have an image loaded up and then sent that and then you've got to send a technical guide to the store and you've got to implement it all there. Forget all that. That's just, that's just a ridiculous waste of time. So what we've done is we've created the ability where the server can just be sent to the store. You can get your barista or your chef just to plug it in, right? You don't need to send any technical person over there. As long as we have access to it, we get access to it and we provision the whole thing from bare metal up and then we can maintain it according to the standards that are needed and upgrade accordingly. And that gives standardization across all your stores or edge locations or 5G towers or whatever it is, distribution centers. And we can create nice governance and good standardization which allows them to innovate fast as well. >> So this is a real opportunity for you guys. >> Yeah. >> This is an advantage from your expertise. >> Yes. >> The edge piece, dropping in a box, self-provisioning. >> That's right. So yeah. >> Can people do that? What's the, >> No, actually it, it's, it's very difficult to do. I I, from my understanding, we're the only people that can provision it from bare metal up, right? So if anyone has a different story, I'd love to hear about that. But that's my understanding today. >> That's a good value purpose. So talk about the value of the customer. What kind of scope do you got? Can you scope some of the customer environments you have from >> Sure. >> From, you know, small to the large, how give us an idea of the order of magnitude of the >> Yeah, so, so small customers may have 20 clusters or something like that. 20 nodes, I beg your pardon. Our large customers, like we're we are scaling one particular distributed environment from 2200 nodes to 10,000 nodes by the end of this year and 26,000 nodes next year. We have another customer that's scaling up to 10,000 nodes this year as well. So we have some very large scale, but some smaller ones too. And we're, we're happy to work with either end. >> Okay, so pretend I'm a customer. I'm really, I got pain and Kubernetes like I want to, I can't hire enough people. I want to have my all focus. What's the pitch? >> Okay. So skill shortage is something that that everyone is facing right now. And if, if you've got skill shortage it's going to be really hard to hire if you are competing against really, you know, high salary you know, offering companies that are out there. So the pitch is, let us do it for you. We have, we have a team of excellent probably the best Kubernetes engineers on the planet. We will create your environment for you. We will get it up and running. We will allow you to, you know, run your applica, just consume the platform, we'll run it for you. We'll have SLAs and up times guaranteed and you can just focus on delivering the software and the value needed to your customers. >> What are some of the testimonials that you get from people? Just anecdotally, what do they say? Oh my god, you guys save. >> Yeah. >> Our butts. >> Yeah. >> This is amazing. We just shipped our code out much faster. >> Yeah. >> What are some of the things that you hear? >> So, so the number one thing I hear is it just works right? It's, we don't have to worry about it, it just works. So that, that's a really great feedback that we get. The other thing I hear is if we do have issues that your team are amazing, they they fix things, they're proactive, you know, they're we really enjoy working with you. So from, from that perspective, that's great. But the other side of it is we hear things like if we were to do that ourselves we would've taken six to 12 months to build that. And you guys have just saved us six to 12 months. The other thing that we hear is with the same two engineers we started on, you know, a hundred nodes we're now running thousands of nodes. We have not had to increase the size of the team and expand and scale exponentially. >> Awesome. What's next for you guys? What's on your, your plate? >> Yeah. >> With CRO, what's some of the goals you have? >> Yeah, so growth of course as a CRO, you don't get away from that. We've got some very exciting, actually, initiatives coming up. One of the things that we are seeing a lot of demand for and is, is in the area of virtualization bringing virtual machine, virtual virtual containers, sorry I'm saying that all wrong. Bringing virtual machine, the virtual machines onto the cloud native infrastructure using Kubernetes technology. So that provides a, an excellent stepping stone for those guys who are in the virtualization world. And they can't move to containers, they can't refactor their applications and workloads fast enough. So just bring your virtual machine and put it onto the container infrastructure. So we're seeing a lot of demand for that, because it provides an excellent stepping stone. Why not use Kubernetes to orchestrate virtual the virtual world? And then we've got some really interesting cost optimization. >> So a lot of migration kind of thinking around VMs and >> Oh, tremendous. The, the VM world is just massively bigger than the container world right now. So you can't ignore that. So we are providing basically the evolution, the the journey for the customers to utilize the greatest of technologies without having to do that in a, in a in a way that just breaks the bank and they can't get there fast enough. So we provide those stepping stones for them. Yeah. >> Amelia thank you for coming on. Sharing. >> Thank you. >> The update on platform nine. Congratulations on your big accounts you have and >> thank you. >> And the world could get more complex, which Means >> indeed >> have more customers. >> Thank you, thank you John. Appreciate that. Thank you. >> I'm John Furry. You're watching Platform nine and the Cube Conversations here. Thanks for watching. (gentle music)

(gentle upbeat music) >> Hello, everyone, welcome to theCUBE's coverage of International Women's Day. I'm John Furrier host here in Palo Alto, California. theCUBE's second year covering International Women's Day. It's been a great celebration of all the smart leaders in the world who are making a difference from all kinds of backgrounds, from technology to business and everything in between. Today we've got a great guest, Dominique Bastos, who's the senior Vice President of Cloud at Persistent Systems, formerly with AWS. That's where we first met at re:Invent. Dominique, great to have you on the program here for International Women's Day. Thanks for coming on. >> Thank you John, for having me back on theCUBE. This is an honor, especially given the theme. >> Well, I'm excited to have you on, I consider you one of those typecast personas where you've kind of done a lot of things. You're powerful, you've got great business acumen you're technical, and we're in a world where, you know the world's coming completely digital and 50% of the world is women, 51%, some say. So you got mostly male dominated industry and you have a dual engineering background and that's super impressive as well. Again, technical world, male dominated you're in there in the mix. What inspires you to get these engineering degrees? >> I think even it was more so shifted towards males. When I had the inspiration to go to engineering school I was accused as a young girl of being a tomboy and fiddling around with all my brother's toys versus focusing on my dolls and other kind of stereotypical toys that you would give a girl. I really had a curiosity for building, a curiosity for just breaking things apart and putting them back together. I was very lucky in that my I guess you call it primary school, maybe middle school, had a program for, it was like electronics, that was the class electronics. So building circuit boards and things like that. And I really enjoyed that aspect of building. I think it was more actually going into engineering school. Picking that as a discipline was a little bit, my mom's reaction to when I announced that I wanted to do engineering which was, "No, that's for boys." >> Really. >> And that really, you know, I think she, it came from a good place in trying to protect me from what she has experienced herself in terms of how women are received in those spaces. So I kind of shrugged it off and thought "Okay, well I'm definitely now going to do this." >> (laughs) If I was told not to, you're going to do it. >> I was told not to, that's all I needed to hear. And also, I think my passion was to design cars and I figured if I enroll in an industrial engineering program I could focus on ergonomic design and ultimately, you know have a career doing something that I'm passionate about. So yeah, so my inspiration was kind of a little bit of don't do this, a lot of curiosity. I'm also a very analytical person. I've been, and I don't know what the science is around left right brain to be honest, but been told that I'm a very much a logical person versus a feeler. So I don't know if that's good or bad. >> Straight shooter. What were your engineering degrees if you don't mind sharing? >> So I did industrial engineering and so I did a dual degree, industrial engineering and robotics. At the time it was like a manufacturing robotics program. It was very, very cool because we got to, I mean now looking back, the evolution of robotics is just insane. But you, you know, programmed a robotic arm to pick things up. I actually crashed the Civil Engineering School's Concrete Canoe Building Competition where you literally have to design a concrete canoe and do all the load testing and the strength testing of the materials and basically then, you know you go against other universities to race the canoe in a body of water. We did that at, in Alabama and in Georgia. So I was lucky to experience that two times. It was a lot of fun. >> But you knew, so you knew, deep down, you were technical you had a nerd vibe you were geeking out on math, tech, robotics. What happened next? I mean, what were some of the challenges you faced? How did you progress forward? Did you have any blockers and roadblocks in front of you and how did you handle those? >> Yeah, I mean I had, I had a very eye-opening experience with, in my freshman year of engineering school. I kind of went in gung-ho with zero hesitation, all the confidence in the world, 'cause I was always a very big nerd academically, I hate admitting this but myself and somebody else got most intellectual, voted by the students in high school. It's like, you don't want to be voted most intellectual when you're in high school. >> Now it's a big deal. (laughs) >> Yeah, you want to be voted like popular or anything like that? No, I was a nerd, but in engineering school, it's a, it was very humbling. That whole confidence that I had. I experienced prof, ooh, I don't want to name the school. Everybody can google it though, but, so anyway so I had experience with some professors that actually looked at me and said, "You're in the wrong program. This is difficult." I, and I think I've shared this before in other forums where, you know, my thermodynamic teacher basically told me "Cheerleading's down the hall," and it it was a very shocking thing to hear because it really made me wonder like, what am I up against here? Is this what it's going to be like going forward? And I decided not to pay attention to that. I think at the moment when you hear something like that you just, you absorb it and you also don't know how to react. And I decided immediately to just walk right past him and sit down front center in the class. In my head I was cursing him, of course, 'cause I mean, let's be real. And I was like, I'm going to show this bleep bleep. And proceeded to basically set the curve class crushed it and was back to be the teacher's assistant. So I think that was one. >> But you became his teacher assistant after, or another one? >> Yeah, I gave him a mini speech. I said, do not do this. You, you could, you could have broken me and if you would've done this to somebody who wasn't as steadfast in her goals or whatever, I was really focused like I'm doing this, I would've backed out potentially and said, you know this isn't something I want to experience on the daily. So I think that was actually a good experience because it gave me an opportunity to understand what I was up against but also double down in how I was going to deal with it. >> Nice to slay the misogynistic teachers who typecast people. Now you had a very technical career but also you had a great career at AWS on the business side you've handled 'em all of the big accounts, I won't say the names, but like we're talking about monster accounts, sales and now basically it's not really selling, you're managing a big account, it's like a big business. It's a business development thing. Technical to business transition, how do you handle that? Was that something you were natural for? Obviously you, you stared down the naysayers out of the gate in college and then in business, did that continue and how did you drive through that? >> So I think even when I was coming out of university I knew that I wanted to have a balance between the engineering program and business. A lot of my colleagues went on to do their PEs so continue to get their masters basically in engineering or their PhDs in engineering. I didn't really have an interest for that. I did international business and finance as my MBA because I wanted to explore the ability of taking what I had learned in engineering school and applying it to building businesses. I mean, at the time I didn't have it in my head that I would want to do startups but I definitely knew that I wanted to get a feel for what are they learning in business school that I missed out in engineering school. So I think that helped me when I transitioned, well when I applied, I was asked to come apply at AWS and I kind of went, no I'm going to, the DNA is going to be rejected. >> You thought, you thought you'd be rejected from AWS. >> I thought I'd be, yeah, because I have very much a startup founder kind of disruptive personality. And to me, when I first saw AWS at the stage early 2016 I saw it as a corporation. Even though from a techie standpoint, I was like, these people are insane. This is amazing what they're building. But I didn't know what the cultural vibe would feel like. I had been with GE at the beginning of my career for almost three years. So I kind of equated AWS Amazon to GE given the size because in between, I had done startups. So when I went to AWS I think initially, and I do have to kind of shout out, you know Todd Weatherby basically was the worldwide leader for ProServe and it was being built, he built it and I went into ProServe to help from that standpoint. >> John: ProServe, Professional services >> Professional services, right. To help these big enterprise customers. And specifically my first customer was an amazing experience in taking, basically the company revolves around strategic selling, right? It's not like you take a salesperson with a conventional schooling that salespeople would have and plug them into AWS in 2016. It was very much a consultative strategic approach. And for me, having a technical background and loving to solve problems for customers, working with the team, I would say, it was a dream team that I joined. And also the ability to come to the table with a technical background, knowing how to interact with senior executives to help them envision where they want to go, and then to bring a team along with you to make that happen. I mean, that was like magical for me. I loved that experience. >> So you like the culture, I mean, Andy Jassy, I've interviewed many times, always talked about builders and been a builder mentality. You mentioned that earlier at the top of this interview you've always building things, curious and you mentioned potentially your confidence might have been shaken. So you, you had the confidence. So being a builder, you know, being curious and having confidence seems to be what your superpower is. A lot of people talk about the confidence angle. How important is that and how important is that for encouraging more women to get into tech? Because I still hear that all the time. Not that they don't have confidence, but there's so many signals that potentially could shake confidence in industry >> Yeah, that's actually a really good point that you're making. A lot of signals that women get could shake their confidence and that needs to be, I mean, it's easy to say that it should be innate. I mean that's kind of like textbook, "Oh it has to come from within." Of course it does. But also, you know, we need to understand that in a population where 50% of the population is women but only 7% of the positions in tech, and I don't know the most current number in tech leadership, is women, and probably a smaller percentage in the C-suite. When you're looking at a woman who's wanting to go up the trajectory in a tech company and then there's a subconscious understanding that there's a limit to how far you'll go, your confidence, you know, in even subconsciously gets shaken a little bit because despite your best efforts, you're already seeing the cap. I would say that we need to coach girls to speak confidently to navigate conflict versus running away from it, to own your own success and be secure in what you bring to the table. And then I think a very important thing is to celebrate each other and the wins that we see for women in tech, in the industry. >> That's awesome. What's, the, in your opinion, the, you look at that, the challenges for this next generation women, and women in general, what are some of the challenges for them and that they need to overcome today? I mean, obviously the world's changed for the better. Still not there. I mean the numbers one in four women, Rachel Thornton came on, former CMO of AWS, she's at MessageBird now. They had a study where only one in four women go to the executive board level. And so there's still, still numbers are bad and then the numbers still got to get up, up big time. That's, and the industry's working on that, but it's changed. But today, what are some of the challenges for this current generation and the next generation of women and how can we and the industry meet, we being us, women in the industry, be strong role models for them? >> Well, I think the challenge is one of how many women are there in the pipeline and what are we doing to retain them and how are we offering up the opportunities to fill. As you know, as Rachel said and I haven't had an opportunity to see her, in how are we giving them this opportunity to take up those seats in the C-suite right, in these leadership roles. And I think this is a little bit exacerbated with the pandemic in that, you know when everything shut down when people were going back to deal with family and work at the same time, for better or for worse the brunt of it fell on probably, you know the maternal type caregiver within the family unit. You know, I've been, I raised my daughter alone and for me, even without the pandemic it was a struggle constantly to balance the risk that I was willing to take to show up for those positions versus investing even more of that time raising a child, right? Nevermind the unconscious bias or cultural kind of expectations that you get from the male counterparts where there's zero understanding of what a mom might go through at home to then show up to a meeting, you know fully fresh and ready to kind of spit out some wisdom. It's like, you know, your kid just freaking lost their whatever and you know, they, so you have to sort a bunch of things out. I think the challenge that women are still facing and will we have to keep working at it is making sure that there's a good pipeline. A good amount of young ladies of people taking interest in tech. And then as they're, you know, going through the funnel at stages in their career, we're providing the mentoring we're, there's representation, right? To what they're aspiring to. We're celebrating their interest in the field, right? And, and I think also we're doing things to retain them, because again, the pandemic affected everybody. I think women specifically and I don't know the statistics but I was reading something about this were the ones to tend to kind of pull it back and say well now I need to be home with, you know you name how many kids and pets and the aging parents, people that got sick to take on that position. In addition to the career aspirations that they might have. We need to make it easier basically. >> I think that's a great call out and I appreciate you bringing that up about family and being a single mom. And by the way, you're savage warrior to doing that. It's amazing. You got to, I know you have a daughter in computer science at Stanford, I want to get to that in a second. But that empathy and I mentioned Rachel Thornton, who's the CMO MessageBird and former CMO of AWS. Her thing right now to your point is mentoring and sponsorship is very key. And her company and the video that's on the site here people should look at that and reference that. They talk a lot about that empathy of people's situation whether it's a single mom, family life, men and women but mainly women because they're the ones who people aren't having a lot of empathy for in that situation, as you called it out. This is huge. And I think remote work has opened up this whole aperture of everyone has to have a view into how people are coming to the table at work. So, you know, props are bringing that up, and I recommend everyone look at check out Rachel Thornton. So how do you balance that, that home life and talk about your daughter's journey because sounds like she's nerding out at Stanford 'cause you know Stanford's called Nerd Nation, that's their motto, so you must be proud. >> I am so proud, I'm so proud. And I will say, I have to admit, because I did encounter so many obstacles and so many hurdles in my journey, it's almost like I forgot that I should set that aside and not worry about my daughter. My hope for her was for her to kind of be artistic and a painter or go into something more lighthearted and fun because I just wanted to think, I guess my mom had the same idea, right? She, always been very driven. She, I want to say that I got very lucky that she picked me to be her mom. Biologically I'm her mom, but I told her she was like a little star that fell from the sky and I, and ended up with me. I think for me, balancing being a single mom and a career where I'm leading and mentoring and making big decisions that affect people's lives as well. You have to take the best of everything you get from each of those roles. And I think that the best way is play to your strengths, right? So having been kind of a nerd and very organized person and all about, you know, systems for effectiveness, I mean, industrial engineering, parenting for me was, I'm going to make it sound super annoying and horrible, but (laughs) >> It's funny, you know, Dave Vellante and I when we started SiliconANGLE and theCUBE years ago, one of the things we were all like sports lovers. So we liked sports and we are like we looked at the people in tech as tech athletes and except there's no men and women teams, it's one team. It's all one thing. So, you know, I consider you a tech athlete you're hard charging strong and professional and smart and beautiful and brilliant, all those good things. >> Thank you. >> Now this game is changing and okay, and you've done startups, and you've done corporate jobs, now you're in a new role. What's the current tech landscape from a, you know I won't say athletic per standpoint but as people who are smart. You have all kinds of different skill sets. You have the startup warriors, you have the folks who like to be in the middle of the corporate world grow up through corporate, climb the corporate ladder. You have investors, you have, you know, creatives. What have you enjoyed most and where do you see all the action? >> I mean, I think what I've enjoyed the most has been being able to bring all of the things that I feel I'm strong at and bring it together to apply that to whatever the problem is at hand, right? So kind of like, you know if you look at a renaissance man who can kind of pop in anywhere and, oh, he's good at, you know sports and he's good at reading and, or she's good at this or, take all of those strengths and somehow bring them together to deal with the issue at hand, versus breaking up your mindset into this is textbook what I learned and this is how business should be done and I'm going to draw these hard lines between personal life and work life, or between how you do selling and how you do engineering. So I think my, the thing that I loved, really loved about AWS was a lot of leaders saw something in me that I potentially didn't see, which was, yeah you might be great at running that big account but we need help over here doing go to market for a new product launch and boom, there you go. Now I'm in a different org helping solve that problem and getting something launched. And I think if you don't box yourself in to I'm only good at this, or, you know put a label on yourself as being the rockstar in that. It leaves room for opportunities to present themselves but also it leaves room within your own mind to see yourself as somebody capable of doing anything. Right, I don't know if I answered the question accurately. >> No, that's good, no, that's awesome. I love the sharing, Yeah, great, great share there. Question is, what do you see, what do you currently during now you're building a business of Persistent for the cloud, obviously AWS and Persistent's a leader global system integrator around the world, thousands and thousands of customers from what we know and been reporting on theCUBE, what's next for you? Where do you see yourself going? Obviously you're going to knock this out of the park. Where do you see yourself as you kind of look at the continuing journey of your mission, personal, professional what's on your mind? Where do you see yourself going next? >> Well, I think, you know, again, going back to not boxing yourself in. This role is an amazing one where I have an opportunity to take all the pieces of my career in tech and apply them to building a business within a business. And that involves all the goodness of coaching and mentoring and strategizing. And I'm loving it. I'm loving the opportunity to work with such great leaders. Persistent itself is very, very good at providing opportunities, very diverse opportunities. We just had a huge Semicolon; Hackathon. Some of the winners were females. The turnout was amazing in the CTO's office. We have very strong women leading the charge for innovation. I think to answer your question about the future and where I may see myself going next, I think now that my job, well they say the job is never done. But now that Chloe's kind of settled into Stanford and kind of doing her own thing, I have always had a passion to continue leading in a way that brings me to, into the fold a lot more. So maybe, you know, maybe in a VC firm partner mode or another, you know CEO role in a startup, or my own startup. I mean, I never, I don't know right now I'm super happy but you never know, you know where your drive might go. And I also want to be able to very deliberately be in a role where I can continue to mentor and support up and coming women in tech. >> Well, you got the smarts but you got really the building mentality, the curiosity and the confidence really sets you up nicely. Dominique great story, great inspiration. You're a role model for many women, young girls out there and women in tech and in celebration. It's a great day and thank you for sharing that story and all the good nuggets there. Appreciate you coming on theCUBE, and it's been my pleasure. Thanks for coming on. >> Thank you, John. Thank you so much for having me. >> Okay, theCUBE's coverage of International Women's Day. I'm John Furrier, host of theCUBE here in Palo Alto getting all the content, check out the other interviews some amazing stories, lessons learned, and some, you know some funny stories and some serious stories. So have some fun and enjoy the rest of the videos here for International Women's Days, thanks for watching. (gentle inspirational music)

(mellow music) >> Hello everyone, welcome to theCUBE's coverage of AWS Startup Showcase. This is the closing panel session on AI machine learning, the top startups generating generative AI on AWS. It's a great panel. This is going to be the experts talking about riding the wave in generative AI. We got Ankur Mehrotra, who's the director and general manager of AI and machine learning at AWS, and Clem Delangue, co-founder and CEO of Hugging Face, and Ori Goshen, who's the co-founder and CEO of AI21 Labs. Ori from Tel Aviv dialing in, and rest coming in here on theCUBE. Appreciate you coming on for this closing session for the Startup Showcase. >> Thanks for having us. >> Thank you for having us. >> Thank you. >> I'm super excited to have you all on. Hugging Face was recently in the news with the AWS relationship, so congratulations. Open source, open science, really driving the machine learning. And we got the AI21 Labs access to the LLMs, generating huge scale live applications, commercial applications, coming to the market, all powered by AWS. So everyone, congratulations on all your success, and thank you for headlining this panel. Let's get right into it. AWS is powering this wave here. We're seeing a lot of push here from applications. Ankur, set the table for us on the AI machine learning. It's not new, it's been goin' on for a while. Past three years have been significant advancements, but there's been a lot of work done in AI machine learning. Now it's released to the public. Everybody's super excited and now says, "Oh, the future's here!" It's kind of been going on for a while and baking. Now it's kind of coming out. What's your view here? Let's get it started. >> Yes, thank you. So, yeah, as you may be aware, Amazon has been in investing in machine learning research and development since quite some time now. And we've used machine learning to innovate and improve user experiences across different Amazon products, whether it's Alexa or Amazon.com. But we've also brought in our expertise to extend what we are doing in the space and add more generative AI technology to our AWS products and services, starting with CodeWhisperer, which is an AWS service that we announced a few months ago, which is, you can think of it as a coding companion as a service, which uses generative AI models underneath. And so this is a service that customers who have no machine learning expertise can just use. And we also are talking to customers, and we see a lot of excitement about generative AI, and customers who want to build these models themselves, who have the talent and the expertise and resources. For them, AWS has a number of different options and capabilities they can leverage, such as our custom silicon, such as Trainium and Inferentia, as well as distributed machine learning capabilities that we offer as part of SageMaker, which is an end-to-end machine learning development service. At the same time, many of our customers tell us that they're interested in not training and building these generative AI models from scratch, given they can be expensive and can require specialized talent and skills to build. And so for those customers, we are also making it super easy to bring in existing generative AI models into their machine learning development environment within SageMaker for them to use. So we recently announced our partnership with Hugging Face, where we are making it super easy for customers to bring in those models into their SageMaker development environment for fine tuning and deployment. And then we are also partnering with other proprietary model providers such as AI21 and others, where we making these generative AI models available within SageMaker for our customers to use. So our approach here is to really provide customers options and choices and help them accelerate their generative AI journey. >> Ankur, thank you for setting the table there. Clem and Ori, I want to get your take, because the riding the waves, the theme of this session, and to me being in California, I imagine the big surf, the big waves, the big talent out there. This is like alpha geeks, alpha coders, developers are really leaning into this. You're seeing massive uptake from the smartest people. Whether they're young or around, they're coming in with their kind of surfboards, (chuckles) if you will. These early adopters, they've been on this for a while; Now the waves are hitting. This is a big wave, everyone sees it. What are some of those early adopter devs doing? What are some of the use cases you're seeing right out of the gate? And what does this mean for the folks that are going to come in and get on this wave? Can you guys share your perspective on this? Because you're seeing the best talent now leaning into this. >> Yeah, absolutely. I mean, from Hugging Face vantage points, it's not even a a wave, it's a tidal wave, or maybe even the tide itself. Because actually what we are seeing is that AI and machine learning is not something that you add to your products. It's very much a new paradigm to do all technology. It's this idea that we had in the past 15, 20 years, one way to build software and to build technology, which was writing a million lines of code, very rule-based, and then you get your product. Now what we are seeing is that every single product, every single feature, every single company is starting to adopt AI to build the next generation of technology. And that works both to make the existing use cases better, if you think of search, if you think of social network, if you think of SaaS, but also it's creating completely new capabilities that weren't possible with the previous paradigm. Now AI can generate text, it can generate image, it can describe your image, it can do so many new things that weren't possible before. >> It's going to really make the developers really productive, right? I mean, you're seeing the developer uptake strong, right? >> Yes, we have over 15,000 companies using Hugging Face now, and it keeps accelerating. I really think that maybe in like three, five years, there's not going to be any company not using AI. It's going to be really kind of the default to build all technology. >> Ori, weigh in on this. APIs, the cloud. Now I'm a developer, I want to have live applications, I want the commercial applications on this. What's your take? Weigh in here. >> Yeah, first, I absolutely agree. I mean, we're in the midst of a technology shift here. I think not a lot of people realize how big this is going to be. Just the number of possibilities is endless, and I think hard to imagine. And I don't think it's just the use cases. I think we can think of it as two separate categories. We'll see companies and products enhancing their offerings with these new AI capabilities, but we'll also see new companies that are AI first, that kind of reimagine certain experiences. They build something that wasn't possible before. And that's why I think it's actually extremely exciting times. And maybe more philosophically, I think now these large language models and large transformer based models are helping us people to express our thoughts and kind of making the bridge from our thinking to a creative digital asset in a speed we've never imagined before. I can write something down and get a piece of text, or an image, or a code. So I'll start by saying it's hard to imagine all the possibilities right now, but it's certainly big. And if I had to bet, I would say it's probably at least as big as the mobile revolution we've seen in the last 20 years. >> Yeah, this is the biggest. I mean, it's been compared to the Enlightenment Age. I saw the Wall Street Journal had a recent story on this. We've been saying that this is probably going to be bigger than all inflection points combined in the tech industry, given what transformation is coming. I guess I want to ask you guys, on the early adopters, we've been hearing on these interviews and throughout the industry that there's already a set of big companies, a set of companies out there that have a lot of data and they're already there, they're kind of tinkering. Kind of reminds me of the old hyper scaler days where they were building their own scale, and they're eatin' glass, spittin' nails out, you know, they're hardcore. Then you got everybody else kind of saying board level, "Hey team, how do I leverage this?" How do you see those two things coming together? You got the fast followers coming in behind the early adopters. What's it like for the second wave coming in? What are those conversations for those developers like? >> I mean, I think for me, the important switch for companies is to change their mindset from being kind of like a traditional software company to being an AI or machine learning company. And that means investing, hiring machine learning engineers, machine learning scientists, infrastructure in members who are working on how to put these models in production, team members who are able to optimize models, specialized models, customized models for the company's specific use cases. So it's really changing this mindset of how you build technology and optimize your company building around that. Things are moving so fast that I think now it's kind of like too late for low hanging fruits or small, small adjustments. I think it's important to realize that if you want to be good at that, and if you really want to surf this wave, you need massive investments. If there are like some surfers listening with this analogy of the wave, right, when there are waves, it's not enough just to stand and make a little bit of adjustments. You need to position yourself aggressively, paddle like crazy, and that's how you get into the waves. So that's what companies, in my opinion, need to do right now. >> Ori, what's your take on the generative models out there? We hear a lot about foundation models. What's your experience running end-to-end applications for large foundation models? Any insights you can share with the app developers out there who are looking to get in? >> Yeah, I think first of all, it's start create an economy, where it probably doesn't make sense for every company to create their own foundation models. You can basically start by using an existing foundation model, either open source or a proprietary one, and start deploying it for your needs. And then comes the second round when you are starting the optimization process. You bootstrap, whether it's a demo, or a small feature, or introducing new capability within your product, and then start collecting data. That data, and particularly the human feedback data, helps you to constantly improve the model, so you create this data flywheel. And I think we're now entering an era where customers have a lot of different choice of how they want to start their generative AI endeavor. And it's a good thing that there's a variety of choices. And the really amazing thing here is that every industry, any company you speak with, it could be something very traditional like industrial or financial, medical, really any company. I think peoples now start to imagine what are the possibilities, and seriously think what's their strategy for adopting this generative AI technology. And I think in that sense, the foundation model actually enabled this to become scalable. So the barrier to entry became lower; Now the adoption could actually accelerate. >> There's a lot of integration aspects here in this new wave that's a little bit different. Before it was like very monolithic, hardcore, very brittle. A lot more integration, you see a lot more data coming together. I have to ask you guys, as developers come in and grow, I mean, when I went to college and you were a software engineer, I mean, I got a degree in computer science, and software engineering, that's all you did was code, (chuckles) you coded. Now, isn't it like everyone's a machine learning engineer at this point? Because that will be ultimately the science. So, (chuckles) you got open source, you got open software, you got the communities. Swami called you guys the GitHub of machine learning, Hugging Face is the GitHub of machine learning, mainly because that's where people are going to code. So this is essentially, machine learning is computer science. What's your reaction to that? >> Yes, my co-founder Julien at Hugging Face have been having this thing for quite a while now, for over three years, which was saying that actually software engineering as we know it today is a subset of machine learning, instead of the other way around. People would call us crazy a few years ago when we're seeing that. But now we are realizing that you can actually code with machine learning. So machine learning is generating code. And we are starting to see that every software engineer can leverage machine learning through open models, through APIs, through different technology stack. So yeah, it's not crazy anymore to think that maybe in a few years, there's going to be more people doing AI and machine learning. However you call it, right? Maybe you'll still call them software engineers, maybe you'll call them machine learning engineers. But there might be more of these people in a couple of years than there is software engineers today. >> I bring this up as more tongue in cheek as well, because Ankur, infrastructure's co is what made Cloud great, right? That's kind of the DevOps movement. But here the shift is so massive, there will be a game-changing philosophy around coding. Machine learning as code, you're starting to see CodeWhisperer, you guys have had coding companions for a while on AWS. So this is a paradigm shift. How is the cloud playing into this for you guys? Because to me, I've been riffing on some interviews where it's like, okay, you got the cloud going next level. This is an example of that, where there is a DevOps-like moment happening with machine learning, whether you call it coding or whatever. It's writing code on its own. Can you guys comment on what this means on top of the cloud? What comes out of the scale? What comes out of the benefit here? >> Absolutely, so- >> Well first- >> Oh, go ahead. >> Yeah, so I think as far as scale is concerned, I think customers are really relying on cloud to make sure that the applications that they build can scale along with the needs of their business. But there's another aspect to it, which is that until a few years ago, John, what we saw was that machine learning was a data scientist heavy activity. They were data scientists who were taking the data and training models. And then as machine learning found its way more and more into production and actual usage, we saw the MLOps become a thing, and MLOps engineers become more involved into the process. And then we now are seeing, as machine learning is being used to solve more business critical problems, we're seeing even legal and compliance teams get involved. We are seeing business stakeholders more engaged. So, more and more machine learning is becoming an activity that's not just performed by data scientists, but is performed by a team and a group of people with different skills. And for them, we as AWS are focused on providing the best tools and services for these different personas to be able to do their job and really complete that end-to-end machine learning story. So that's where, whether it's tools related to MLOps or even for folks who cannot code or don't know any machine learning. For example, we launched SageMaker Canvas as a tool last year, which is a UI-based tool which data analysts and business analysts can use to build machine learning models. So overall, the spectrum in terms of persona and who can get involved in the machine learning process is expanding, and the cloud is playing a big role in that process. >> Ori, Clem, can you guys weigh in too? 'Cause this is just another abstraction layer of scale. What's it mean for you guys as you look forward to your customers and the use cases that you're enabling? >> Yes, I think what's important is that the AI companies and providers and the cloud kind of work together. That's how you make a seamless experience and you actually reduce the barrier to entry for this technology. So that's what we've been super happy to do with AWS for the past few years. We actually announced not too long ago that we are doubling down on our partnership with AWS. We're excited to have many, many customers on our shared product, the Hugging Face deep learning container on SageMaker. And we are working really closely with the Inferentia team and the Trainium team to release some more exciting stuff in the coming weeks and coming months. So I think when you have an ecosystem and a system where the AWS and the AI providers, AI startups can work hand in hand, it's to the benefit of the customers and the companies, because it makes it orders of magnitude easier for them to adopt this new paradigm to build technology AI. >> Ori, this is a scale on reasoning too. The data's out there and making sense out of it, making it reason, getting comprehension, having it make decisions is next, isn't it? And you need scale for that. >> Yes. Just a comment about the infrastructure side. So I think really the purpose is to streamline and make these technologies much more accessible. And I think we'll see, I predict that we'll see in the next few years more and more tooling that make this technology much more simple to consume. And I think it plays a very important role. There's so many aspects, like the monitoring the models and their kind of outputs they produce, and kind of containing and running them in a production environment. There's so much there to build on, the infrastructure side will play a very significant role. >> All right, that's awesome stuff. I'd love to change gears a little bit and get a little philosophy here around AI and how it's going to transform, if you guys don't mind. There's been a lot of conversations around, on theCUBE here as well as in some industry areas, where it's like, okay, all the heavy lifting is automated away with machine learning and AI, the complexity, there's some efficiencies, it's horizontal and scalable across all industries. Ankur, good point there. Everyone's going to use it for something. And a lot of stuff gets brought to the table with large language models and other things. But the key ingredient will be proprietary data or human input, or some sort of AI whisperer kind of role, or prompt engineering, people are saying. So with that being said, some are saying it's automating intelligence. And that creativity will be unleashed from this. If the heavy lifting goes away and AI can fill the void, that shifts the value to the intellect or the input. And so that means data's got to come together, interact, fuse, and understand each other. This is kind of new. I mean, old school AI was, okay, got a big model, I provisioned it long time, very expensive. Now it's all free flowing. Can you guys comment on where you see this going with this freeform, data flowing everywhere, heavy lifting, and then specialization? >> Yeah, I think- >> Go ahead. >> Yeah, I think, so what we are seeing with these large language models or generative models is that they're really good at creating stuff. But I think it's also important to recognize their limitations. They're not as good at reasoning and logic. And I think now we're seeing great enthusiasm, I think, which is justified. And the next phase would be how to make these systems more reliable. How to inject more reasoning capabilities into these models, or augment with other mechanisms that actually perform more reasoning so we can achieve more reliable results. And we can count on these models to perform for critical tasks, whether it's medical tasks, legal tasks. We really want to kind of offload a lot of the intelligence to these systems. And then we'll have to get back, we'll have to make sure these are reliable, we'll have to make sure we get some sort of explainability that we can understand the process behind the generated results that we received. So I think this is kind of the next phase of systems that are based on these generated models. >> Clem, what's your view on this? Obviously you're at open community, open source has been around, it's been a great track record, proven model. I'm assuming creativity's going to come out of the woodwork, and if we can automate open source contribution, and relationships, and onboarding more developers, there's going to be unleashing of creativity. >> Yes, it's been so exciting on the open source front. We all know Bert, Bloom, GPT-J, T5, Stable Diffusion, that work up. The previous or the current generation of open source models that are on Hugging Face. It has been accelerating in the past few months. So I'm super excited about ControlNet right now that is really having a lot of impact, which is kind of like a way to control the generation of images. Super excited about Flan UL2, which is like a new model that has been recently released and is open source. So yeah, it's really fun to see the ecosystem coming together. Open source has been the basis for traditional software, with like open source programming languages, of course, but also all the great open source that we've gotten over the years. So we're happy to see that the same thing is happening for machine learning and AI, and hopefully can help a lot of companies reduce a little bit the barrier to entry. So yeah, it's going to be exciting to see how it evolves in the next few years in that respect. >> I think the developer productivity angle that's been talked about a lot in the industry will be accelerated significantly. I think security will be enhanced by this. I think in general, applications are going to transform at a radical rate, accelerated, incredible rate. So I think it's not a big wave, it's the water, right? I mean, (chuckles) it's the new thing. My final question for you guys, if you don't mind, I'd love to get each of you to answer the question I'm going to ask you, which is, a lot of conversations around data. Data infrastructure's obviously involved in this. And the common thread that I'm hearing is that every company that looks at this is asking themselves, if we don't rebuild our company, start thinking about rebuilding our business model around AI, we might be dinosaurs, we might be extinct. And it reminds me that scene in Moneyball when, at the end, it's like, if we're not building the model around your model, every company will be out of business. What's your advice to companies out there that are having those kind of moments where it's like, okay, this is real, this is next gen, this is happening. I better start thinking and putting into motion plans to refactor my business, 'cause it's happening, business transformation is happening on the cloud. This kind of puts an exclamation point on, with the AI, as a next step function. Big increase in value. So it's an opportunity for leaders. Ankur, we'll start with you. What's your advice for folks out there thinking about this? Do they put their toe in the water? Do they jump right into the deep end? What's your advice? >> Yeah, John, so we talk to a lot of customers, and customers are excited about what's happening in the space, but they often ask us like, "Hey, where do we start?" So we always advise our customers to do a lot of proof of concepts, understand where they can drive the biggest ROI. And then also leverage existing tools and services to move fast and scale, and try and not reinvent the wheel where it doesn't need to be. That's basically our advice to customers. >> Get it. Ori, what's your advice to folks who are scratching their head going, "I better jump in here. "How do I get started?" What's your advice? >> So I actually think that need to think about it really economically. Both on the opportunity side and the challenges. So there's a lot of opportunities for many companies to actually gain revenue upside by building these new generative features and capabilities. On the other hand, of course, this would probably affect the cogs, and incorporating these capabilities could probably affect the cogs. So I think we really need to think carefully about both of these sides, and also understand clearly if this is a project or an F word towards cost reduction, then the ROI is pretty clear, or revenue amplifier, where there's, again, a lot of different opportunities. So I think once you think about this in a structured way, I think, and map the different initiatives, then it's probably a good way to start and a good way to start thinking about these endeavors. >> Awesome. Clem, what's your take on this? What's your advice, folks out there? >> Yes, all of these are very good advice already. Something that you said before, John, that I disagreed a little bit, a lot of people are talking about the data mode and proprietary data. Actually, when you look at some of the organizations that have been building the best models, they don't have specialized or unique access to data. So I'm not sure that's so important today. I think what's important for companies, and it's been the same for the previous generation of technology, is their ability to build better technology faster than others. And in this new paradigm, that means being able to build machine learning faster than others, and better. So that's how, in my opinion, you should approach this. And kind of like how can you evolve your company, your teams, your products, so that you are able in the long run to build machine learning better and faster than your competitors. And if you manage to put yourself in that situation, then that's when you'll be able to differentiate yourself to really kind of be impactful and get results. That's really hard to do. It's something really different, because machine learning and AI is a different paradigm than traditional software. So this is going to be challenging, but I think if you manage to nail that, then the future is going to be very interesting for your company. >> That's a great point. Thanks for calling that out. I think this all reminds me of the cloud days early on. If you went to the cloud early, you took advantage of it when the pandemic hit. If you weren't native in the cloud, you got hamstrung by that, you were flatfooted. So just get in there. (laughs) Get in the cloud, get into AI, you're going to be good. Thanks for for calling that. Final parting comments, what's your most exciting thing going on right now for you guys? Ori, Clem, what's the most exciting thing on your plate right now that you'd like to share with folks? >> I mean, for me it's just the diversity of use cases and really creative ways of companies leveraging this technology. Every day I speak with about two, three customers, and I'm continuously being surprised by the creative ideas. And the future is really exciting of what can be achieved here. And also I'm amazed by the pace that things move in this industry. It's just, there's not at dull moment. So, definitely exciting times. >> Clem, what are you most excited about right now? >> For me, it's all the new open source models that have been released in the past few weeks, and that they'll keep being released in the next few weeks. I'm also super excited about more and more companies getting into this capability of chaining different models and different APIs. I think that's a very, very interesting development, because it creates new capabilities, new possibilities, new functionalities that weren't possible before. You can plug an API with an open source embedding model, with like a no-geo transcription model. So that's also very exciting. This capability of having more interoperable machine learning will also, I think, open a lot of interesting things in the future. >> Clem, congratulations on your success at Hugging Face. Please pass that on to your team. Ori, congratulations on your success, and continue to, just day one. I mean, it's just the beginning. It's not even scratching the service. Ankur, I'll give you the last word. What are you excited for at AWS? More cloud goodness coming here with AI. Give you the final word. >> Yeah, so as both Clem and Ori said, I think the research in the space is moving really, really fast, so we are excited about that. But we are also excited to see the speed at which enterprises and other AWS customers are applying machine learning to solve real business problems, and the kind of results they're seeing. So when they come back to us and tell us the kind of improvement in their business metrics and overall customer experience that they're driving and they're seeing real business results, that's what keeps us going and inspires us to continue inventing on their behalf. >> Gentlemen, thank you so much for this awesome high impact panel. Ankur, Clem, Ori, congratulations on all your success. We'll see you around. Thanks for coming on. Generative AI, riding the wave, it's a tidal wave, it's the water, it's all happening. All great stuff. This is season three, episode one of AWS Startup Showcase closing panel. This is the AI ML episode, the top startups building generative AI on AWS. I'm John Furrier, your host. Thanks for watching. (mellow music)

(chill electronic music) >> Hello everyone, welcome to theCUBE's presentation of the AWS Startups Showcase, AI and machine learning, the top startups building generative AI on AWS. This is the season three, episode one of the ongoing series covering the exciting startups from the AWS ecosystem, talk about AI and machine learning. Can't believe it's three years and season one. I'm your host, John Furrier. Got a great guest today, we're joined by Joseph Nelson, the co-founder and CEO of Roboflow, doing some cutting edge stuff around computer vision and really at the front end of this massive wave coming around, large language models, computer vision. The next gen AI is here, and it's just getting started. We haven't even scratched a service. Thanks for joining us today. >> Thanks for having me. >> So you got to love the large language model, foundation models, really educating the mainstream world. ChatGPT has got everyone in the frenzy. This is educating the world around this next gen AI capabilities, enterprise, image and video data, all a big part of it. I mean the edge of the network, Mobile World Conference is happening right now, this month, and it's just ending up, it's just continue to explode. Video is huge. So take us through the company, do a quick explanation of what you guys are doing, when you were founded. Talk about what the company's mission is, and what's your North Star, why do you exist? >> Yeah, Roboflow exists to really kind of make the world programmable. I like to say make the world be read and write access. And our North Star is enabling developers, predominantly, to build that future. If you look around, anything that you see will have software related to it, and can kind of be turned into software. The limiting reactant though, is how to enable computers and machines to understand things as well as people can. And in a lot of ways, computer vision is that missing element that enables anything that you see to become software. So in the virtue of, if software is eating the world, computer vision kind of makes the aperture infinitely wide. It's something that I kind of like, the way I like to frame it. And the capabilities are there, the open source models are there, the amount of data is there, the computer capabilities are only improving annually, but there's a pretty big dearth of tooling, and an early but promising sign of the explosion of use cases, models, and data sets that companies, developers, hobbyists alike will need to bring these capabilities to bear. So Roboflow is in the game of building the community around that capability, building the use cases that allow developers and enterprises to use computer vision, and providing the tooling for companies and developers to be able to add computer vision, create better data sets, and deploy to production, quickly, easily, safely, invaluably. >> You know, Joseph, the word in production is actually real now. You're seeing a lot more people doing in production activities. That's a real hot one and usually it's slower, but it's gone faster, and I think that's going to be more the same. And I think the parallel between what we're seeing on the large language models coming into computer vision, and as you mentioned, video's data, right? I mean we're doing video right now, we're transcribing it into a transcript, linking up to your linguistics, times and the timestamp, I mean everything's data and that really kind of feeds. So this connection between what we're seeing, the large language and computer vision are coming together kind of cousins, brothers. I mean, how would you compare, how would you explain to someone, because everyone's like on this wave of watching people bang out their homework assignments, and you know, write some hacks on code with some of the open AI technologies, there is a corollary directly related to to the vision side. Can you explain? >> Yeah, the rise of large language models are showing what's possible, especially with text, and I think increasingly will get multimodal as the images and video become ingested. Though there's kind of this still core missing element of basically like understanding. So the rise of large language models kind of create this new area of generative AI, and generative AI in the context of computer vision is a lot of, you know, creating video and image assets and content. There's also this whole surface area to understanding what's already created. Basically digitizing physical, real world things. I mean the Metaverse can't be built if we don't know how to mirror or create or identify the objects that we want to interact with in our everyday lives. And where computer vision comes to play in, especially what we've seen at Roboflow is, you know, a little over a hundred thousand developers now have built with our tools. That's to the tune of a hundred million labeled open source images, over 10,000 pre-trained models. And they've kind of showcased to us all of the ways that computer vision is impacting and bringing the world to life. And these are things that, you know, even before large language models and generative AI, you had pretty impressive capabilities, and when you add the two together, it actually unlocks these kind of new capabilities. So for example, you know, one of our users actually powers the broadcast feeds at Wimbledon. So here we're talking about video, we're streaming, we're doing things live, we've got folks that are cropping and making sure we look good, and audio/visual all plugged in correctly. When you broadcast Wimbledon, you'll notice that the camera controllers need to do things like track the ball, which is moving at extremely high speeds and zoom crop, pan tilt, as well as determine if the ball bounced in or out. The very controversial but critical key to a lot of tennis matches. And a lot of that has been historically done with the trained, but fallible human eye and computer vision is, you know, well suited for this task to say, how do we track, pan, tilt, zoom, and see, track the tennis ball in real time, run at 30 plus frames per second, and do it all on the edge. And those are capabilities that, you know, were kind of like science fiction, maybe even a decade ago, and certainly five years ago. Now the interesting thing, is that with the advent of of generative AI, you can start to do things like create your own training data sets, or kind of create logic around once you have this visual input. And teams at Tesla have actually been speaking about, of course the autopilot team's focused on doing vision tasks, but they've combined large language models to add reasoning and logic. So given that you see, let's say the tennis ball, what do you want to do? And being able to combine the capabilities of what LLM's represent, which is really a lot of basically, core human reasoning and logic, with computer vision for the inputs of what's possible, creates these new capabilities, let alone multimodality, which I'm sure we'll talk more about. >> Yeah, and it's really, I mean it's almost intoxicating. It's amazing that this is so capable because the cloud scales here, you got the edge developing, you can decouple compute power, and let Moore's law and all the new silicone and the processors and the GPUs do their thing, and you got open source booming. You're kind of getting at this next segment I wanted to get into, which is the, how people should be thinking about these advances of the computer vision. So this is now a next wave, it's here. I mean I'd love to have that for baseball because I'm always like, "Oh, it should have been a strike." I'm sure that's going to be coming soon, but what is the computer vision capable of doing today? I guess that's my first question. You hit some of it, unpack that a little bit. What does general AI mean in computer vision? What's the new thing? Because there are old technology's been around, proprietary, bolted onto hardware, but hardware advances at a different pace, but now you got new capabilities, generative AI for vision, what does that mean? >> Yeah, so computer vision, you know, at its core is basically enabling machines, computers, to understand, process, and act on visual data as effective or more effective than people can. Traditionally this has been, you know, task types like classification, which you know, identifying if a given image belongs in a certain category of goods on maybe a retail site, is the shoes or is it clothing? Or object detection, which is, you know, creating bounding boxes, which allows you to do things like count how many things are present, or maybe measure the speed of something, or trigger an alert when something becomes visible in frame that wasn't previously visible in frame, or instant segmentation where you're creating pixel wise segmentations for both instance and semantic segmentation, where you often see these kind of beautiful visuals of the polygon surrounding objects that you see. Then you have key point detection, which is where you see, you know, athletes, and each of their joints are kind of outlined is another more traditional type problem in signal processing and computer vision. With generative AI, you kind of get a whole new class of problem types that are opened up. So in a lot of ways I think about generative AI in computer vision as some of the, you know, problems that you aimed to tackle, might still be better suited for one of the previous task types we were discussing. Some of those problem types may be better suited for using a generative technique, and some are problem types that just previously wouldn't have been possible absent generative AI. And so if you make that kind of Venn diagram in your head, you can think about, okay, you know, visual question answering is a task type where if I give you an image and I say, you know, "How many people are in this image?" We could either build an object detection model that might count all those people, or maybe a visual question answering system would sufficiently answer this type of problem. Let alone generative AI being able to create new training data for old systems. And that's something that we've seen be an increasingly prominent use case for our users, as much as things that we advise our customers and the community writ large to take advantage of. So ultimately those are kind of the traditional task types. I can give you some insight, maybe, into how I think about what's possible today, or five years or ten years as you sort go back. >> Yes, definitely. Let's get into that vision. >> So I kind of think about the types of use cases in terms of what's possible. If you just imagine a very simple bell curve, your normal distribution, for the longest time, the types of things that are in the center of that bell curve are identifying objects that are very common or common objects in context. Microsoft published the COCO Dataset in 2014 of common objects and contexts, of hundreds of thousands of images of chairs, forks, food, person, these sorts of things. And you know, the challenge of the day had always been, how do you identify just those 80 objects? So if we think about the bell curve, that'd be maybe the like dead center of the curve, where there's a lot of those objects present, and it's a very common thing that needs to be identified. But it's a very, very, very small sliver of the distribution. Now if you go out to the way long tail, let's go like deep into the tail of this imagined visual normal distribution, you're going to have a problem like one of our customers, Rivian, in tandem with AWS, is tackling, to do visual quality assurance and manufacturing in production processes. Now only Rivian knows what a Rivian is supposed to look like. Only they know the imagery of what their goods that are going to be produced are. And then between those long tails of proprietary data of highly specific things that need to be understood, in the center of the curve, you have a whole kind of messy middle, type of problems I like to say. The way I think about computer vision advancing, is it's basically you have larger and larger and more capable models that eat from the center out, right? So if you have a model that, you know, understands the 80 classes in COCO, well, pretty soon you have advances like Clip, which was trained on 400 million image text pairs, and has a greater understanding of a wider array of objects than just 80 classes in context. And over time you'll get more and more of these larger models that kind of eat outwards from that center of the distribution. And so the question becomes for companies, when can you rely on maybe a model that just already exists? How do you use your data to get what may be capable off the shelf, so to speak, into something that is usable for you? Or, if you're in those long tails and you have proprietary data, how do you take advantage of the greatest asset you have, which is observed visual information that you want to put to work for your customers, and you're kind of living in the long tails, and you need to adapt state of the art for your capabilities. So my mental model for like how computer vision advances is you have that bell curve, and you have increasingly powerful models that eat outward. And multimodality has a role to play in that, larger models have a role to play in that, more compute, more data generally has a role to play in that. But it will be a messy and I think long condition. >> Well, the thing I want to get, first of all, it's great, great mental model, I appreciate that, 'cause I think that makes a lot of sense. The question is, it seems now more than ever, with the scale and compute that's available, that not only can you eat out to the middle in your example, but there's other models you can integrate with. In the past there was siloed, static, almost bespoke. Now you're looking at larger models eating into the bell curve, as you said, but also integrating in with other stuff. So this seems to be part of that interaction. How does, first of all, is that really happening? Is that true? And then two, what does that mean for companies who want to take advantage of this? Because the old model was operational, you know? I have my cameras, they're watching stuff, whatever, and like now you're in this more of a, distributed computing, computer science mindset, not, you know, put the camera on the wall kind of- I'm oversimplifying, but you know what I'm saying. What's your take on that? >> Well, to the first point of, how are these advances happening? What I was kind of describing was, you know, almost uni-dimensional in that you have like, you're only thinking about vision, but the rise of generative techniques and multi-modality, like Clip is a multi-modal model, it has 400 million image text pairs. That will advance the generalizability at a faster rate than just treating everything as only vision. And that's kind of where LLMs and vision will intersect in a really nice and powerful way. Now in terms of like companies, how should they be thinking about taking advantage of these trends? The biggest thing that, and I think it's different, obviously, on the size of business, if you're an enterprise versus a startup. The biggest thing that I think if you're an enterprise, and you have an established scaled business model that is working for your customers, the question becomes, how do you take advantage of that established data moat, potentially, resource moats, and certainly, of course, establish a way of providing value to an end user. So for example, one of our customers, Walmart, has the advantage of one of the largest inventory and stock of any company in the world. And they also of course have substantial visual data, both from like their online catalogs, or understanding what's in stock or out of stock, or understanding, you know, the quality of things that they're going from the start of their supply chain to making it inside stores, for delivery of fulfillments. All these are are visual challenges. Now they already have a substantial trove of useful imagery to understand and teach and train large models to understand each of the individual SKUs and products that are in their stores. And so if I'm a Walmart, what I'm thinking is, how do I make sure that my petabytes of visual information is utilized in a way where I capture the proprietary benefit of the models that I can train to do tasks like, what item was this? Or maybe I'm going to create AmazonGo-like technology, or maybe I'm going to build like delivery robots, or I want to automatically know what's in and out of stock from visual input fees that I have across my in-store traffic. And that becomes the question and flavor of the day for enterprises. I've got this large amount of data, I've got an established way that I can provide more value to my own customers. How do I ensure I take advantage of the data advantage I'm already sitting on? If you're a startup, I think it's a pretty different question, and I'm happy to talk about. >> Yeah, what's startup angle on this? Because you know, they're going to want to take advantage. It's like cloud startups, cloud native startups, they were born in the cloud, they never had an IT department. So if you're a startup, is there a similar role here? And if I'm a computer vision startup, what's that mean? So can you share your your take on that, because there'll be a lot of people starting up from this. >> So the startup on the opposite advantage and disadvantage, right? Like a startup doesn't have an proven way of delivering repeatable value in the same way that a scaled enterprise does. But it does have the nimbleness to identify and take advantage of techniques that you can start from a blank slate. And I think the thing that startups need to be wary of in the generative AI enlarged language model, in multimodal world, is building what I like to call, kind of like sandcastles. A sandcastle is maybe a business model or a capability that's built on top of an assumption that is going to be pretty quickly wiped away by improving underlying model technology. So almost like if you imagine like the ocean, the waves are coming in, and they're going to wipe away your progress. You don't want to be in the position of building sandcastle business where, you don't want to bet on the fact that models aren't going to get good enough to solve the task type that you might be solving. In other words, don't take a screenshot of what's capable today. Assume that what's capable today is only going to continue to become possible. And so for a startup, what you can do, that like enterprises are quite comparatively less good at, is embedding these capabilities deeply within your products and delivering maybe a vertical based experience, where AI kind of exists in the background. >> Yeah. >> And we might not think of companies as, you know, even AI companies, it's just so embedded in the experience they provide, but that's like the vertical application example of taking AI and making it be immediately usable. Or, of course there's tons of picks and shovels businesses to be built like Roboflow, where you're enabling these enterprises to take advantage of something that they have, whether that's their data sets, their computes, or their intellect. >> Okay, so if I hear that right, by the way, I love, that's horizontally scalable, that's the large language models, go up and build them the apps, hence your developer focus. I'm sure that's probably the reason that the tsunami of developer's action. So you're saying picks and shovels tools, don't try to replicate the platform of what could be the platform. Oh, go to a VC, I'm going to build a platform. No, no, no, no, those are going to get wiped away by the large language models. Is there one large language model that will rule the world, or do you see many coming? >> Yeah, so to be clear, I think there will be useful platforms. I just think a lot of people think that they're building, let's say, you know, if we put this in the cloud context, you're building a specific type of EC2 instance. Well, it turns out that Amazon can offer that type of EC2 instance, and immediately distribute it to all of their customers. So you don't want to be in the position of just providing something that actually ends up looking like a feature, which in the context of AI, might be like a small incremental improvement on the model. If that's all you're doing, you're a sandcastle business. Now there's a lot of platform businesses that need to be built that enable businesses to get to value and do things like, how do I monitor my models? How do I create better models with my given data sets? How do I ensure that my models are doing what I want them to do? How do I find the right models to use? There's all these sorts of platform wide problems that certainly exist for businesses. I just think a lot of startups that I'm seeing right now are making the mistake of assuming the advances we're seeing are not going to accelerate or even get better. >> So if I'm a customer, if I'm a company, say I'm a startup or an enterprise, either one, same question. And I want to stand up, and I have developers working on stuff, I want to start standing up an environment to start doing stuff. Is that a service provider? Is that a managed service? Is that you guys? So how do you guys fit into your customers leaning in? Is it just for developers? Are you targeting with a specific like managed service? What's the product consumption? How do you talk to customers when they come to you? >> The thing that we do is enable, we give developers superpowers to build automated inventory tracking, self-checkout systems, identify if this image is malignant cancer or benign cancer, ensure that these products that I've produced are correct. Make sure that that the defect that might exist on this electric vehicle makes its way back for review. All these sorts of problems are immediately able to be solved and tackled. In terms of the managed services element, we have solutions as integrators that will often build on top of our tools, or we'll have companies that look to us for guidance, but ultimately the company is in control of developing and building and creating these capabilities in house. I really think the distinction is maybe less around managed service and tool, and more around ownership in the era of AI. So for example, if I'm using a managed service, in that managed service, part of their benefit is that they are learning across their customer sets, then it's a very different relationship than using a managed service where I'm developing some amount of proprietary advantages for my data sets. And I think that's a really important thing that companies are becoming attuned to, just the value of the data that they have. And so that's what we do. We tell companies that you have this proprietary, immense treasure trove of data, use that to your advantage, and think about us more like a set of tools that enable you to get value from that capability. You know, the HashiCorp's and GitLab's of the world have proven like what these businesses look like at scale. >> And you're targeting developers. When you go into a company, do you target developers with freemium, is there a paid service? Talk about the business model real quick. >> Sure, yeah. The tools are free to use and get started. When someone signs up for Roboflow, they may elect to make their work open source, in which case we're able to provide even more generous usage limits to basically move the computer vision community forward. If you elect to make your data private, you can use our hosted data set managing, data set training, model deployment, annotation tooling up to some limits. And then usually when someone validates that what they're doing gets them value, they purchase a subscription license to be able to scale up those capabilities. So like most developer centric products, it's free to get started, free to prove, free to poke around, develop what you think is possible. And then once you're getting to value, then we're able to capture the commercial upside in the value that's being provided. >> Love the business model. It's right in line with where the market is. There's kind of no standards bodies these days. The developers are the ones who are deciding kind of what the standards are by their adoption. I think making that easy for developers to get value as the model open sources continuing to grow, you can see more of that. Great perspective Joseph, thanks for sharing that. Put a plug in for the company. What are you guys doing right now? Where are you in your growth? What are you looking for? How should people engage? Give the quick commercial for the company. >> So as I mentioned, Roboflow is I think one of the largest, if not the largest collections of computer vision models and data sets that are open source, available on the web today, and have a private set of tools that over half the Fortune 100 now rely on those tools. So we're at the stage now where we know people want what we're working on, and we're continuing to drive that type of adoption. So companies that are looking to make better models, improve their data sets, train and deploy, often will get a lot of value from our tools, and certainly reach out to talk. I'm sure there's a lot of talented engineers that are tuning in too, we're aggressively hiring. So if you are interested in being a part of making the world programmable, and being at the ground floor of the company that's creating these capabilities to be writ large, we'd love to hear from you. >> Amazing, Joseph, thanks so much for coming on and being part of the AWS Startup Showcase. Man, if I was in my twenties, I'd be knocking on your door, because it's the hottest trend right now, it's super exciting. Generative AI is just the beginning of massive sea change. Congratulations on all your success, and we'll be following you guys. Thanks for spending the time, really appreciate it. >> Thanks for having me. >> Okay, this is season three, episode one of the ongoing series covering the exciting startups from the AWS ecosystem, talking about the hottest things in tech. I'm John Furrier, your host. Thanks for watching. (chill electronic music)

(upbeat music) >> Welcome everyone to theCUBE's presentation of the AWS Startup Showcase AI Machine Learning Top Startups Building Generative AI on AWS. This is season 3, episode 1 of the ongoing series covering the exciting startup from the AWS ecosystem to talk about AI and machine learning. I'm your host, John Furrier. I'm joined by two great guests here, Adam Wenchel, who's the CEO of Arthur, and Chief Scientist of Arthur, John Dickerson. Talk about how they help people build better LLM AI systems to get them into the market faster. Gentlemen, thank you for coming on. >> Yeah, thanks for having us, John. >> Well, I got to say I got to temper my enthusiasm because the last few months explosion of interest in LLMs with ChatGPT, has opened the eyes to everybody around the reality of that this is going next gen, this is it, this is the moment, this is the the point we're going to look back and say, this is the time where AI really hit the scene for real applications. So, a lot of Large Language Models, also known as LLMs, foundational models, and generative AI is all booming. This is where all the alpha developers are going. This is where everyone's focusing their business model transformations on. This is where developers are seeing action. So it's all happening, the wave is here. So I got to ask you guys, what are you guys seeing right now? You're in the middle of it, it's hitting you guys right on. You're in the front end of this massive wave. >> Yeah, John, I don't think you have to temper your enthusiasm at all. I mean, what we're seeing every single day is, everything from existing enterprise customers coming in with new ways that they're rethinking, like business things that they've been doing for many years that they can now do an entirely different way, as well as all manner of new companies popping up, applying LLMs to everything from generating code and SQL statements to generating health transcripts and just legal briefs. Everything you can imagine. And when you actually sit down and look at these systems and the demos we get of them, the hype is definitely justified. It's pretty amazing what they're going to do. And even just internally, we built, about a month ago in January, we built an Arthur chatbot so customers could ask questions, technical questions from our, rather than read our product documentation, they could just ask this LLM a particular question and get an answer. And at the time it was like state of the art, but then just last week we decided to rebuild it because the tooling has changed so much that we, last week, we've completely rebuilt it. It's now way better, built on an entirely different stack. And the tooling has undergone a full generation worth of change in six weeks, which is crazy. So it just tells you how much energy is going into this and how fast it's evolving right now. >> John, weigh in as a chief scientist. I mean, you must be blown away. Talk about kid in the candy store. I mean, you must be looking like this saying, I mean, she must be super busy to begin with, but the change, the acceleration, can you scope the kind of change you're seeing and be specific around the areas you're seeing movement and highly accelerated change? >> Yeah, definitely. And it is very, very exciting actually, thinking back to when ChatGPT was announced, that was a night our company was throwing an event at NeurIPS, which is maybe the biggest machine learning conference out there. And the hype when that happened was palatable and it was just shocking to see how well that performed. And then obviously over the last few months since then, as LLMs have continued to enter the market, we've seen use cases for them, like Adam mentioned all over the place. And so, some things I'm excited about in this space are the use of LLMs and more generally, foundation models to redesign traditional operations, research style problems, logistics problems, like auctions, decisioning problems. So moving beyond the already amazing news cases, like creating marketing content into more core integration and a lot of the bread and butter companies and tasks that drive the American ecosystem. And I think we're just starting to see some of that. And in the next 12 months, I think we're going to see a lot more. If I had to make other predictions, I think we're going to continue seeing a lot of work being done on managing like inference time costs via shrinking models or distillation. And I don't know how to make this prediction, but at some point we're going to be seeing lots of these very large scale models operating on the edge as well. So the time scales are extremely compressed, like Adam mentioned, 12 months from now, hard to say. >> We were talking on theCUBE prior to this session here. We had theCUBE conversation here and then the Wall Street Journal just picked up on the same theme, which is the printing press moment created the enlightenment stage of the history. Here we're in the whole nother automating intellect efficiency, doing heavy lifting, the creative class coming back, a whole nother level of reality around the corner that's being hyped up. The question is, is this justified? Is there really a breakthrough here or is this just another result of continued progress with AI? Can you guys weigh in, because there's two schools of thought. There's the, "Oh my God, we're entering a new enlightenment tech phase, of the equivalent of the printing press in all areas. Then there's, Ah, it's just AI (indistinct) inch by inch. What's your guys' opinion? >> Yeah, I think on the one hand when you're down in the weeds of building AI systems all day, every day, like we are, it's easy to look at this as an incremental progress. Like we have customers who've been building on foundation models since we started the company four years ago, particular in computer vision for classification tasks, starting with pre-trained models, things like that. So that part of it doesn't feel real new, but what does feel new is just when you apply these things to language with all the breakthroughs and computational efficiency, algorithmic improvements, things like that, when you actually sit down and interact with ChatGPT or one of the other systems that's out there that's building on top of LLMs, it really is breathtaking, like, the level of understanding that they have and how quickly you can accelerate your development efforts and get an actual working system in place that solves a really important real world problem and makes people way faster, way more efficient. So I do think there's definitely something there. It's more than just incremental improvement. This feels like a real trajectory inflection point for the adoption of AI. >> John, what's your take on this? As people come into the field, I'm seeing a lot of people move from, hey, I've been coding in Python, I've been doing some development, I've been a software engineer, I'm a computer science student. I'm coding in C++ old school, OG systems person. Where do they come in? Where's the focus, where's the action? Where are the breakthroughs? Where are people jumping in and rolling up their sleeves and getting dirty with this stuff? >> Yeah, all over the place. And it's funny you mentioned students in a different life. I wore a university professor hat and so I'm very, very familiar with the teaching aspects of this. And I will say toward Adam's point, this really is a leap forward in that techniques like in a co-pilot for example, everybody's using them right now and they really do accelerate the way that we develop. When I think about the areas where people are really, really focusing right now, tooling is certainly one of them. Like you and I were chatting about LangChain right before this interview started, two or three people can sit down and create an amazing set of pipes that connect different aspects of the LLM ecosystem. Two, I would say is in engineering. So like distributed training might be one, or just understanding better ways to even be able to train large models, understanding better ways to then distill them or run them. So like this heavy interaction now between engineering and what I might call traditional machine learning from 10 years ago where you had to know a lot of math, you had to know calculus very well, things like that. Now you also need to be, again, a very strong engineer, which is exciting. >> I interviewed Swami when he talked about the news. He's ahead of Amazon's machine learning and AI when they announced Hugging Face announcement. And I reminded him how Amazon was easy to get into if you were developing a startup back in 2007,8, and that the language models had that similar problem. It's step up a lot of content and a lot of expense to get provisioned up, now it's easy. So this is the next wave of innovation. So how do you guys see that from where we are right now? Are we at that point where it's that moment where it's that cloud-like experience for LLMs and large language models? >> Yeah, go ahead John. >> I think the answer is yes. We see a number of large companies that are training these and serving these, some of which are being co-interviewed in this episode. I think we're at that. Like, you can hit one of these with a simple, single line of Python, hitting an API, you can boot this up in seconds if you want. It's easy. >> Got it. >> So I (audio cuts out). >> Well let's take a step back and talk about the company. You guys being featured here on the Showcase. Arthur, what drove you to start the company? How'd this all come together? What's the origination story? Obviously you got a big customers, how'd get started? What are you guys doing? How do you make money? Give a quick overview. >> Yeah, I think John and I come at it from slightly different angles, but for myself, I have been a part of a number of technology companies. I joined Capital One, they acquired my last company and shortly after I joined, they asked me to start their AI team. And so even though I've been doing AI for a long time, I started my career back in DARPA. It was the first time I was really working at scale in AI at an organization where there were hundreds of millions of dollars in revenue at stake with the operation of these models and that they were impacting millions of people's financial livelihoods. And so it just got me hyper-focused on these issues around making sure that your AI worked well and it worked well for your company and it worked well for the people who were being affected by it. At the time when I was doing this 2016, 2017, 2018, there just wasn't any tooling out there to support this production management model monitoring life phase of the life cycle. And so we basically left to start the company that I wanted. And John has a his own story. I'll let let you share that one, John. >> Go ahead John, you're up. >> Yeah, so I'm coming at this from a different world. So I'm on leave now from a tenured role in academia where I was leading a large lab focusing on the intersection of machine learning and economics. And so questions like fairness or the response to the dynamism on the underlying environment have been around for quite a long time in that space. And so I've been thinking very deeply about some of those more like R and D style questions as well as having deployed some automation code across a couple of different industries, some in online advertising, some in the healthcare space and so on, where concerns of, again, fairness come to bear. And so Adam and I connected to understand the space of what that might look like in the 2018 20 19 realm from a quantitative and from a human-centered point of view. And so booted things up from there. >> Yeah, bring that applied engineering R and D into the Capital One, DNA that he had at scale. I could see that fit. I got to ask you now, next step, as you guys move out and think about LLMs and the recent AI news around the generative models and the foundational models like ChatGPT, how should we be looking at that news and everyone watching might be thinking the same thing. I know at the board level companies like, we should refactor our business, this is the future. It's that kind of moment, and the tech team's like, okay, boss, how do we do this again? Or are they prepared? How should we be thinking? How should people watching be thinking about LLMs? >> Yeah, I think they really are transformative. And so, I mean, we're seeing companies all over the place. Everything from large tech companies to a lot of our large enterprise customers are launching significant projects at core parts of their business. And so, yeah, I would be surprised, if you're serious about becoming an AI native company, which most leading companies are, then this is a trend that you need to be taking seriously. And we're seeing the adoption rate. It's funny, I would say the AI adoption in the broader business world really started, let's call it four or five years ago, and it was a relatively slow adoption rate, but I think all that kind of investment in and scaling the maturity curve has paid off because the rate at which people are adopting and deploying systems based on this is tremendous. I mean, this has all just happened in the few months and we're already seeing people get systems into production. So, now there's a lot of things you have to guarantee in order to put these in production in a way that basically is added into your business and doesn't cause more headaches than it solves. And so that's where we help customers is where how do you put these out there in a way that they're going to represent your company well, they're going to perform well, they're going to do their job and do it properly. >> So in the use case, as a customer, as I think about this, there's workflows. They might have had an ML AI ops team that's around IT. Their inference engines are out there. They probably don't have a visibility on say how much it costs, they're kicking the tires. When you look at the deployment, there's a cost piece, there's a workflow piece, there's fairness you mentioned John, what should be, I should be thinking about if I'm going to be deploying stuff into production, I got to think about those things. What's your opinion? >> Yeah, I'm happy to dive in on that one. So monitoring in general is extremely important once you have one of these LLMs in production, and there have been some changes versus traditional monitoring that we can dive deeper into that LLMs are really accelerated. But a lot of that bread and butter style of things you should be looking out for remain just as important as they are for what you might call traditional machine learning models. So the underlying environment of data streams, the way users interact with these models, these are all changing over time. And so any performance metrics that you care about, traditional ones like an accuracy, if you can define that for an LLM, ones around, for example, fairness or bias. If that is a concern for your particular use case and so on. Those need to be tracked. Now there are some interesting changes that LLMs are bringing along as well. So most ML models in production that we see are relatively static in the sense that they're not getting flipped in more than maybe once a day or once a week or they're just set once and then not changed ever again. With LLMs, there's this ongoing value alignment or collection of preferences from users that is often constantly updating the model. And so that opens up all sorts of vectors for, I won't say attack, but for problems to arise in production. Like users might learn to use your system in a different way and thus change the way those preferences are getting collected and thus change your system in ways that you never intended. So maybe that went through governance already internally at the company and now it's totally, totally changed and it's through no fault of your own, but you need to be watching over that for sure. >> Talk about the reinforced learnings from human feedback. How's that factoring in to the LLMs? Is that part of it? Should people be thinking about that? Is that a component that's important? >> It certainly is, yeah. So this is one of the big tweaks that happened with InstructGPT, which is the basis model behind ChatGPT and has since gone on to be used all over the place. So value alignment I think is through RLHF like you mentioned is a very interesting space to get into and it's one that you need to watch over. Like, you're asking humans for feedback over outputs from a model and then you're updating the model with respect to that human feedback. And now you've thrown humans into the loop here in a way that is just going to complicate things. And it certainly helps in many ways. You can ask humans to, let's say that you're deploying an internal chat bot at an enterprise, you could ask humans to align that LLM behind the chatbot to, say company values. And so you're listening feedback about these company values and that's going to scoot that chatbot that you're running internally more toward the kind of language that you'd like to use internally on like a Slack channel or something like that. Watching over that model I think in that specific case, that's a compliance and HR issue as well. So while it is part of the greater LLM stack, you can also view that as an independent bit to watch over. >> Got it, and these are important factors. When people see the Bing news, they freak out how it's doing great. Then it goes off the rails, it goes big, fails big. (laughing) So these models people see that, is that human interaction or is that feedback, is that not accepting it or how do people understand how to take that input in and how to build the right apps around LLMs? This is a tough question. >> Yeah, for sure. So some of the examples that you'll see online where these chatbots go off the rails are obviously humans trying to break the system, but some of them clearly aren't. And that's because these are large statistical models and we don't know what's going to pop out of them all the time. And even if you're doing as much in-house testing at the big companies like the Go-HERE's and the OpenAI's of the world, to try to prevent things like toxicity or racism or other sorts of bad content that might lead to bad pr, you're never going to catch all of these possible holes in the model itself. And so, again, it's very, very important to keep watching over that while it's in production. >> On the business model side, how are you guys doing? What's the approach? How do you guys engage with customers? Take a minute to explain the customer engagement. What do they need? What do you need? How's that work? >> Yeah, I can talk a little bit about that. So it's really easy to get started. It's literally a matter of like just handing out an API key and people can get started. And so we also offer alternative, we also offer versions that can be installed on-prem for models that, we find a lot of our customers have models that deal with very sensitive data. So you can run it in your cloud account or use our cloud version. And so yeah, it's pretty easy to get started with this stuff. We find people start using it a lot of times during the validation phase 'cause that way they can start baselining performance models, they can do champion challenger, they can really kind of baseline the performance of, maybe they're considering different foundation models. And so it's a really helpful tool for understanding differences in the way these models perform. And then from there they can just flow that into their production inferencing, so that as these systems are out there, you have really kind of real time monitoring for anomalies and for all sorts of weird behaviors as well as that continuous feedback loop that helps you make make your product get better and observability and you can run all sorts of aggregated reports to really understand what's going on with these models when they're out there deciding. I should also add that we just today have another way to adopt Arthur and that is we are in the AWS marketplace, and so we are available there just to make it that much easier to use your cloud credits, skip the procurement process, and get up and running really quickly. >> And that's great 'cause Amazon's got SageMaker, which handles a lot of privacy stuff, all kinds of cool things, or you can get down and dirty. So I got to ask on the next one, production is a big deal, getting stuff into production. What have you guys learned that you could share to folks watching? Is there a cost issue? I got to monitor, obviously you brought that up, we talked about the even reinforcement issues, all these things are happening. What is the big learnings that you could share for people that are going to put these into production to watch out for, to plan for, or be prepared for, hope for the best plan for the worst? What's your advice? >> I can give a couple opinions there and I'm sure Adam has. Well, yeah, the big one from my side is, again, I had mentioned this earlier, it's just the input data streams because humans are also exploring how they can use these systems to begin with. It's really, really hard to predict the type of inputs you're going to be seeing in production. Especially, we always talk about chatbots, but then any generative text tasks like this, let's say you're taking in news articles and summarizing them or something like that, it's very hard to get a good sampling even of the set of news articles in such a way that you can really predict what's going to pop out of that model. So to me, it's, adversarial maybe isn't the word that I would use, but it's an unnatural shifting input distribution of like prompts that you might see for these models. That's certainly one. And then the second one that I would talk about is, it can be hard to understand the costs, the inference time costs behind these LLMs. So the pricing on these is always changing as the models change size, it might go up, it might go down based on model size, based on energy cost and so on, but your pricing per token or per a thousand tokens and that I think can be difficult for some clients to wrap their head around. Again, you don't know how these systems are going to be used after all so it can be tough. And so again that's another metric that really should be tracked. >> Yeah, and there's a lot of trade off choices in there with like, how many tokens do you want at each step and in the sequence and based on, you have (indistinct) and you reject these tokens and so based on how your system's operating, that can make the cost highly variable. And that's if you're using like an API version that you're paying per token. A lot of people also choose to run these internally and as John mentioned, the inference time on these is significantly higher than a traditional classifi, even NLP classification model or tabular data model, like orders of magnitude higher. And so you really need to understand how that, as you're constantly iterating on these models and putting out new versions and new features in these models, how that's affecting the overall scale of that inference cost because you can use a lot of computing power very quickly with these profits. >> Yeah, scale, performance, price all come together. I got to ask while we're here on the secret sauce of the company, if you had to describe to people out there watching, what's the secret sauce of the company? What's the key to your success? >> Yeah, so John leads our research team and they've had a number of really cool, I think AI as much as it's been hyped for a while, it's still commercial AI at least is really in its infancy. And so the way we're able to pioneer new ways to think about performance for computer vision NLP LLMs is probably the thing that I'm proudest about. John and his team publish papers all the time at Navs and other places. But I think it's really being able to define what performance means for basically any kind of model type and give people really powerful tools to understand that on an ongoing basis. >> John, secret sauce, how would you describe it? You got all the action happening all around you. >> Yeah, well I going to appreciate Adam talking me up like that. No, I. (all laughing) >> Furrier: Robs to you. >> I would also say a couple of other things here. So we have a very strong engineering team and so I think some early hires there really set the standard at a very high bar that we've maintained as we've grown. And I think that's really paid dividends as scalabilities become even more of a challenge in these spaces, right? And so that's not just scalability when it comes to LLMs, that's scalability when it comes to millions of inferences per day, that kind of thing as well in traditional ML models. And I think that's compared to potential competitors, that's really... Well, it's made us able to just operate more efficiently and pass that along to the client. >> Yeah, and I think the infancy comment is really important because it's the beginning. You really is a long journey ahead. A lot of change coming, like I said, it's a huge wave. So I'm sure you guys got a lot of plannings at the foundation even for your own company, so I appreciate the candid response there. Final question for you guys is, what should the top things be for a company in 2023? If I'm going to set the agenda and I'm a customer moving forward, putting the pedal to the metal, so to speak, what are the top things I should be prioritizing or I need to do to be successful with AI in 2023? >> Yeah, I think, so number one, as we talked about, we've been talking about this entire episode, the things are changing so quickly and the opportunities for business transformation and really disrupting different applications, different use cases, is almost, I don't think we've even fully comprehended how big it is. And so really digging in to your business and understanding where I can apply these new sets of foundation models is, that's a top priority. The interesting thing is I think there's another force at play, which is the macroeconomic conditions and a lot of places are, they're having to work harder to justify budgets. So in the past, couple years ago maybe, they had a blank check to spend on AI and AI development at a lot of large enterprises that was limited primarily by the amount of talent they could scoop up. Nowadays these expenditures are getting scrutinized more. And so one of the things that we really help our customers with is like really calculating the ROI on these things. And so if you have models out there performing and you have a new version that you can put out that lifts the performance by 3%, how many tens of millions of dollars does that mean in business benefit? Or if I want to go to get approval from the CFO to spend a few million dollars on this new project, how can I bake in from the beginning the tools to really show the ROI along the way? Because I think in these systems when done well for a software project, the ROI can be like pretty spectacular. Like we see over a hundred percent ROI in the first year on some of these projects. And so, I think in 2023, you just need to be able to show what you're getting for that spend. >> It's a needle moving moment. You see it all the time with some of these aha moments or like, whoa, blown away. John, I want to get your thoughts on this because one of the things that comes up a lot for companies that I talked to, that are on my second wave, I would say coming in, maybe not, maybe the front wave of adopters is talent and team building. You mentioned some of the hires you got were game changing for you guys and set the bar high. As you move the needle, new developers going to need to come in. What's your advice given that you've been a professor, you've seen students, I know a lot of computer science people want to shift, they might not be yet skilled in AI, but they're proficient in programming, is that's going to be another opportunity with open source when things are happening. How do you talk to that next level of talent that wants to come in to this market to supplement teams and be on teams, lead teams? Any advice you have for people who want to build their teams and people who are out there and want to be a coder in AI? >> Yeah, I've advice, and this actually works for what it would take to be a successful AI company in 2023 as well, which is, just don't be afraid to iterate really quickly with these tools. The space is still being explored on what they can be used for. A lot of the tasks that they're used for now right? like creating marketing content using a machine learning is not a new thing to do. It just works really well now. And so I'm excited to see what the next year brings in terms of folks from outside of core computer science who are, other engineers or physicists or chemists or whatever who are learning how to use these increasingly easy to use tools to leverage LLMs for tasks that I think none of us have really thought about before. So that's really, really exciting. And so toward that I would say iterate quickly. Build things on your own, build demos, show them the friends, host them online and you'll learn along the way and you'll have somebody to show for it. And also you'll help us explore that space. >> Guys, congratulations with Arthur. Great company, great picks and shovels opportunities out there for everybody. Iterate fast, get in quickly and don't be afraid to iterate. Great advice and thank you for coming on and being part of the AWS showcase, thanks. >> Yeah, thanks for having us on John. Always a pleasure. >> Yeah, great stuff. Adam Wenchel, John Dickerson with Arthur. Thanks for coming on theCUBE. I'm John Furrier, your host. Generative AI and AWS. Keep it right there for more action with theCUBE. Thanks for watching. (upbeat music)

(upbeat music) >> Hello, everyone, and welcome to theCUBE's presentation of the "AWS Startup Showcase." This is season three, episode one. The focus of this episode is AI/ML: Top Startups Building Foundational Models, Infrastructure, and AI. It's great topics, super-relevant, and it's part of our ongoing coverage of startups in the AWS ecosystem. I'm your host, John Furrier, with theCUBE. Today, we're excited to be joined by Jay Marshall, VP of Business Development at Neural Magic. Jay, thanks for coming on theCUBE. >> Hey, John, thanks so much. Thanks for having us. >> We had a great CUBE conversation with you guys. This is very much about the company focuses. It's a feature presentation for the "Startup Showcase," and the machine learning at scale is the topic, but in general, it's more, (laughs) and we should call it "Machine Learning and AI: How to Get Started," because everybody is retooling their business. Companies that aren't retooling their business right now with AI first will be out of business, in my opinion. You're seeing massive shift. This is really truly the beginning of the next-gen machine learning AI trend. It's really seeing ChatGPT. Everyone sees that. That went mainstream. But this is just the beginning. This is scratching the surface of this next-generation AI with machine learning powering it, and with all the goodness of cloud, cloud scale, and how horizontally scalable it is. The resources are there. You got the Edge. Everything's perfect for AI 'cause data infrastructure's exploding in value. AI is just the applications. This is a super topic, so what do you guys see in this general area of opportunities right now in the headlines? And I'm sure you guys' phone must be ringing off the hook, metaphorically speaking, or emails and meetings and Zooms. What's going on over there at Neural Magic? >> No, absolutely, and you pretty much nailed most of it. I think that, you know, my background, we've seen for the last 20-plus years. Even just getting enterprise applications kind of built and delivered at scale, obviously, amazing things with AWS and the cloud to help accelerate that. And we just kind of figured out in the last five or so years how to do that productively and efficiently, kind of from an operations perspective. Got development and operations teams. We even came up with DevOps, right? But now, we kind of have this new kind of persona and new workload that developers have to talk to, and then it has to be deployed on those ITOps solutions. And so you pretty much nailed it. Folks are saying, "Well, how do I do this?" These big, generational models or foundational models, as we're calling them, they're great, but enterprises want to do that with their data, on their infrastructure, at scale, at the edge. So for us, yeah, we're helping enterprises accelerate that through optimizing models and then delivering them at scale in a more cost-effective fashion. >> Yeah, and I think one of the things, the benefits of OpenAI we saw, was not only is it open source, then you got also other models that are more proprietary, is that it shows the world that this is really happening, right? It's a whole nother level, and there's also new landscape kind of maps coming out. You got the generative AI, and you got the foundational models, large LLMs. Where do you guys fit into the landscape? Because you guys are in the middle of this. How do you talk to customers when they say, "I'm going down this road. I need help. I'm going to stand this up." This new AI infrastructure and applications, where do you guys fit in the landscape? >> Right, and really, the answer is both. I think today, when it comes to a lot of what for some folks would still be considered kind of cutting edge around computer vision and natural language processing, a lot of our optimization tools and our runtime are based around most of the common computer vision and natural language processing models. So your YOLOs, your BERTs, you know, your DistilBERTs and what have you, so we work to help optimize those, again, who've gotten great performance and great value for customers trying to get those into production. But when you get into the LLMs, and you mentioned some of the open source components there, our research teams have kind of been right in the trenches with those. So kind of the GPT open source equivalent being OPT, being able to actually take, you know, a multi-$100 billion parameter model and sparsify that or optimize that down, shaving away a ton of parameters, and being able to run it on smaller infrastructure. So I think the evolution here, you know, all this stuff came out in the last six months in terms of being turned loose into the wild, but we're staying in the trenches with folks so that we can help optimize those as well and not require, again, the heavy compute, the heavy cost, the heavy power consumption as those models evolve as well. So we're staying right in with everybody while they're being built, but trying to get folks into production today with things that help with business value today. >> Jay, I really appreciate you coming on theCUBE, and before we came on camera, you said you just were on a customer call. I know you got a lot of activity. What specific things are you helping enterprises solve? What kind of problems? Take us through the spectrum from the beginning, people jumping in the deep end of the pool, some people kind of coming in, starting out slow. What are the scale? Can you scope the kind of use cases and problems that are emerging that people are calling you for? >> Absolutely, so I think if I break it down to kind of, like, your startup, or I maybe call 'em AI native to kind of steal from cloud native years ago, that group, it's pretty much, you know, part and parcel for how that group already runs. So if you have a data science team and an ML engineering team, you're building models, you're training models, you're deploying models. You're seeing firsthand the expense of starting to try to do that at scale. So it's really just a pure operational efficiency play. They kind of speak natively to our tools, which we're doing in the open source. So it's really helping, again, with the optimization of the models they've built, and then, again, giving them an alternative to expensive proprietary hardware accelerators to have to run them. Now, on the enterprise side, it varies, right? You have some kind of AI native folks there that already have these teams, but you also have kind of, like, AI curious, right? Like, they want to do it, but they don't really know where to start, and so for there, we actually have an open source toolkit that can help you get into this optimization, and then again, that runtime, that inferencing runtime, purpose-built for CPUs. It allows you to not have to worry, again, about do I have a hardware accelerator available? How do I integrate that into my application stack? If I don't already know how to build this into my infrastructure, does my ITOps teams, do they know how to do this, and what does that runway look like? How do I cost for this? How do I plan for this? When it's just x86 compute, we've been doing that for a while, right? So it obviously still requires more, but at least it's a little bit more predictable. >> It's funny you mentioned AI native. You know, born in the cloud was a phrase that was out there. Now, you have startups that are born in AI companies. So I think you have this kind of cloud kind of vibe going on. You have lift and shift was a big discussion. Then you had cloud native, kind of in the cloud, kind of making it all work. Is there a existing set of things? People will throw on this hat, and then what's the difference between AI native and kind of providing it to existing stuff? 'Cause we're a lot of people take some of these tools and apply it to either existing stuff almost, and it's not really a lift and shift, but it's kind of like bolting on AI to something else, and then starting with AI first or native AI. >> Absolutely. It's a- >> How would you- >> It's a great question. I think that probably, where I'd probably pull back to kind of allow kind of retail-type scenarios where, you know, for five, seven, nine years or more even, a lot of these folks already have data science teams, you know? I mean, they've been doing this for quite some time. The difference is the introduction of these neural networks and deep learning, right? Those kinds of models are just a little bit of a paradigm shift. So, you know, I obviously was trying to be fun with the term AI native, but I think it's more folks that kind of came up in that neural network world, so it's a little bit more second nature, whereas I think for maybe some traditional data scientists starting to get into neural networks, you have the complexity there and the training overhead, and a lot of the aspects of getting a model finely tuned and hyperparameterization and all of these aspects of it. It just adds a layer of complexity that they're just not as used to dealing with. And so our goal is to help make that easy, and then of course, make it easier to run anywhere that you have just kind of standard infrastructure. >> Well, the other point I'd bring out, and I'd love to get your reaction to, is not only is that a neural network team, people who have been focused on that, but also, if you look at some of the DataOps lately, AIOps markets, a lot of data engineering, a lot of scale, folks who have been kind of, like, in that data tsunami cloud world are seeing, they kind of been in this, right? They're, like, been experiencing that. >> No doubt. I think it's funny the data lake concept, right? And you got data oceans now. Like, the metaphors just keep growing on us, but where it is valuable in terms of trying to shift the mindset, I've always kind of been a fan of some of the naming shift. I know with AWS, they always talk about purpose-built databases. And I always liked that because, you know, you don't have one database that can do everything. Even ones that say they can, like, you still have to do implementation detail differences. So sitting back and saying, "What is my use case, and then which database will I use it for?" I think it's kind of similar here. And when you're building those data teams, if you don't have folks that are doing data engineering, kind of that data harvesting, free processing, you got to do all that before a model's even going to care about it. So yeah, it's definitely a central piece of this as well, and again, whether or not you're going to be AI negative as you're making your way to kind of, you know, on that journey, you know, data's definitely a huge component of it. >> Yeah, you would have loved our Supercloud event we had. Talk about naming and, you know, around data meshes was talked about a lot. You're starting to see the control plane layers of data. I think that was the beginning of what I saw as that data infrastructure shift, to be horizontally scalable. So I have to ask you, with Neural Magic, when your customers and the people that are prospects for you guys, they're probably asking a lot of questions because I think the general thing that we see is, "How do I get started? Which GPU do I use?" I mean, there's a lot of things that are kind of, I won't say technical or targeted towards people who are living in that world, but, like, as the mainstream enterprises come in, they're going to need a playbook. What do you guys see, what do you guys offer your clients when they come in, and what do you recommend? >> Absolutely, and I think where we hook in specifically tends to be on the training side. So again, I've built a model. Now, I want to really optimize that model. And then on the runtime side when you want to deploy it, you know, we run that optimized model. And so that's where we're able to provide. We even have a labs offering in terms of being able to pair up our engineering teams with a customer's engineering teams, and we can actually help with most of that pipeline. So even if it is something where you have a dataset and you want some help in picking a model, you want some help training it, you want some help deploying that, we can actually help there as well. You know, there's also a great partner ecosystem out there, like a lot of folks even in the "Startup Showcase" here, that extend beyond into kind of your earlier comment around data engineering or downstream ITOps or the all-up MLOps umbrella. So we can absolutely engage with our labs, and then, of course, you know, again, partners, which are always kind of key to this. So you are spot on. I think what's happened with the kind of this, they talk about a hockey stick. This is almost like a flat wall now with the rate of innovation right now in this space. And so we do have a lot of folks wanting to go straight from curious to native. And so that's definitely where the partner ecosystem comes in so hard 'cause there just isn't anybody or any teams out there that, I literally do from, "Here's my blank database, and I want an API that does all the stuff," right? Like, that's a big chunk, but we can definitely help with the model to delivery piece. >> Well, you guys are obviously a featured company in this space. Talk about the expertise. A lot of companies are like, I won't say faking it till they make it. You can't really fake security. You can't really fake AI, right? So there's going to be a learning curve. They'll be a few startups who'll come out of the gate early. You guys are one of 'em. Talk about what you guys have as expertise as a company, why you're successful, and what problems do you solve for customers? >> No, appreciate that. Yeah, we actually, we love to tell the story of our founder, Nir Shavit. So he's a 20-year professor at MIT. Actually, he was doing a lot of work on kind of multicore processing before there were even physical multicores, and actually even did a stint in computational neurobiology in the 2010s, and the impetus for this whole technology, has a great talk on YouTube about it, where he talks about the fact that his work there, he kind of realized that the way neural networks encode and how they're executed by kind of ramming data layer by layer through these kind of HPC-style platforms, actually was not analogous to how the human brain actually works. So we're on one side, we're building neural networks, and we're trying to emulate neurons. We're not really executing them that way. So our team, which one of the co-founders, also an ex-MIT, that was kind of the birth of why can't we leverage this super-performance CPU platform, which has those really fat, fast caches attached to each core, and actually start to find a way to break that model down in a way that I can execute things in parallel, not having to do them sequentially? So it is a lot of amazing, like, talks and stuff that show kind of the magic, if you will, a part of the pun of Neural Magic, but that's kind of the foundational layer of all the engineering that we do here. And in terms of how we're able to bring it to reality for customers, I'll give one customer quote where it's a large retailer, and it's a people-counting application. So a very common application. And that customer's actually been able to show literally double the amount of cameras being run with the same amount of compute. So for a one-to-one perspective, two-to-one, business leaders usually like that math, right? So we're able to show pure cost savings, but even performance-wise, you know, we have some of the common models like your ResNets and your YOLOs, where we can actually even perform better than hardware-accelerated solutions. So we're trying to do, I need to just dumb it down to better, faster, cheaper, but from a commodity perspective, that's where we're accelerating. >> That's not a bad business model. Make things easier to use, faster, and reduce the steps it takes to do stuff. So, you know, that's always going to be a good market. Now, you guys have DeepSparse, which we've talked about on our CUBE conversation prior to this interview, delivers ML models through the software so the hardware allows for a decoupling, right? >> Yep. >> Which is going to drive probably a cost advantage. Also, it's also probably from a deployment standpoint it must be easier. Can you share the benefits? Is it a cost side? Is it more of a deployment? What are the benefits of the DeepSparse when you guys decouple the software from the hardware on the ML models? >> No you actually, you hit 'em both 'cause that really is primarily the value. Because ultimately, again, we're so early. And I came from this world in a prior life where I'm doing Java development, WebSphere, WebLogic, Tomcat open source, right? When we were trying to do innovation, we had innovation buckets, 'cause everybody wanted to be on the web and have their app and a browser, right? We got all the money we needed to build something and show, hey, look at the thing on the web, right? But when you had to get in production, that was the challenge. So to what you're speaking to here, in this situation, we're able to show we're just a Python package. So whether you just install it on the operating system itself, or we also have a containerized version you can drop on any container orchestration platform, so ECS or EKS on AWS. And so you get all the auto-scaling features. So when you think about that kind of a world where you have everything from real-time inferencing to kind of after hours batch processing inferencing, the fact that you can auto scale that hardware up and down and it's CPU based, so you're paying by the minute instead of maybe paying by the hour at a lower cost shelf, it does everything from pure cost to, again, I can have my standard IT team say, "Hey, here's the Kubernetes in the container," and it just runs on the infrastructure we're already managing. So yeah, operational, cost and again, and many times even performance. (audio warbles) CPUs if I want to. >> Yeah, so that's easier on the deployment too. And you don't have this kind of, you know, blank check kind of situation where you don't know what's on the backend on the cost side. >> Exactly. >> And you control the actual hardware and you can manage that supply chain. >> And keep in mind, exactly. Because the other thing that sometimes gets lost in the conversation, depending on where a customer is, some of these workloads, like, you know, you and I remember a world where even like the roundtrip to the cloud and back was a problem for folks, right? We're used to extremely low latency. And some of these workloads absolutely also adhere to that. But there's some workloads where the latency isn't as important. And we actually even provide the tuning. Now, if we're giving you five milliseconds of latency and you don't need that, you can tune that back. So less CPU, lower cost. Now, throughput and other things come into play. But that's the kind of configurability and flexibility we give for operations. >> All right, so why should I call you if I'm a customer or prospect Neural Magic, what problem do I have or when do I know I need you guys? When do I call you in and what does my environment look like? When do I know? What are some of the signals that would tell me that I need Neural Magic? >> No, absolutely. So I think in general, any neural network, you know, the process I mentioned before called sparcification, it's, you know, an optimization process that we specialize in. Any neural network, you know, can be sparcified. So I think if it's a deep-learning neural network type model. If you're trying to get AI into production, you have cost concerns even performance-wise. I certainly hate to be too generic and say, "Hey, we'll talk to everybody." But really in this world right now, if it's a neural network, it's something where you're trying to get into production, you know, we are definitely offering, you know, kind of an at-scale performant deployable solution for deep learning models. >> So neural network you would define as what? Just devices that are connected that need to know about each other? What's the state-of-the-art current definition of neural network for customers that may think they have a neural network or might not know they have a neural network architecture? What is that definition for neural network? >> That's a great question. So basically, machine learning models that fall under this kind of category, you hear about transformers a lot, or I mentioned about YOLO, the YOLO family of computer vision models, or natural language processing models like BERT. If you have a data science team or even developers, some even regular, I used to call myself a nine to five developer 'cause I worked in the enterprise, right? So like, hey, we found a new open source framework, you know, I used to use Spring back in the day and I had to go figure it out. There's developers that are pulling these models down and they're figuring out how to get 'em into production, okay? So I think all of those kinds of situations, you know, if it's a machine learning model of the deep learning variety that's, you know, really specifically where we shine. >> Okay, so let me pretend I'm a customer for a minute. I have all these videos, like all these transcripts, I have all these people that we've interviewed, CUBE alumnis, and I say to my team, "Let's AI-ify, sparcify theCUBE." >> Yep. >> What do I do? I mean, do I just like, my developers got to get involved and they're going to be like, "Well, how do I upload it to the cloud? Do I use a GPU?" So there's a thought process. And I think a lot of companies are going through that example of let's get on this AI, how can it help our business? >> Absolutely. >> What does that progression look like? Take me through that example. I mean, I made up theCUBE example up, but we do have a lot of data. We have large data models and we have people and connect to the internet and so we kind of seem like there's a neural network. I think every company might have a neural network in place. >> Well, and I was going to say, I think in general, you all probably do represent even the standard enterprise more than most. 'Cause even the enterprise is going to have a ton of video content, a ton of text content. So I think it's a great example. So I think that that kind of sea or I'll even go ahead and use that term data lake again, of data that you have, you're probably going to want to be setting up kind of machine learning pipelines that are going to be doing all of the pre-processing from kind of the raw data to kind of prepare it into the format that say a YOLO would actually use or let's say BERT for natural language processing. So you have all these transcripts, right? So we would do a pre-processing path where we would create that into the file format that BERT, the machine learning model would know how to train off of. So that's kind of all the pre-processing steps. And then for training itself, we actually enable what's called sparse transfer learning. So that's transfer learning is a very popular method of doing training with existing models. So we would be able to retrain that BERT model with your transcript data that we have now done the pre-processing with to get it into the proper format. And now we have a BERT natural language processing model that's been trained on your data. And now we can deploy that onto DeepSparse runtime so that now you can ask that model whatever questions, or I should say pass, you're not going to ask it those kinds of questions ChatGPT, although we can do that too. But you're going to pass text through the BERT model and it's going to give you answers back. It could be things like sentiment analysis or text classification. You just call the model, and now when you pass text through it, you get the answers better, faster or cheaper. I'll use that reference again. >> Okay, we can create a CUBE bot to give us questions on the fly from the the AI bot, you know, from our previous guests. >> Well, and I will tell you using that as an example. So I had mentioned OPT before, kind of the open source version of ChatGPT. So, you know, typically that requires multiple GPUs to run. So our research team, I may have mentioned earlier, we've been able to sparcify that over 50% already and run it on only a single GPU. And so in that situation, you could train OPT with that corpus of data and do exactly what you say. Actually we could use Alexa, we could use Alexa to actually respond back with voice. How about that? We'll do an API call and we'll actually have an interactive Alexa-enabled bot. >> Okay, we're going to be a customer, let's put it on the list. But this is a great example of what you guys call software delivered AI, a topic we chatted about on theCUBE conversation. This really means this is a developer opportunity. This really is the convergence of the data growth, the restructuring, how data is going to be horizontally scalable, meets developers. So this is an AI developer model going on right now, which is kind of unique. >> It is, John, I will tell you what's interesting. And again, folks don't always think of it this way, you know, the AI magical goodness is now getting pushed in the middle where the developers and IT are operating. And so it again, that paradigm, although for some folks seem obvious, again, if you've been around for 20 years, that whole all that plumbing is a thing, right? And so what we basically help with is when you deploy the DeepSparse runtime, we have a very rich API footprint. And so the developers can call the API, ITOps can run it, or to your point, it's developer friendly enough that you could actually deploy our off-the-shelf models. We have something called the SparseZoo where we actually publish pre-optimized or pre-sparcified models. And so developers could literally grab those right off the shelf with the training they've already had and just put 'em right into their applications and deploy them as containers. So yeah, we enable that for sure as well. >> It's interesting, DevOps was infrastructure as code and we had a last season, a series on data as code, which we kind of coined. This is data as code. This is a whole nother level of opportunity where developers just want to have programmable data and apps with AI. This is a whole new- >> Absolutely. >> Well, absolutely great, great stuff. Our news team at SiliconANGLE and theCUBE said you guys had a little bit of a launch announcement you wanted to make here on the "AWS Startup Showcase." So Jay, you have something that you want to launch here? >> Yes, and thank you John for teeing me up. So I'm going to try to put this in like, you know, the vein of like an AWS, like main stage keynote launch, okay? So we're going to try this out. So, you know, a lot of our product has obviously been built on top of x86. I've been sharing that the past 15 minutes or so. And with that, you know, we're seeing a lot of acceleration for folks wanting to run on commodity infrastructure. But we've had customers and prospects and partners tell us that, you know, ARM and all of its kind of variance are very compelling, both cost performance-wise and also obviously with Edge. And wanted to know if there was anything we could do from a runtime perspective with ARM. And so we got the work and, you know, it's a hard problem to solve 'cause the instructions set for ARM is very different than the instruction set for x86, and our deep tensor column technology has to be able to work with that lower level instruction spec. But working really hard, the engineering team's been at it and we are happy to announce here at the "AWS Startup Showcase," that DeepSparse inference now has, or inference runtime now has support for AWS Graviton instances. So it's no longer just x86, it is also ARM and that obviously also opens up the door to Edge and further out the stack so that optimize once run anywhere, we're not going to open up. So it is an early access. So if you go to neuralmagic.com/graviton, you can sign up for early access, but we're excited to now get into the ARM side of the fence as well on top of Graviton. >> That's awesome. Our news team is going to jump on that news. We'll get it right up. We get a little scoop here on the "Startup Showcase." Jay Marshall, great job. That really highlights the flexibility that you guys have when you decouple the software from the hardware. And again, we're seeing open source driving a lot more in AI ops now with with machine learning and AI. So to me, that makes a lot of sense. And congratulations on that announcement. Final minute or so we have left, give a summary of what you guys are all about. Put a plug in for the company, what you guys are looking to do. I'm sure you're probably hiring like crazy. Take the last few minutes to give a plug for the company and give a summary. >> No, I appreciate that so much. So yeah, joining us out neuralmagic.com, you know, part of what we didn't spend a lot of time here, our optimization tools, we are doing all of that in the open source. It's called SparseML and I mentioned SparseZoo briefly. So we really want the data scientists community and ML engineering community to join us out there. And again, the DeepSparse runtime, it's actually free to use for trial purposes and for personal use. So you can actually run all this on your own laptop or on an AWS instance of your choice. We are now live in the AWS marketplace. So push button, deploy, come try us out and reach out to us on neuralmagic.com. And again, sign up for the Graviton early access. >> All right, Jay Marshall, Vice President of Business Development Neural Magic here, talking about performant, cost effective machine learning at scale. This is season three, episode one, focusing on foundational models as far as building data infrastructure and AI, AI native. I'm John Furrier with theCUBE. Thanks for watching. (bright upbeat music)

(soft music) >> Hello, everyone. Welcome to theCUBE's presentation of the AWS Startup Showcase. AI and Machine Learning: Top Startups Building Foundational Model Infrastructure. This is season 3, episode 1 of the ongoing series covering the exciting stuff from the AWS ecosystem, talking about machine learning and AI. I'm your host, John Furrier and today we are excited to be joined by Luis Ceze who's the CEO of OctoML and Anna Connolly, VP of customer success and experience OctoML. Great to have you on again, Luis. Anna, thanks for coming on. Appreciate it. >> Thank you, John. It's great to be here. >> Thanks for having us. >> I love the company. We had a CUBE conversation about this. You guys are really addressing how to run foundational models faster for less. And this is like the key theme. But before we get into it, this is a hot trend, but let's explain what you guys do. Can you set the narrative of what the company's about, why it was founded, what's your North Star and your mission? >> Yeah, so John, our mission is to make AI sustainable and accessible for everyone. And what we offer customers is, you know, a way of taking their models into production in the most efficient way possible by automating the process of getting a model and optimizing it for a variety of hardware and making cost-effective. So better, faster, cheaper model deployment. >> You know, the big trend here is AI. Everyone's seeing the ChatGPT, kind of the shot heard around the world. The BingAI and this fiasco and the ongoing experimentation. People are into it, and I think the business impact is clear. I haven't seen this in all of my career in the technology industry of this kind of inflection point. And every senior leader I talk to is rethinking about how to rebuild their business with AI because now the large language models have come in, these foundational models are here, they can see value in their data. This is a 10 year journey in the big data world. Now it's impacting that, and everyone's rebuilding their company around this idea of being AI first 'cause they see ways to eliminate things and make things more efficient. And so now they telling 'em to go do it. And they're like, what do we do? So what do you guys think? Can you explain what is this wave of AI and why is it happening, why now, and what should people pay attention to? What does it mean to them? >> Yeah, I mean, it's pretty clear by now that AI can do amazing things that captures people's imaginations. And also now can show things that are really impactful in businesses, right? So what people have the opportunity to do today is to either train their own model that adds value to their business or find open models out there that can do very valuable things to them. So the next step really is how do you take that model and put it into production in a cost-effective way so that the business can actually get value out of it, right? >> Anna, what's your take? Because customers are there, you're there to make 'em successful, you got the new secret weapon for their business. >> Yeah, I think we just see a lot of companies struggle to get from a trained model into a model that is deployed in a cost-effective way that actually makes sense for the application they're building. I think that's a huge challenge we see today, kind of across the board across all of our customers. >> Well, I see this, everyone asking the same question. I have data, I want to get value out of it. I got to get these big models, I got to train it. What's it going to cost? So I think there's a reality of, okay, I got to do it. Then no one has any visibility on what it costs. When they get into it, this is going to break the bank. So I have to ask you guys, the cost of training these models is on everyone's mind. OctoML, your company's focus on the cost side of it as well as the efficiency side of running these models in production. Why are the production costs such a concern and where specifically are people looking at it and why did it get here? >> Yeah, so training costs get a lot of attention because normally a large number, but we shouldn't forget that it's a large, typically one time upfront cost that customers pay. But, you know, when the model is put into production, the cost grows directly with model usage and you actually want your model to be used because it's adding value, right? So, you know, the question that a customer faces is, you know, they have a model, they have a trained model and now what? So how much would it cost to run in production, right? And now without the big wave in generative AI, which rightfully is getting a lot of attention because of the amazing things that it can do. It's important for us to keep in mind that generative AI models like ChatGPT are huge, expensive energy hogs. They cost a lot to run, right? And given that model usage growth directly, model cost grows directly with usage, what you want to do is make sure that once you put a model into production, you have the best cost structure possible so that you're not surprised when it's gets popular, right? So let me give you an example. So if you have a model that costs, say 1 to $2 million to train, but then it costs about one to two cents per session to use it, right? So if you have a million active users, even if they use just once a day, it's 10 to $20,000 a day to operate that model in production. And that very, very quickly, you know, get beyond what you paid to train it. >> Anna, these aren't small numbers, and it's cost to train and cost to operate, it kind of reminds me of when the cloud came around and the data center versus cloud options. Like, wait a minute, one, it costs a ton of cash to deploy, and then running it. This is kind of a similar dynamic. What are you seeing? >> Yeah, absolutely. I think we are going to see increasingly the cost and production outpacing the costs and training by a lot. I mean, people talk about training costs now because that's what they're confronting now because people are so focused on getting models performant enough to even use in an application. And now that we have them and they're that capable, we're really going to start to see production costs go up a lot. >> Yeah, Luis, if you don't mind, I know this might be a little bit of a tangent, but, you know, training's super important. I get that. That's what people are doing now, but then there's the deployment side of production. Where do people get caught up and miss the boat or misconfigure? What's the gotcha? Where's the trip wire or so to speak? Where do people mess up on the cost side? What do they do? Is it they don't think about it, they tie it to proprietary hardware? What's the issue? >> Yeah, several things, right? So without getting really technical, which, you know, I might get into, you know, you have to understand relationship between performance, you know, both in terms of latency and throughput and cost, right? So reducing latency is important because you improve responsiveness of the model. But it's really important to keep in mind that it often leads diminishing returns. Below a certain latency, making it faster won't make a measurable difference in experience, but it's going to cost a lot more. So understanding that is important. Now, if you care more about throughputs, which is the time it takes for you to, you know, units per period of time, you care about time to solution, we should think about this throughput per dollar. And understand what you want is the highest throughput per dollar, which may come at the cost of higher latency, which you're not going to care about, right? So, and the reality here, John, is that, you know, humans and especially folks in this space want to have the latest and greatest hardware. And often they commit a lot of money to get access to them and have to commit upfront before they understand the needs that their models have, right? So common mistake here, one is not spending time to understand what you really need, and then two, over-committing and using more hardware than you actually need. And not giving yourself enough freedom to get your workload to move around to the more cost-effective choice, right? So this is just a metaphoric choice. And then another thing that's important here too is making a model run faster on the hardware directly translates to lower cost, right? So, but it takes a lot of engineers, you need to think of ways of producing very efficient versions of your model for the target hardware that you're going to use. >> Anna, what's the customer angle here? Because price performance has been around for a long time, people get that, but now latency and throughput, that's key because we're starting to see this in apps. I mean, there's an end user piece. I even seeing it on the infrastructure side where they're taking a heavy lifting away from operational costs. So you got, you know, application specific to the user and/or top of the stack, and then you got actually being used in operations where they want both. >> Yeah, absolutely. Maybe I can illustrate this with a quick story with the customer that we had recently been working with. So this customer is planning to run kind of a transformer based model for tech generation at super high scale on Nvidia T4 GPU, so kind of a commodity GPU. And the scale was so high that they would've been paying hundreds of thousands of dollars in cloud costs per year just to serve this model alone. You know, one of many models in their application stack. So we worked with this team to optimize our model and then benchmark across several possible targets. So that matching the hardware that Luis was just talking about, including the newer kind of Nvidia A10 GPUs. And what they found during this process was pretty interesting. First, the team was able to shave a quarter of their spend just by using better optimization techniques on the T4, the older hardware. But actually moving to a newer GPU would allow them to serve this model in a sub two milliseconds latency, so super fast, which was able to unlock an entirely new kind of user experience. So they were able to kind of change the value they're delivering in their application just because they were able to move to this new hardware easily. So they ultimately decided to plan their deployment on the more expensive A10 because of this, but because of the hardware specific optimizations that we helped them with, they managed to even, you know, bring costs down from what they had originally planned. And so if you extend this kind of example to everything that's happening with generative AI, I think the story we just talked about was super relevant, but the scale can be even higher, you know, it can be tenfold that. We were recently conducting kind of this internal study using GPT-J as a proxy to illustrate the experience of just a company trying to use one of these large language models with an example scenario of creating a chatbot to help job seekers prepare for interviews. So if you imagine kind of a conservative usage scenario where the model generates just 3000 words per user per day, which is, you know, pretty conservative for how people are interacting with these models. It costs 5 cents a session and if you're a company and your app goes viral, so from, you know, beginning of the year there's nobody, at the end of the year there's a million daily active active users in that year alone, going from zero to a million. You'll be spending about $6 million a year, which is pretty unmanageable. That's crazy, right? >> Yeah. >> For a company or a product that's just launching. So I think, you know, for us we see the real way to make these kind of advancements accessible and sustainable, as we said is to bring down cost to serve using these techniques. >> That's a great story and I think that illustrates this idea that deployment cost can vary from situation to situation, from model to model and that the efficiency is so strong with this new wave, it eliminates heavy lifting, creates more efficiency, automates intellect. I mean, this is the trend, this is radical, this is going to increase. So the cost could go from nominal to millions, literally, potentially. So, this is what customers are doing. Yeah, that's a great story. What makes sense on a financial, is there a cost of ownership? Is there a pattern for best practice for training? What do you guys advise cuz this is a lot of time and money involved in all potential, you know, good scenarios of upside. But you can get over your skis as they say, and be successful and be out of business if you don't manage it. I mean, that's what people are talking about, right? >> Yeah, absolutely. I think, you know, we see kind of three main vectors to reduce cost. I think one is make your deployment process easier overall, so that your engineering effort to even get your app running goes down. Two, would be get more from the compute you're already paying for, you're already paying, you know, for your instances in the cloud, but can you do more with that? And then three would be shop around for lower cost hardware to match your use case. So on the first one, I think making the deployment easier overall, there's a lot of manual work that goes into benchmarking, optimizing and packaging models for deployment. And because the performance of machine learning models can be really hardware dependent, you have to go through this process for each target you want to consider running your model on. And this is hard, you know, we see that every day. But for teams who want to incorporate some of these large language models into their applications, it might be desirable because licensing a model from a large vendor like OpenAI can leave you, you know, over provision, kind of paying for capabilities you don't need in your application or can lock you into them and you lose flexibility. So we have a customer whose team actually prepares models for deployment in a SaaS application that many of us use every day. And they told us recently that without kind of an automated benchmarking and experimentation platform, they were spending several days each to benchmark a single model on a single hardware type. So this is really, you know, manually intensive and then getting more from the compute you're already paying for. We do see customers who leave money on the table by running models that haven't been optimized specifically for the hardware target they're using, like Luis was mentioning. And for some teams they just don't have the time to go through an optimization process and for others they might lack kind of specialized expertise and this is something we can bring. And then on shopping around for different hardware types, we really see a huge variation in model performance across hardware, not just CPU vs. GPU, which is, you know, what people normally think of. But across CPU vendors themselves, high memory instances and across cloud providers even. So the best strategy here is for teams to really be able to, we say, look before you leap by running real world benchmarking and not just simulations or predictions to find the best software, hardware combination for their workload. >> Yeah. You guys sound like you have a very impressive customer base deploying large language models. Where would you categorize your current customer base? And as you look out, as you guys are growing, you have new customers coming in, take me through the progression. Take me through the profile of some of your customers you have now, size, are they hyperscalers, are they big app folks, are they kicking the tires? And then as people are out there scratching heads, I got to get in this game, what's their psychology like? Are they coming in with specific problems or do they have specific orientation point of view about what they want to do? Can you share some data around what you're seeing? >> Yeah, I think, you know, we have customers that kind of range across the spectrum of sophistication from teams that basically don't have MLOps expertise in their company at all. And so they're really looking for us to kind of give a full service, how should I do everything from, you know, optimization, find the hardware, prepare for deployment. And then we have teams that, you know, maybe already have their serving and hosting infrastructure up and ready and they already have models in production and they're really just looking to, you know, take the extra juice out of the hardware and just do really specific on that optimization piece. I think one place where we're doing a lot more work now is kind of in the developer tooling, you know, model selection space. And that's kind of an area that we're creating more tools for, particularly within the PyTorch ecosystem to bring kind of this power earlier in the development cycle so that as people are grabbing a model off the shelf, they can, you know, see how it might perform and use that to inform their development process. >> Luis, what's the big, I like this idea of picking the models because isn't that like going to the market and picking the best model for your data? It's like, you know, it's like, isn't there a certain approaches? What's your view on this? 'Cause this is where everyone, I think it's going to be a land rush for this and I want to get your thoughts. >> For sure, yeah. So, you know, I guess I'll start with saying the one main takeaway that we got from the GPT-J study is that, you know, having a different understanding of what your model's compute and memory requirements are, very quickly, early on helps with the much smarter AI model deployments, right? So, and in fact, you know, Anna just touched on this, but I want to, you know, make sure that it's clear that OctoML is putting that power into user's hands right now. So in partnership with AWS, we are launching this new PyTorch native profiler that allows you with a single, you know, one line, you know, code decorator allows you to see how your code runs on a variety of different hardware after accelerations. So it gives you very clear, you know, data on how you should think about your model deployments. And this ties back to choices of models. So like, if you have a set of choices that are equally good of models in terms of functionality and you want to understand after acceleration how are you going to deploy, how much they're going to cost or what are the options using a automated process of making a decision is really, really useful. And in fact, so I think these events can get early access to this by signing up for the Octopods, you know, this is exclusive group for insiders here, so you can go to OctoML.ai/pods to sign up. >> So that Octopod, is that a program? What is that, is that access to code? Is that a beta, what is that? Explain, take a minute and explain Octopod. >> I think the Octopod would be a group of people who is interested in experiencing this functionality. So it is the friends and users of OctoML that would be the Octopod. And then yes, after you sign up, we would provide you essentially the tool in code form for you to try out in your own. I mean, part of the benefit of this is that it happens in your own local environment and you're in control of everything kind of within the workflow that developers are already using to create and begin putting these models into their applications. So it would all be within your control. >> Got it. I think the big question I have for you is when do you, when does that one of your customers know they need to call you? What's their environment look like? What are they struggling with? What are the conversations they might be having on their side of the fence? If anyone's watching this, they're like, "Hey, you know what, I've got my team, we have a lot of data. Do we have our own language model or do I use someone else's?" There's a lot of this, I will say discovery going on around what to do, what path to take, what does that customer look like, if someone's listening, when do they know to call you guys, OctoML? >> Well, I mean the most obvious one is that you have a significant spend on AI/ML, come and talk to us, you know, putting AIML into production. So that's the clear one. In fact, just this morning I was talking to someone who is in life sciences space and is having, you know, 15 to $20 million a year cloud related to AI/ML deployment is a clear, it's a pretty clear match right there, right? So that's on the cost side. But I also want to emphasize something that Anna said earlier that, you know, the hardware and software complexity involved in putting model into production is really high. So we've been able to abstract that away, offering a clean automation flow enables one, to experiment early on, you know, how models would run and get them to production. And then two, once they are into production, gives you an automated flow to continuously updating your model and taking advantage of all this acceleration and ability to run the model on the right hardware. So anyways, let's say one then is cost, you know, you have significant cost and then two, you have an automation needs. And Anna please compliment that. >> Yeah, Anna you can please- >> Yeah, I think that's exactly right. Maybe the other time is when you are expecting a big scale up in serving your application, right? You're launching a new feature, you expect to get a lot of usage or, and you want to kind of anticipate maybe your CTO, your CIO, whoever pays your cloud bills is going to come after you, right? And so they want to know, you know, what's the return on putting this model essentially into my application stack? Am I going to, is the usage going to match what I'm paying for it? And then you can understand that. >> So you guys have a lot of the early adopters, they got big data teams, they're pushed in the production, they want to get a little QA, test the waters, understand, use your technology to figure it out. Is there any cases where people have gone into production, they have to pull it out? It's like the old lemon laws with your car, you buy a car and oh my god, it's not the way I wanted it. I mean, I can imagine the early people through the wall, so to speak, in the wave here are going to be bloody in the sense that they've gone in and tried stuff and get stuck with huge bills. Are you seeing that? Are people pulling stuff out of production and redeploying? Or I can imagine that if I had a bad deployment, I'd want to refactor that or actually replatform that. Do you see that too? >> Definitely after a sticker shock, yes, your customers will come and make sure that, you know, the sticker shock won't happen again. >> Yeah. >> But then there's another more thorough aspect here that I think we likely touched on, be worth elaborating a bit more is just how are you going to scale in a way that's feasible depending on the allocation that you get, right? So as we mentioned several times here, you know, model deployment is so hardware dependent and so complex that you tend to get a model for a hardware choice and then you want to scale that specific type of instance. But what if, when you want to scale because suddenly luckily got popular and, you know, you want to scale it up and then you don't have that instance anymore. So how do you live with whatever you have at that moment is something that we see customers needing as well. You know, so in fact, ideally what we want is customers to not think about what kind of specific instances they want. What they want is to know what their models need. Say, they know the SLA and then find a set of hybrid targets and instances that hit the SLA whenever they're also scaling, they're going to scale with more freedom, right? Instead of having to wait for AWS to give them more specific allocation for a specific instance. What if you could live with other types of hardware and scale up in a more free way, right? So that's another thing that we see customers, you know, like they need more freedom to be able to scale with whatever is available. >> Anna, you touched on this with the business model impact to that 6 million cost, if that goes out of control, there's a business model aspect and there's a technical operation aspect to the cost side too. You want to be mindful of riding the wave in a good way, but not getting over your skis. So that brings up the point around, you know, confidence, right? And teamwork. Because if you're in production, there's probably a team behind it. Talk about the team aspect of your customers. I mean, they're dedicated, they go put stuff into production, they're developers, there're data. What's in it for them? Are they getting better, are they in the beach, you know, reading the book. Are they, you know, are there easy street for them? What's the customer benefit to the teams? >> Yeah, absolutely. With just a few clicks of a button, you're in production, right? That's the dream. So yeah, I mean I think that, you know, we illustrated it before a little bit. I think the automated kind of benchmarking and optimization process, like when you think about the effort it takes to get that data by hand, which is what people are doing today, they just don't do it. So they're making decisions without the best information because it's, you know, there just isn't the bandwidth to get the information that they need to make the best decision and then know exactly how to deploy it. So I think it's actually bringing kind of a new insight and capability to these teams that they didn't have before. And then maybe another aspect on the team side is that it's making the hand-off of the models from the data science teams to the model deployment teams more seamless. So we have, you know, we have seen in the past that this kind of transition point is the place where there are a lot of hiccups, right? The data science team will give a model to the production team and it'll be too slow for the application or it'll be too expensive to run and it has to go back and be changed and kind of this loop. And so, you know, with the PyTorch profiler that Luis was talking about, and then also, you know, the other ways we do optimization that kind of prevents that hand-off problem from happening. >> Luis and Anna, you guys have a great company. Final couple minutes left. Talk about the company, the people there, what's the culture like, you know, if Intel has Moore's law, which is, you know, doubling the performance in few years, what's the culture like there? Is it, you know, more throughput, better pricing? Explain what's going on with the company and put a plug in. Luis, we'll start with you. >> Yeah, absolutely. I'm extremely proud of the team that we built here. You know, we have a people first culture, you know, very, very collaborative and folks, we all have a shared mission here of making AI more accessible and sustainable. We have a very diverse team in terms of backgrounds and life stories, you know, to do what we do here, we need a team that has expertise in software engineering, in machine learning, in computer architecture. Even though we don't build chips, we need to understand how they work, right? So, and then, you know, the fact that we have this, this very really, really varied set of backgrounds makes the environment, you know, it's say very exciting to learn more about, you know, assistance end-to-end. But also makes it for a very interesting, you know, work environment, right? So people have different backgrounds, different stories. Some of them went to grad school, others, you know, were in intelligence agencies and now are working here, you know. So we have a really interesting set of people and, you know, life is too short not to work with interesting humans. You know, that's something that I like to think about, you know. >> I'm sure your off-site meetings are a lot of fun, people talking about computer architectures, silicon advances, the next GPU, the big data models coming in. Anna, what's your take? What's the culture like? What's the company vibe and what are you guys looking to do? What's the customer success pattern? What's up? >> Yeah, absolutely. I mean, I, you know, second all of the great things that Luis just said about the team. I think one that I, an additional one that I'd really like to underscore is kind of this customer obsession, to use a term you all know well. And focus on the end users and really making the experiences that we're bringing to our user who are developers really, you know, useful and valuable for them. And so I think, you know, all of these tools that we're trying to put in the hands of users, the industry and the market is changing so rapidly that our products across the board, you know, all of the companies that, you know, are part of the showcase today, we're all evolving them so quickly and we can only do that kind of really hand in glove with our users. So that would be another thing I'd emphasize. >> I think the change dynamic, the power dynamics of this industry is just the beginning. I'm very bullish that this is going to be probably one of the biggest inflection points in history of the computer industry because of all the dynamics of the confluence of all the forces, which you mentioned some of them, I mean PC, you know, interoperability within internetworking and you got, you know, the web and then mobile. Now we have this, I mean, I wouldn't even put social media even in the close to this. Like, this is like, changes user experience, changes infrastructure. There's going to be massive accelerations in performance on the hardware side from AWS's of the world and cloud and you got the edge and more data. This is really what big data was going to look like. This is the beginning. Final question, what do you guys see going forward in the future? >> Well, it's undeniable that machine learning and AI models are becoming an integral part of an interesting application today, right? So, and the clear trends here are, you know, more and more competitional needs for these models because they're only getting more and more powerful. And then two, you know, seeing the complexity of the infrastructure where they run, you know, just considering the cloud, there's like a wide variety of choices there, right? So being able to live with that and making the most out of it in a way that does not require, you know, an impossible to find team is something that's pretty clear. So the need for automation, abstracting with the complexity is definitely here. And we are seeing this, you know, trends are that you also see models starting to move to the edge as well. So it's clear that we're seeing, we are going to live in a world where there's no large models living in the cloud. And then, you know, edge models that talk to these models in the cloud to form, you know, an end-to-end truly intelligent application. >> Anna? >> Yeah, I think, you know, our, Luis said it at the beginning. Our vision is to make AI sustainable and accessible. And I think as this technology just expands in every company and every team, that's going to happen kind of on its own. And we're here to help support that. And I think you can't do that without tools like those like OctoML. >> I think it's going to be an error of massive invention, creativity, a lot of the format heavy lifting is going to allow the talented people to automate their intellect. I mean, this is really kind of what we see going on. And Luis, thank you so much. Anna, thanks for coming on this segment. Thanks for coming on theCUBE and being part of the AWS Startup Showcase. I'm John Furrier, your host. Thanks for watching. (upbeat music)

(upbeat music) >> Hello everyone. Welcome to theCube's presentation of the "AWS Startup Showcase." The topic this episode is AI and machine learning, top startups building foundational model infrastructure. This is season three, episode one of the ongoing series covering exciting startups from the AWS ecosystem. And this time we're talking about AI and machine learning. I'm your host, John Furrier. I'm excited I'm joined today by Robert Nishihara, who's the co-founder and CEO of a hot startup called Anyscale. He's here to talk about Ray, the open source project, Anyscale's infrastructure for foundation as well. Robert, thank you for joining us today. >> Yeah, thanks so much as well. >> I've been following your company since the founding pre pandemic and you guys really had a great vision scaled up and in a perfect position for this big wave that we all see with ChatGPT and OpenAI that's gone mainstream. Finally, AI has broken out through the ropes and now gone mainstream, so I think you guys are really well positioned. I'm looking forward to to talking with you today. But before we get into it, introduce the core mission for Anyscale. Why do you guys exist? What is the North Star for Anyscale? >> Yeah, like you mentioned, there's a tremendous amount of excitement about AI right now. You know, I think a lot of us believe that AI can transform just every different industry. So one of the things that was clear to us when we started this company was that the amount of compute needed to do AI was just exploding. Like to actually succeed with AI, companies like OpenAI or Google or you know, these companies getting a lot of value from AI, were not just running these machine learning models on their laptops or on a single machine. They were scaling these applications across hundreds or thousands or more machines and GPUs and other resources in the Cloud. And so to actually succeed with AI, and this has been one of the biggest trends in computing, maybe the biggest trend in computing in, you know, in recent history, the amount of compute has been exploding. And so to actually succeed with that AI, to actually build these scalable applications and scale the AI applications, there's a tremendous software engineering lift to build the infrastructure to actually run these scalable applications. And that's very hard to do. So one of the reasons many AI projects and initiatives fail is that, or don't make it to production, is the need for this scale, the infrastructure lift, to actually make it happen. So our goal here with Anyscale and Ray, is to make that easy, is to make scalable computing easy. So that as a developer or as a business, if you want to do AI, if you want to get value out of AI, all you need to know is how to program on your laptop. Like, all you need to know is how to program in Python. And if you can do that, then you're good to go. Then you can do what companies like OpenAI or Google do and get value out of machine learning. >> That programming example of how easy it is with Python reminds me of the early days of Cloud, when infrastructure as code was talked about was, it was just code the infrastructure programmable. That's super important. That's what AI people wanted, first program AI. That's the new trend. And I want to understand, if you don't mind explaining, the relationship that Anyscale has to these foundational models and particular the large language models, also called LLMs, was seen with like OpenAI and ChatGPT. Before you get into the relationship that you have with them, can you explain why the hype around foundational models? Why are people going crazy over foundational models? What is it and why is it so important? >> Yeah, so foundational models and foundation models are incredibly important because they enable businesses and developers to get value out of machine learning, to use machine learning off the shelf with these large models that have been trained on tons of data and that are useful out of the box. And then, of course, you know, as a business or as a developer, you can take those foundational models and repurpose them or fine tune them or adapt them to your specific use case and what you want to achieve. But it's much easier to do that than to train them from scratch. And I think there are three, for people to actually use foundation models, there are three main types of workloads or problems that need to be solved. One is training these foundation models in the first place, like actually creating them. The second is fine tuning them and adapting them to your use case. And the third is serving them and actually deploying them. Okay, so Ray and Anyscale are used for all of these three different workloads. Companies like OpenAI or Cohere that train large language models. Or open source versions like GPTJ are done on top of Ray. There are many startups and other businesses that fine tune, that, you know, don't want to train the large underlying foundation models, but that do want to fine tune them, do want to adapt them to their purposes, and build products around them and serve them, those are also using Ray and Anyscale for that fine tuning and that serving. And so the reason that Ray and Anyscale are important here is that, you know, building and using foundation models requires a huge scale. It requires a lot of data. It requires a lot of compute, GPUs, TPUs, other resources. And to actually take advantage of that and actually build these scalable applications, there's a lot of infrastructure that needs to happen under the hood. And so you can either use Ray and Anyscale to take care of that and manage the infrastructure and solve those infrastructure problems. Or you can build the infrastructure and manage the infrastructure yourself, which you can do, but it's going to slow your team down. It's going to, you know, many of the businesses we work with simply don't want to be in the business of managing infrastructure and building infrastructure. They want to focus on product development and move faster. >> I know you got a keynote presentation we're going to go to in a second, but I think you hit on something I think is the real tipping point, doing it yourself, hard to do. These are things where opportunities are and the Cloud did that with data centers. Turned a data center and made it an API. The heavy lifting went away and went to the Cloud so people could be more creative and build their product. In this case, build their creativity. Is that kind of what's the big deal? Is that kind of a big deal happening that you guys are taking the learnings and making that available so people don't have to do that? >> That's exactly right. So today, if you want to succeed with AI, if you want to use AI in your business, infrastructure work is on the critical path for doing that. To do AI, you have to build infrastructure. You have to figure out how to scale your applications. That's going to change. We're going to get to the point, and you know, with Ray and Anyscale, we're going to remove the infrastructure from the critical path so that as a developer or as a business, all you need to focus on is your application logic, what you want the the program to do, what you want your application to do, how you want the AI to actually interface with the rest of your product. Now the way that will happen is that Ray and Anyscale will still, the infrastructure work will still happen. It'll just be under the hood and taken care of by Ray in Anyscale. And so I think something like this is really necessary for AI to reach its potential, for AI to have the impact and the reach that we think it will, you have to make it easier to do. >> And just for clarification to point out, if you don't mind explaining the relationship of Ray and Anyscale real quick just before we get into the presentation. >> So Ray is an open source project. We created it. We were at Berkeley doing machine learning. We started Ray so that, in order to provide an easy, a simple open source tool for building and running scalable applications. And Anyscale is the managed version of Ray, basically we will run Ray for you in the Cloud, provide a lot of tools around the developer experience and managing the infrastructure and providing more performance and superior infrastructure. >> Awesome. I know you got a presentation on Ray and Anyscale and you guys are positioning as the infrastructure for foundational models. So I'll let you take it away and then when you're done presenting, we'll come back, I'll probably grill you with a few questions and then we'll close it out so take it away. >> Robert: Sounds great. So I'll say a little bit about how companies are using Ray and Anyscale for foundation models. The first thing I want to mention is just why we're doing this in the first place. And the underlying observation, the underlying trend here, and this is a plot from OpenAI, is that the amount of compute needed to do machine learning has been exploding. It's been growing at something like 35 times every 18 months. This is absolutely enormous. And other people have written papers measuring this trend and you get different numbers. But the point is, no matter how you slice and dice it, it' a astronomical rate. Now if you compare that to something we're all familiar with, like Moore's Law, which says that, you know, the processor performance doubles every roughly 18 months, you can see that there's just a tremendous gap between the needs, the compute needs of machine learning applications, and what you can do with a single chip, right. So even if Moore's Law were continuing strong and you know, doing what it used to be doing, even if that were the case, there would still be a tremendous gap between what you can do with the chip and what you need in order to do machine learning. And so given this graph, what we've seen, and what has been clear to us since we started this company, is that doing AI requires scaling. There's no way around it. It's not a nice to have, it's really a requirement. And so that led us to start Ray, which is the open source project that we started to make it easy to build these scalable Python applications and scalable machine learning applications. And since we started the project, it's been adopted by a tremendous number of companies. Companies like OpenAI, which use Ray to train their large models like ChatGPT, companies like Uber, which run all of their deep learning and classical machine learning on top of Ray, companies like Shopify or Spotify or Instacart or Lyft or Netflix, ByteDance, which use Ray for their machine learning infrastructure. Companies like Ant Group, which makes Alipay, you know, they use Ray across the board for fraud detection, for online learning, for detecting money laundering, you know, for graph processing, stream processing. Companies like Amazon, you know, run Ray at a tremendous scale and just petabytes of data every single day. And so the project has seen just enormous adoption since, over the past few years. And one of the most exciting use cases is really providing the infrastructure for building training, fine tuning, and serving foundation models. So I'll say a little bit about, you know, here are some examples of companies using Ray for foundation models. Cohere trains large language models. OpenAI also trains large language models. You can think about the workloads required there are things like supervised pre-training, also reinforcement learning from human feedback. So this is not only the regular supervised learning, but actually more complex reinforcement learning workloads that take human input about what response to a particular question, you know is better than a certain other response. And incorporating that into the learning. There's open source versions as well, like GPTJ also built on top of Ray as well as projects like Alpa coming out of UC Berkeley. So these are some of the examples of exciting projects in organizations, training and creating these large language models and serving them using Ray. Okay, so what actually is Ray? Well, there are two layers to Ray. At the lowest level, there's the core Ray system. This is essentially low level primitives for building scalable Python applications. Things like taking a Python function or a Python class and executing them in the cluster setting. So Ray core is extremely flexible and you can build arbitrary scalable applications on top of Ray. So on top of Ray, on top of the core system, what really gives Ray a lot of its power is this ecosystem of scalable libraries. So on top of the core system you have libraries, scalable libraries for ingesting and pre-processing data, for training your models, for fine tuning those models, for hyper parameter tuning, for doing batch processing and batch inference, for doing model serving and deployment, right. And a lot of the Ray users, the reason they like Ray is that they want to run multiple workloads. They want to train and serve their models, right. They want to load their data and feed that into training. And Ray provides common infrastructure for all of these different workloads. So this is a little overview of what Ray, the different components of Ray. So why do people choose to go with Ray? I think there are three main reasons. The first is the unified nature. The fact that it is common infrastructure for scaling arbitrary workloads, from data ingest to pre-processing to training to inference and serving, right. This also includes the fact that it's future proof. AI is incredibly fast moving. And so many people, many companies that have built their own machine learning infrastructure and standardized on particular workflows for doing machine learning have found that their workflows are too rigid to enable new capabilities. If they want to do reinforcement learning, if they want to use graph neural networks, they don't have a way of doing that with their standard tooling. And so Ray, being future proof and being flexible and general gives them that ability. Another reason people choose Ray in Anyscale is the scalability. This is really our bread and butter. This is the reason, the whole point of Ray, you know, making it easy to go from your laptop to running on thousands of GPUs, making it easy to scale your development workloads and run them in production, making it easy to scale, you know, training to scale data ingest, pre-processing and so on. So scalability and performance, you know, are critical for doing machine learning and that is something that Ray provides out of the box. And lastly, Ray is an open ecosystem. You can run it anywhere. You can run it on any Cloud provider. Google, you know, Google Cloud, AWS, Asure. You can run it on your Kubernetes cluster. You can run it on your laptop. It's extremely portable. And not only that, it's framework agnostic. You can use Ray to scale arbitrary Python workloads. You can use it to scale and it integrates with libraries like TensorFlow or PyTorch or JAX or XG Boost or Hugging Face or PyTorch Lightning, right, or Scikit-learn or just your own arbitrary Python code. It's open source. And in addition to integrating with the rest of the machine learning ecosystem and these machine learning frameworks, you can use Ray along with all of the other tooling in the machine learning ecosystem. That's things like weights and biases or ML flow, right. Or you know, different data platforms like Databricks, you know, Delta Lake or Snowflake or tools for model monitoring for feature stores, all of these integrate with Ray. And that's, you know, Ray provides that kind of flexibility so that you can integrate it into the rest of your workflow. And then Anyscale is the scalable compute platform that's built on top, you know, that provides Ray. So Anyscale is a managed Ray service that runs in the Cloud. And what Anyscale does is it offers the best way to run Ray. And if you think about what you get with Anyscale, there are fundamentally two things. One is about moving faster, accelerating the time to market. And you get that by having the managed service so that as a developer you don't have to worry about managing infrastructure, you don't have to worry about configuring infrastructure. You also, it provides, you know, optimized developer workflows. Things like easily moving from development to production, things like having the observability tooling, the debug ability to actually easily diagnose what's going wrong in a distributed application. So things like the dashboards and the other other kinds of tooling for collaboration, for monitoring and so on. And then on top of that, so that's the first bucket, developer productivity, moving faster, faster experimentation and iteration. The second reason that people choose Anyscale is superior infrastructure. So this is things like, you know, cost deficiency, being able to easily take advantage of spot instances, being able to get higher GPU utilization, things like faster cluster startup times and auto scaling. Things like just overall better performance and faster scheduling. And so these are the kinds of things that Anyscale provides on top of Ray. It's the managed infrastructure. It's fast, it's like the developer productivity and velocity as well as performance. So this is what I wanted to share about Ray in Anyscale. >> John: Awesome. >> Provide that context. But John, I'm curious what you think. >> I love it. I love the, so first of all, it's a platform because that's the platform architecture right there. So just to clarify, this is an Anyscale platform, not- >> That's right. >> Tools. So you got tools in the platform. Okay, that's key. Love that managed service. Just curious, you mentioned Python multiple times, is that because of PyTorch and TensorFlow or Python's the most friendly with machine learning or it's because it's very common amongst all developers? >> That's a great question. Python is the language that people are using to do machine learning. So it's the natural starting point. Now, of course, Ray is actually designed in a language agnostic way and there are companies out there that use Ray to build scalable Java applications. But for the most part right now we're focused on Python and being the best way to build these scalable Python and machine learning applications. But, of course, down the road there always is that potential. >> So if you're slinging Python code out there and you're watching that, you're watching this video, get on Anyscale bus quickly. Also, I just, while you were giving the presentation, I couldn't help, since you mentioned OpenAI, which by the way, congratulations 'cause they've had great scale, I've noticed in their rapid growth 'cause they were the fastest company to the number of users than anyone in the history of the computer industry, so major successor, OpenAI and ChatGPT, huge fan. I'm not a skeptic at all. I think it's just the beginning, so congratulations. But I actually typed into ChatGPT, what are the top three benefits of Anyscale and came up with scalability, flexibility, and ease of use. Obviously, scalability is what you guys are called. >> That's pretty good. >> So that's what they came up with. So they nailed it. Did you have an inside prompt training, buy it there? Only kidding. (Robert laughs) >> Yeah, we hard coded that one. >> But that's the kind of thing that came up really, really quickly if I asked it to write a sales document, it probably will, but this is the future interface. This is why people are getting excited about the foundational models and the large language models because it's allowing the interface with the user, the consumer, to be more human, more natural. And this is clearly will be in every application in the future. >> Absolutely. This is how people are going to interface with software, how they're going to interface with products in the future. It's not just something, you know, not just a chat bot that you talk to. This is going to be how you get things done, right. How you use your web browser or how you use, you know, how you use Photoshop or how you use other products. Like you're not going to spend hours learning all the APIs and how to use them. You're going to talk to it and tell it what you want it to do. And of course, you know, if it doesn't understand it, it's going to ask clarifying questions. You're going to have a conversation and then it'll figure it out. >> This is going to be one of those things, we're going to look back at this time Robert and saying, "Yeah, from that company, that was the beginning of that wave." And just like AWS and Cloud Computing, the folks who got in early really were in position when say the pandemic came. So getting in early is a good thing and that's what everyone's talking about is getting in early and playing around, maybe replatforming or even picking one or few apps to refactor with some staff and managed services. So people are definitely jumping in. So I have to ask you the ROI cost question. You mentioned some of those, Moore's Law versus what's going on in the industry. When you look at that kind of scale, the first thing that jumps out at people is, "Okay, I love it. Let's go play around." But what's it going to cost me? Am I going to be tied to certain GPUs? What's the landscape look like from an operational standpoint, from the customer? Are they locked in and the benefit was flexibility, are you flexible to handle any Cloud? What is the customers, what are they looking at? Basically, that's my question. What's the customer looking at? >> Cost is super important here and many of the companies, I mean, companies are spending a huge amount on their Cloud computing, on AWS, and on doing AI, right. And I think a lot of the advantage of Anyscale, what we can provide here is not only better performance, but cost efficiency. Because if we can run something faster and more efficiently, it can also use less resources and you can lower your Cloud spending, right. We've seen companies go from, you know, 20% GPU utilization with their current setup and the current tools they're using to running on Anyscale and getting more like 95, you know, 100% GPU utilization. That's something like a five x improvement right there. So depending on the kind of application you're running, you know, it's a significant cost savings. We've seen companies that have, you know, processing petabytes of data every single day with Ray going from, you know, getting order of magnitude cost savings by switching from what they were previously doing to running their application on Ray. And when you have applications that are spending, you know, potentially $100 million a year and getting a 10 X cost savings is just absolutely enormous. So these are some of the kinds of- >> Data infrastructure is super important. Again, if the customer, if you're a prospect to this and thinking about going in here, just like the Cloud, you got infrastructure, you got the platform, you got SaaS, same kind of thing's going to go on in AI. So I want to get into that, you know, ROI discussion and some of the impact with your customers that are leveraging the platform. But first I hear you got a demo. >> Robert: Yeah, so let me show you, let me give you a quick run through here. So what I have open here is the Anyscale UI. I've started a little Anyscale Workspace. So Workspaces are the Anyscale concept for interactive developments, right. So here, imagine I'm just, you want to have a familiar experience like you're developing on your laptop. And here I have a terminal. It's not on my laptop. It's actually in the cloud running on Anyscale. And I'm just going to kick this off. This is going to train a large language model, so OPT. And it's doing this on 32 GPUs. We've got a cluster here with a bunch of CPU cores, bunch of memory. And as that's running, and by the way, if I wanted to run this on instead of 32 GPUs, 64, 128, this is just a one line change when I launch the Workspace. And what I can do is I can pull up VS code, right. Remember this is the interactive development experience. I can look at the actual code. Here it's using Ray train to train the torch model. We've got the training loop and we're saying that each worker gets access to one GPU and four CPU cores. And, of course, as I make the model larger, this is using deep speed, as I make the model larger, I could increase the number of GPUs that each worker gets access to, right. And how that is distributed across the cluster. And if I wanted to run on CPUs instead of GPUs or a different, you know, accelerator type, again, this is just a one line change. And here we're using Ray train to train the models, just taking my vanilla PyTorch model using Hugging Face and then scaling that across a bunch of GPUs. And, of course, if I want to look at the dashboard, I can go to the Ray dashboard. There are a bunch of different visualizations I can look at. I can look at the GPU utilization. I can look at, you know, the CPU utilization here where I think we're currently loading the model and running that actual application to start the training. And some of the things that are really convenient here about Anyscale, both I can get that interactive development experience with VS code. You know, I can look at the dashboards. I can monitor what's going on. It feels, I have a terminal, it feels like my laptop, but it's actually running on a large cluster. And I can, with however many GPUs or other resources that I want. And so it's really trying to combine the best of having the familiar experience of programming on your laptop, but with the benefits, you know, being able to take advantage of all the resources in the Cloud to scale. And it's like when, you know, you're talking about cost efficiency. One of the biggest reasons that people waste money, one of the silly reasons for wasting money is just forgetting to turn off your GPUs. And what you can do here is, of course, things will auto terminate if they're idle. But imagine you go to sleep, I have this big cluster. You can turn it off, shut off the cluster, come back tomorrow, restart the Workspace, and you know, your big cluster is back up and all of your code changes are still there. All of your local file edits. It's like you just closed your laptop and came back and opened it up again. And so this is the kind of experience we want to provide for our users. So that's what I wanted to share with you. >> Well, I think that whole, couple of things, lines of code change, single line of code change, that's game changing. And then the cost thing, I mean human error is a big deal. People pass out at their computer. They've been coding all night or they just forget about it. I mean, and then it's just like leaving the lights on or your water running in your house. It's just, at the scale that it is, the numbers will add up. That's a huge deal. So I think, you know, compute back in the old days, there's no compute. Okay, it's just compute sitting there idle. But you know, data cranking the models is doing, that's a big point. >> Another thing I want to add there about cost efficiency is that we make it really easy to use, if you're running on Anyscale, to use spot instances and these preemptable instances that can just be significantly cheaper than the on-demand instances. And so when we see our customers go from what they're doing before to using Anyscale and they go from not using these spot instances 'cause they don't have the infrastructure around it, the fault tolerance to handle the preemption and things like that, to being able to just check a box and use spot instances and save a bunch of money. >> You know, this was my whole, my feature article at Reinvent last year when I met with Adam Selipsky, this next gen Cloud is here. I mean, it's not auto scale, it's infrastructure scale. It's agility. It's flexibility. I think this is where the world needs to go. Almost what DevOps did for Cloud and what you were showing me that demo had this whole SRE vibe. And remember Google had site reliability engines to manage all those servers. This is kind of like an SRE vibe for data at scale. I mean, a similar kind of order of magnitude. I mean, I might be a little bit off base there, but how would you explain it? >> It's a nice analogy. I mean, what we are trying to do here is get to the point where developers don't think about infrastructure. Where developers only think about their application logic. And where businesses can do AI, can succeed with AI, and build these scalable applications, but they don't have to build, you know, an infrastructure team. They don't have to develop that expertise. They don't have to invest years in building their internal machine learning infrastructure. They can just focus on the Python code, on their application logic, and run the stuff out of the box. >> Awesome. Well, I appreciate the time. Before we wrap up here, give a plug for the company. I know you got a couple websites. Again, go, Ray's got its own website. You got Anyscale. You got an event coming up. Give a plug for the company looking to hire. Put a plug in for the company. >> Yeah, absolutely. Thank you. So first of all, you know, we think AI is really going to transform every industry and the opportunity is there, right. We can be the infrastructure that enables all of that to happen, that makes it easy for companies to succeed with AI, and get value out of AI. Now we have, if you're interested in learning more about Ray, Ray has been emerging as the standard way to build scalable applications. Our adoption has been exploding. I mentioned companies like OpenAI using Ray to train their models. But really across the board companies like Netflix and Cruise and Instacart and Lyft and Uber, you know, just among tech companies. It's across every industry. You know, gaming companies, agriculture, you know, farming, robotics, drug discovery, you know, FinTech, we see it across the board. And all of these companies can get value out of AI, can really use AI to improve their businesses. So if you're interested in learning more about Ray and Anyscale, we have our Ray Summit coming up in September. This is going to highlight a lot of the most impressive use cases and stories across the industry. And if your business, if you want to use LLMs, you want to train these LLMs, these large language models, you want to fine tune them with your data, you want to deploy them, serve them, and build applications and products around them, give us a call, talk to us. You know, we can really take the infrastructure piece, you know, off the critical path and make that easy for you. So that's what I would say. And, you know, like you mentioned, we're hiring across the board, you know, engineering, product, go-to-market, and it's an exciting time. >> Robert Nishihara, co-founder and CEO of Anyscale, congratulations on a great company you've built and continuing to iterate on and you got growth ahead of you, you got a tailwind. I mean, the AI wave is here. I think OpenAI and ChatGPT, a customer of yours, have really opened up the mainstream visibility into this new generation of applications, user interface, roll of data, large scale, how to make that programmable so we're going to need that infrastructure. So thanks for coming on this season three, episode one of the ongoing series of the hot startups. In this case, this episode is the top startups building foundational model infrastructure for AI and ML. I'm John Furrier, your host. Thanks for watching. (upbeat music)

(light airy music) >> Hello, everyone, welcome to theCUBE's presentation of the AWS Startup Showcase, AI and machine learning. "Top Startups Building Generative AI on AWS." This is season three, episode one of the ongoing series covering the exciting startups from the AWS ecosystem, talking about AI machine learning. We have three great guests Bratin Saha, VP, Vice President of Machine Learning and AI Services at Amazon Web Services. Tom Mason, the CTO of Stability AI, and Aidan Gomez, CEO and co-founder of Cohere. Two practitioners doing startups and AWS. Gentlemen, thank you for opening up this session, this episode. Thanks for coming on. >> Thank you. >> Thank you. >> Thank you. >> So the topic is hype versus reality. So I think we're all on the reality is great, hype is great, but the reality's here. I want to get into it. Generative AI's got all the momentum, it's going mainstream, it's kind of come out of the behind the ropes, it's now mainstream. We saw the success of ChatGPT, opens up everyone's eyes, but there's so much more going on. Let's jump in and get your early perspectives on what should people be talking about right now? What are you guys working on? We'll start with AWS. What's the big focus right now for you guys as you come into this market that's highly active, highly hyped up, but people see value right out of the gate? >> You know, we have been working on generative AI for some time. In fact, last year we released Code Whisperer, which is about using generative AI for software development and a number of customers are using it and getting real value out of it. So generative AI is now something that's mainstream that can be used by enterprise users. And we have also been partnering with a number of other companies. So, you know, stability.ai, we've been partnering with them a lot. We want to be partnering with other companies as well. In seeing how we do three things, you know, first is providing the most efficient infrastructure for generative AI. And that is where, you know, things like Trainium, things like Inferentia, things like SageMaker come in. And then next is the set of models and then the third is the kind of applications like Code Whisperer and so on. So, you know, it's early days yet, but clearly there's a lot of amazing capabilities that will come out and something that, you know, our customers are starting to pay a lot of attention to. >> Tom, talk about your company and what your focus is and why the Amazon Web Services relationship's important for you? >> So yeah, we're primarily committed to making incredible open source foundation models and obviously stable effusions been our kind of first big model there, which we trained all on AWS. We've been working with them over the last year and a half to develop, obviously a big cluster, and bring all that compute to training these models at scale, which has been a really successful partnership. And we're excited to take it further this year as we develop commercial strategy of the business and build out, you know, the ability for enterprise customers to come and get all the value from these models that we think they can get. So we're really excited about the future. We got hugely exciting pipeline for this year with new modalities and video models and wonderful things and trying to solve images for once and for all and get the kind of general value and value proposition correct for customers. So it's a really exciting time and very honored to be part of it. >> It's great to see some of your customers doing so well out there. Congratulations to your team. Appreciate that. Aidan, let's get into what you guys do. What does Cohere do? What are you excited about right now? >> Yeah, so Cohere builds large language models, which are the backbone of applications like ChatGPT and GPT-3. We're extremely focused on solving the issues with adoption for enterprise. So it's great that you can make a super flashy demo for consumers, but it takes a lot to actually get it into billion user products and large global enterprises. So about six months ago, we released our command models, which are some of the best that exist for large language models. And in December, we released our multilingual text understanding models and that's on over a hundred different languages and it's trained on, you know, authentic data directly from native speakers. And so we're super excited to continue pushing this into enterprise and solving those barriers for adoption, making this transformation a reality. >> Just real quick, while I got you there on the new products coming out. Where are we in the progress? People see some of the new stuff out there right now. There's so much more headroom. Can you just scope out in your mind what that looks like? Like from a headroom standpoint? Okay, we see ChatGPT. "Oh yeah, it writes my papers for me, does some homework for me." I mean okay, yawn, maybe people say that, (Aidan chuckles) people excited or people are blown away. I mean, it's helped theCUBE out, it helps me, you know, feed up a little bit from my write-ups but it's not always perfect. >> Yeah, at the moment it's like a writing assistant, right? And it's still super early in the technologies trajectory. I think it's fascinating and it's interesting but its impact is still really limited. I think in the next year, like within the next eight months, we're going to see some major changes. You've already seen the very first hints of that with stuff like Bing Chat, where you augment these dialogue models with an external knowledge base. So now the models can be kept up to date to the millisecond, right? Because they can search the web and they can see events that happened a millisecond ago. But that's still limited in the sense that when you ask the question, what can these models actually do? Well they can just write text back at you. That's the extent of what they can do. And so the real project, the real effort, that I think we're all working towards is actually taking action. So what happens when you give these models the ability to use tools, to use APIs? What can they do when they can actually affect change out in the real world, beyond just streaming text back at the user? I think that's the really exciting piece. >> Okay, so I wanted to tee that up early in the segment 'cause I want to get into the customer applications. We're seeing early adopters come in, using the technology because they have a lot of data, they have a lot of large language model opportunities and then there's a big fast follower wave coming behind it. I call that the people who are going to jump in the pool early and get into it. They might not be advanced. Can you guys share what customer applications are being used with large language and vision models today and how they're using it to transform on the early adopter side, and how is that a tell sign of what's to come? >> You know, one of the things we have been seeing both with the text models that Aidan talked about as well as the vision models that stability.ai does, Tom, is customers are really using it to change the way you interact with information. You know, one example of a customer that we have, is someone who's kind of using that to query customer conversations and ask questions like, you know, "What was the customer issue? How did we solve it?" And trying to get those kinds of insights that was previously much harder to do. And then of course software is a big area. You know, generating software, making that, you know, just deploying it in production. Those have been really big areas that we have seen customers start to do. You know, looking at documentation, like instead of you know, searching for stuff and so on, you know, you just have an interactive way, in which you can just look at the documentation for a product. You know, all of this goes to where we need to take the technology. One of which is, you know, the models have to be there but they have to work reliably in a production setting at scale, with privacy, with security, and you know, making sure all of this is happening, is going to be really key. That is what, you know, we at AWS are looking to do, which is work with partners like stability and others and in the open source and really take all of these and make them available at scale to customers, where they work reliably. >> Tom, Aidan, what's your thoughts on this? Where are customers landing on this first use cases or set of low-hanging fruit use cases or applications? >> Yeah, so I think like the first group of adopters that really found product market fit were the copywriting companies. So one great example of that is HyperWrite. Another one is Jasper. And so for Cohere, that's the tip of the iceberg, like there's a very long tail of usage from a bunch of different applications. HyperWrite is one of our customers, they help beat writer's block by drafting blog posts, emails, and marketing copy. We also have a global audio streaming platform, which is using us the power of search engine that can comb through podcast transcripts, in a bunch of different languages. Then a global apparel brand, which is using us to transform how they interact with their customers through a virtual assistant, two dozen global news outlets who are using us for news summarization. So really like, these large language models, they can be deployed all over the place into every single industry sector, language is everywhere. It's hard to think of any company on Earth that doesn't use language. So it's, very, very- >> We're doing it right now. We got the language coming in. >> Exactly. >> We'll transcribe this puppy. All right. Tom, on your side, what do you see the- >> Yeah, we're seeing some amazing applications of it and you know, I guess that's partly been, because of the growth in the open source community and some of these applications have come from there that are then triggering this secondary wave of innovation, which is coming a lot from, you know, controllability and explainability of the model. But we've got companies like, you know, Jasper, which Aidan mentioned, who are using stable diffusion for image generation in block creation, content creation. We've got Lensa, you know, which exploded, and is built on top of stable diffusion for fine tuning so people can bring themselves and their pets and you know, everything into the models. So we've now got fine tuned stable diffusion at scale, which is democratized, you know, that process, which is really fun to see your Lensa, you know, exploded. You know, I think it was the largest growing app in the App Store at one point. And lots of other examples like NightCafe and Lexica and Playground. So seeing lots of cool applications. >> So much applications, we'll probably be a customer for all you guys. We'll definitely talk after. But the challenges are there for people adopting, they want to get into what you guys see as the challenges that turn into opportunities. How do you see the customers adopting generative AI applications? For example, we have massive amounts of transcripts, timed up to all the videos. I don't even know what to do. Do I just, do I code my API there. So, everyone has this problem, every vertical has these use cases. What are the challenges for people getting into this and adopting these applications? Is it figuring out what to do first? Or is it a technical setup? Do they stand up stuff, they just go to Amazon? What do you guys see as the challenges? >> I think, you know, the first thing is coming up with where you think you're going to reimagine your customer experience by using generative AI. You know, we talked about Ada, and Tom talked about a number of these ones and you know, you pick up one or two of these, to get that robust. And then once you have them, you know, we have models and we'll have more models on AWS, these large language models that Aidan was talking about. Then you go in and start using these models and testing them out and seeing whether they fit in use case or not. In many situations, like you said, John, our customers want to say, "You know, I know you've trained these models on a lot of publicly available data, but I want to be able to customize it for my use cases. Because, you know, there's some knowledge that I have created and I want to be able to use that." And then in many cases, and I think Aidan mentioned this. You know, you need these models to be up to date. Like you can't have it staying. And in those cases, you augmented with a knowledge base, you know you have to make sure that these models are not hallucinating. And so you need to be able to do the right kind of responsible AI checks. So, you know, you start with a particular use case, and there are a lot of them. Then, you know, you can come to AWS, and then look at one of the many models we have and you know, we are going to have more models for other modalities as well. And then, you know, play around with the models. We have a playground kind of thing where you can test these models on some data and then you can probably, you will probably want to bring your own data, customize it to your own needs, do some of the testing to make sure that the model is giving the right output and then just deploy it. And you know, we have a lot of tools. >> Yeah. >> To make this easy for our customers. >> How should people think about large language models? Because do they think about it as something that they tap into with their IP or their data? Or is it a large language model that they apply into their system? Is the interface that way? What's the interaction look like? >> In many situations, you can use these models out of the box. But in typical, in most of the other situations, you will want to customize it with your own data or with your own expectations. So the typical use case would be, you know, these are models are exposed through APIs. So the typical use case would be, you know you're using these APIs a little bit for testing and getting familiar and then there will be an API that will allow you to train this model further on your data. So you use that AI, you know, make sure you augmented the knowledge base. So then you use those APIs to customize the model and then just deploy it in an application. You know, like Tom was mentioning, a number of companies that are using these models. So once you have it, then you know, you again, use an endpoint API and use it in an application. >> All right, I love the example. I want to ask Tom and Aidan, because like most my experience with Amazon Web Service in 2007, I would stand up in EC2, put my code on there, play around, if it didn't work out, I'd shut it down. Is that a similar dynamic we're going to see with the machine learning where developers just kind of log in and stand up infrastructure and play around and then have a cloud-like experience? >> So I can go first. So I mean, we obviously, with AWS working really closely with the SageMaker team, do fantastic platform there for ML training and inference. And you know, going back to your point earlier, you know, where the data is, is hugely important for companies. Many companies bringing their models to their data in AWS on-premise for them is hugely important. Having the models to be, you know, open sources, makes them explainable and transparent to the adopters of those models. So, you know, we are really excited to work with the SageMaker team over the coming year to bring companies to that platform and make the most of our models. >> Aidan, what's your take on developers? Do they just need to have a team in place, if we want to interface with you guys? Let's say, can they start learning? What do they got to do to set up? >> Yeah, so I think for Cohere, our product makes it much, much easier to people, for people to get started and start building, it solves a lot of the productionization problems. But of course with SageMaker, like Tom was saying, I think that lowers a barrier even further because it solves problems like data privacy. So I want to underline what Bratin was saying earlier around when you're fine tuning or when you're using these models, you don't want your data being incorporated into someone else's model. You don't want it being used for training elsewhere. And so the ability to solve for enterprises, that data privacy and that security guarantee has been hugely important for Cohere, and that's very easy to do through SageMaker. >> Yeah. >> But the barriers for using this technology are coming down super quickly. And so for developers, it's just becoming completely intuitive. I love this, there's this quote from Andrej Karpathy. He was saying like, "It really wasn't on my 2022 list of things to happen that English would become, you know, the most popular programming language." And so the barrier is coming down- >> Yeah. >> Super quickly and it's exciting to see. >> It's going to be awesome for all the companies here, and then we'll do more, we're probably going to see explosion of startups, already seeing that, the maps, ecosystem maps, the landscape maps are happening. So this is happening and I'm convinced it's not yesterday's chat bot, it's not yesterday's AI Ops. It's a whole another ballgame. So I have to ask you guys for the final question before we kick off the company's showcasing here. How do you guys gauge success of generative AI applications? Is there a lens to look through and say, okay, how do I see success? It could be just getting a win or is it a bigger picture? Bratin we'll start with you. How do you gauge success for generative AI? >> You know, ultimately it's about bringing business value to our customers. And making sure that those customers are able to reimagine their experiences by using generative AI. Now the way to get their ease, of course to deploy those models in a safe, effective manner, and ensuring that all of the robustness and the security guarantees and the privacy guarantees are all there. And we want to make sure that this transitions from something that's great demos to actual at scale products, which means making them work reliably all of the time not just some of the time. >> Tom, what's your gauge for success? >> Look, I think this, we're seeing a completely new form of ways to interact with data, to make data intelligent, and directly to bring in new revenue streams into business. So if businesses can use our models to leverage that and generate completely new revenue streams and ultimately bring incredible new value to their customers, then that's fantastic. And we hope we can power that revolution. >> Aidan, what's your take? >> Yeah, reiterating Bratin and Tom's point, I think that value in the enterprise and value in market is like a huge, you know, it's the goal that we're striving towards. I also think that, you know, the value to consumers and actual users and the transformation of the surface area of technology to create experiences like ChatGPT that are magical and it's the first time in human history we've been able to talk to something compelling that's not a human. I think that in itself is just extraordinary and so exciting to see. >> It really brings up a whole another category of markets. B2B, B2C, it's B2D, business to developer. Because I think this is kind of the big trend the consumers have to win. The developers coding the apps, it's a whole another sea change. Reminds me everyone use the "Moneyball" movie as example during the big data wave. Then you know, the value of data. There's a scene in "Moneyball" at the end, where Billy Beane's getting the offer from the Red Sox, then the owner says to the Red Sox, "If every team's not rebuilding their teams based upon your model, there'll be dinosaurs." I think that's the same with AI here. Every company will have to need to think about their business model and how they operate with AI. So it'll be a great run. >> Completely Agree >> It'll be a great run. >> Yeah. >> Aidan, Tom, thank you so much for sharing about your experiences at your companies and congratulations on your success and it's just the beginning. And Bratin, thanks for coming on representing AWS. And thank you, appreciate for what you do. Thank you. >> Thank you, John. Thank you, Aidan. >> Thank you John. >> Thanks so much. >> Okay, let's kick off season three, episode one. I'm John Furrier, your host. Thanks for watching. (light airy music)

(energetic music) >> Everyone, welcome back to theCUBE's coverage of WiDS 2023. This is the eighth annual Women in Data Science Conference. As you know, WiDS is not just a conference or an event, it's a movement. This is going to include over 100,000 people in the next year WiDS 2023 in 200-plus countries. It is such a powerful movement. If you've had a chance to be part of the Livestream or even be here in person with us at Stanford University, you know what I'm talking about. This is Lisa Martin. I have had the pleasure all day of working with two fantastic graduate students in Stanford's Data Journalism Master's Program. Hannah Freitag has been here. Tracy Zhang, ladies, it's been such a pleasure working with you today. >> Same wise. >> I want to ask you both what are, as we wrap the day, I'm so inspired, I feel like I could go build an airplane. >> Exactly. >> Probably can't. But WiDS is just the inspiration that comes from this event. When you walk in the front door, you can feel it. >> Mm-hmm. >> Tracy, talk a little bit about what some of the things are that you heard today that really inspired you. >> I think one of the keyword that's like in my mind right now is like finding a mentor. >> Yeah. >> And I think, like if I leave this conference if I leave the talks, the conversations with one thing is that I'm very positive that if I want to switch, say someday, from Journalism to being a Data Analyst, to being like in Data Science, I'm sure that there are great role models for me to look up to, and I'm sure there are like mentors who can guide me through the way. So, like that, I feel reassured for some reason. >> It's a good feeling, isn't it? What do you, Hannah, what about you? What's your takeaway so far of the day? >> Yeah, one of my key takeaways is that anything's possible. >> Mm-hmm. >> So, if you have your vision, you have the role model, someone you look up to, and even if you have like a different background, not in Data Science, Data Engineering, or Computer Science but you're like, "Wow, this is really inspiring. I would love to do that." As long as you love it, you're passionate about it, and you are willing to, you know, take this path even though it won't be easy. >> Yeah. >> Then you can achieve it, and as you said, Tracy, it's important to have mentors on the way there. >> Exactly. >> But as long as you speak up, you know, you raise your voice, you ask questions, and you're curious, you can make it. >> Yeah. >> And I think that's one of my key takeaways, and I was just so inspiring to hear like all these women speaking on stage, and also here in our conversations and learning about their, you know, career path and what they learned on their way. >> Yeah, you bring up curiosity, and I think that is such an important skill. >> Mm-hmm. >> You know, you could think of Data Science and think about all the hard skills that you need. >> Mm, like coding. >> But as some of our guests said today, you don't have to be a statistician or an engineer, or a developer to get into this. Data Science applies to every facet of every part of the world. >> Mm-hmm. >> Finances, marketing, retail, manufacturing, healthcare, you name it, Data Science has the power and the potential to unlock massive achievements. >> Exactly. >> It's like we're scratching the surface. >> Yeah. >> But that curiosity, I think, is a great skill to bring to anything that you do. >> Mm-hmm. >> And I think we... For the female leaders that we're on stage, and that we had a chance to talk to on theCUBE today, I think they all probably had that I think as a common denominator. >> Exactly. >> That curious mindset, and also something that I think as hard is the courage to raise your hand. I like this, I'm interested in this. I don't see anybody that looks like me. >> But that doesn't mean I shouldn't do it. >> Exactly. >> Exactly, in addition to the curiosity that all the women, you know, bring to the table is that, in addition to that, being optimistic, and even though we don't see gender equality or like general equality in companies yet, we make progress and we're optimistic about it, and we're not like negative and complaining the whole time. But you know, this positive attitude towards a trend that is going in the right direction, and even though there's still a lot to be done- >> Exactly. >> We're moving it that way. >> Right. >> Being optimistic about this. >> Yeah, exactly, like even if it means that it's hard. Even if it means you need to be your own role model it's still like worth a try. And I think they, like all of the great women speakers, all the female leaders, they all have that in them, like they have the courage to like raise their hand and be like, "I want to do this, and I'm going to make it." And they're role models right now, so- >> Absolutely, they have drive. >> They do. >> Right. They have that ambition to take something that's challenging and complicated, and help abstract end users from that. Like we were talking to Intuit. I use Intuit in my small business for financial management, and she was talking about how they can from a machine learning standpoint, pull all this data off of documents that you upload and make that, abstract that, all that complexity from the end user, make something that's painful taxes. >> Mm-hmm. >> Maybe slightly less painful. It's still painful when you have to go, "Do I have to write you a check again?" >> Yeah. (laughs) >> Okay. >> But talking about just all the different applications of Data Science in the world, I found that to be very inspiring and really eye-opening. >> Definitely. >> I hadn't thought about, you know, we talk about climate change all the time, especially here in California, but I never thought about Data Science as a facilitator of the experts being able to make sense of what's going on historically and in real-time, or the application of Data Science in police violence. We see far too many cases of police violence on the news. It's an epidemic that's a horrible problem. Data Science can be applied to that to help us learn from that, and hopefully, start moving the needle in the right direction. >> Absolutely. >> Exactly. >> And especially like one sentence from Guitry from the very beginnings I still have in my mind is then when she said that arguments, no, that data beats arguments. >> Yes. >> In a conversation that if you be like, okay, I have this data set and it can actually show you this or that, it's much more powerful than just like being, okay, this is my position or opinion on this. And I think in a world where increasing like misinformation, and sometimes, censorship as we heard in one of the talks, it's so important to have like data, reliable data, but also acknowledge, and we talked about it with one of our interviewees that there's spices in data and we also need to be aware of this, and how to, you know, move this forward and use Data Science for social good. >> Mm-hmm. >> Yeah, for social good. >> Yeah, definitely, I think they like data, and the question about, or like the problem-solving part about like the social issues, or like some just questions, they definitely go hand-in-hand. Like either of them standing alone won't be anything that's going to be having an impact, but combining them together, you have a data set that illustrate a point or like solves the problem. I think, yeah, that's definitely like where Data Set Science is headed to, and I'm glad to see all these great women like making their impact and combining those two aspects together. >> It was interesting in the keynote this morning. We were all there when Margot Gerritsen who's one of the founders of WiDS, and Margot's been on the program before and she's a huge supporter of what we do and vice versa. She asked the non-women in the room, "Those who don't identify as women, stand up," and there was a handful of men, and she said, "That's what it's like to be a female in technology." >> Oh, my God. >> And I thought that vision give me goosebumps. >> Powerful. (laughs) >> Very powerful. But she's right, and one of the things I think that thematically another common denominator that I think we heard, I want to get your opinions as well from our conversations today, is the importance of community. >> Mm-hmm. >> You know, I was mentioning this stuff from AnitaB.org that showed that in 2022, the percentage of females and technical roles is 27.6%. It's a little bit of an increase. It's been hovering around 25% for a while. But one of the things that's still a problem is attrition. It doubled last year. >> Right. >> And I was asking some of the guests, and we've all done that today, "How would you advise companies to start moving the needle down on attrition?" >> Mm-hmm. >> And I think the common theme was network, community. >> Exactly. >> It takes a village like this. >> Mm-hmm. >> So you can see what you can be to help start moving that needle and that's, I think, what underscores the value of what WiDS delivers, and what we're able to showcase on theCUBE. >> Yeah, absolutely. >> I think it's very important to like if you're like a woman in tech to be able to know that there's someone for you, that there's a whole community you can rely on, and that like you are, you have the same mindset, you're working towards the same goal. And it's just reassuring and like it feels very nice and warm to have all these women for you. >> Lisa: It's definitely a warm fuzzy, isn't it? >> Yeah, and both the community within the workplace but also outside, like a network of family and friends who support you to- >> Yes. >> To pursue your career goals. I think that was also a common theme we heard that it's, yeah, necessary to both have, you know your community within your company or organization you're working but also outside. >> Definitely, I think that's also like how, why, the reason why we feel like this in like at WiDS, like I think we all feel very positive right now. So, yeah, I think that's like the power of the connection and the community, yeah. >> And the nice thing is this is like I said, WiDS is a movement. >> Yes. >> This is global. >> Mm-hmm. >> We've had some WiDS ambassadors on the program who started WiDS and Tel Aviv, for example, in their small communities. Or in Singapore and Mumbai that are bringing it here and becoming more of a visible part of the community. >> Tracy: Right. >> I loved seeing all the young faces when we walked in the keynote this morning. You know, we come here from a journalistic perspective. You guys are Journalism students. But seeing all the potential in the faces in that room just seeing, and hearing stories, and starting to make tangible connections between Facebook and data, and the end user and the perspectives, and the privacy and the responsibility of AI is all... They're all positive messages that need to be reinforced, and we need to have more platforms like this to be able to not just raise awareness, but sustain it. >> Exactly. >> Right. It's about the long-term, it's about how do we dial down that attrition, what can we do? What can we do? How can we help? >> Mm-hmm. >> Both awareness, but also giving women like a place where they can connect, you know, also outside of conferences. Okay, how do we make this like a long-term thing? So, I think WiDS is a great way to, you know, encourage this connectivity and these women teaming up. >> Yeah, (chuckles) girls help girls. >> Yeah. (laughs) >> It's true. There's a lot of organizations out there, girls who Code, Girls Inc., et cetera, that are all aimed at helping women kind of find their, I think, find their voice. >> Exactly. >> And find that curiosity. >> Yeah. Unlock that somewhere back there. Get some courage- >> Mm-hmm. >> To raise your hand and say, "I think I want to do this," or "I have a question. You explained something and I didn't understand it." Like, that's the advice I would always give to my younger self is never be afraid to raise your hand in a meeting. >> Mm-hmm. >> I guarantee you half the people weren't listening or, and the other half may not have understood what was being talked about. >> Exactly. >> So, raise your hand, there goes Margot Gerritsen, the founder of WiDS, hey, Margot. >> Hi. >> Keep alumni as you know, raise your hand, ask the question, there's no question that's stupid. >> Mm-hmm. >> And I promise you, if you just take that chance once it will open up so many doors, you won't even know which door to go in because there's so many that are opening. >> And if you have a question, there's at least one more person in the room who has the exact same question. >> Exact same question. >> Yeah, we'll definitely keep that in mind as students- >> Well, I'm curious how Data Journalism, what you heard today, Tracy, we'll start with you, and then, Hannah, to you. >> Mm-hmm. How has it influenced how you approach data-driven, and storytelling? Has it inspired you? I imagine it has, or has it given you any new ideas for, as you round out your Master's Program in the next few months? >> I think like one keyword that I found really helpful from like all the conversations today, was problem-solving. >> Yeah. >> Because I think, like we talked a lot about in our program about how to put a face on data sets. How to put a face, put a name on a story that's like coming from like big data, a lot of numbers but you need to like narrow it down to like one person or one anecdote that represents a bigger problem. And I think essentially that's problem-solving. That's like there is a community, there is like say maybe even just one person who has, well, some problem about something, and then we're using data. We're, by giving them a voice, by portraying them in news and like representing them in the media, we're solving this problem somehow. We're at least trying to solve this problem, trying to make some impact. And I think that's like what Data Science is about, is problem-solving, and, yeah, I think I heard a lot from today's conversation, also today's speakers. So, yeah, I think that's like something we should also think about as Journalists when we do pitches or like what kind of problem are we solving? >> I love that. >> Or like kind of what community are we trying to make an impact in? >> Yes. >> Absolutely. Yeah, I think one of the main learnings for me that I want to apply like to my career in Data Journalism is that I don't shy away from complexity because like Data Science is oftentimes very complex. >> Complex. >> And also data, you're using for your stories is complex. >> Mm-hmm. >> So, how can we, on the one hand, reduce complexity in a way that we make it accessible for broader audience? 'Cause, we don't want to be this like tech bubble talking in data jargon, we want to, you know, make it accessible for a broader audience. >> Yeah. >> I think that's like my purpose as a Data Journalist. But at the same time, don't reduce complexity when it's needed, you know, and be open to dive into new topics, and data sets and circling back to this of like raising your hand and asking questions if you don't understand like a certain part. >> Yeah. >> So, that's definitely a main learning from this conference. >> Definitely. >> That like, people are willing to talk to you and explain complex topics, and this will definitely facilitate your work as a Data Journalist. >> Mm-hmm. >> So, that inspired me. >> Well, I can't wait to see where you guys go from here. I've loved co-hosting with you today, thank you. >> Thank you. >> For joining me at our conference. >> Wasn't it fun? >> Thank you. >> It's a great event. It's, we, I think we've all been very inspired and I'm going to leave here probably floating above the ground a few inches, high on the inspiration of what this community can deliver, isn't that great? >> It feels great, I don't know, I just feel great. >> Me too. (laughs) >> So much good energy, positive energy, we love it. >> Yeah, so we want to thank all the organizers of WiDS, Judy Logan, Margot Gerritsen in particular. We also want to thank John Furrier who is here. And if you know Johnny, know he gets FOMO when he is not hosting. But John and Dave Vellante are such great supporters of women in technology, women in technical roles. We wouldn't be here without them. So, shout out to my bosses. Thank you for giving me the keys to theCube at this event. I know it's painful sometimes, but we hope that we brought you great stories all day. We hope we inspired you with the females and the one male that we had on the program today in terms of raise your hand, ask a question, be curious, don't be afraid to pursue what you're interested in. That's my soapbox moment for now. So, for my co-host, I'm Lisa Martin, we want to thank you so much for watching our program today. You can watch all of this on-demand on thecube.net. You'll find write-ups on siliconeangle.com, and, of course, YouTube. Thanks, everyone, stay safe and we'll see you next time. (energetic music)

(upbeat music) >> Hey everyone. Welcome back to "The Cube"'s live coverage of "Women in Data Science 2023". As every year we are here live at Stanford University, profiling some amazing women and men in the fields of data science. I have my co-host for this segment is Hannah Freitag. Hannah is from Stanford's Data Journalism program, really interesting, check it out. We're very pleased to welcome our first guest of the day fresh from the keynote stage, Gayatree Ganu, the VP of Data Science at Meta. Gayatree, It's great to have you on the program. >> Likewise, Thank you for having me. >> So you have a PhD in Computer Science. You shared some really cool stuff. Everyone knows Facebook, everyone uses it. I think my mom might be one of the biggest users (Gayatree laughs) and she's probably watching right now. People don't realize there's so much data behind that and data that drives decisions that we engage with. But talk to me a little bit about you first, PhD in Computer Science, were you always, were you like a STEM kid? Little Gayatree, little STEM, >> Yeah, I was a STEM kid. I grew up in Mumbai, India. My parents are actually pharmacists, so they were not like math or stats or anything like that, but I was always a STEM kid. I don't know, I think it, I think I was in sixth grade when we got our first personal computer and I obviously used it as a Pacman playing machine. >> Oh, that's okay. (all laugh) >> But I was so good at, and I, I honestly believe I think being good at games kind of got me more familiar and comfortable with computers. Yeah. I think I always liked computers, I, yeah. >> And so now you lead, I'm looking at my notes here, the Engagement Ecosystem and Monetization Data Science teams at Facebook, Meta. Talk about those, what are the missions of those teams and how does it impact the everyday user? >> Yeah, so the engagement is basically users coming back to our platform more, there's, no better way for users to tell us that they are finding value on the things that we are doing on Facebook, Instagram, WhatsApp, all the other products than coming back to our platform more. So the Engagement Ecosystem team is looking at trends, looking at where there are needs, looking at how users are changing their behaviors, and you know, helping build strategy for the long term, using that data knowledge. Monetization is very different. You know, obviously the top, top apex goal is have a sustainable business so that we can continue building products for our users. And so, but you know, I said this in my keynote today, it's not about making money, our mission statement is not, you know, maximize as much money as you can make. It's about building a meaningful connection between businesses, customers, users, and, you know especially in these last two or three funky, post-pandemic years, it's been such a big, an important thing to do for small businesses all over all, all around the world for users to find like goods and services and products that they care about and that they can connect to. So, you know, there is truly an connection between my engagement world and the monetization world. And you know, it's not very clear always till you go in to, like, you peel the layers. Everything we do in the ads world is also always first with users as our, you know, guiding principle. >> Yeah, you mentioned how you supported especially small businesses also during the pandemic. You touched a bit upon it in the keynote speech. Can you tell our audience what were like special or certain specific programs you implemented to support especially small businesses during these times? >> Yeah, so there are 200 million businesses on our platform. A lot of them small businesses, 10 million of them run ads. So there is a large number of like businesses on our platform who, you know use the power of social media to connect to the customers that matter to them, to like you, you know use the free products that we built. In the post-pandemic years, we built a lot of stuff very quickly when Covid first hit for business to get the word out, right? Like, they had to announce when special shopping hours existed for at-risk populations, or when certain goods and services were available versus not. We had grants, there's $100 million grant that we gave out to small businesses. Users could show sort of, you know show their support with a bunch of campaigns that we ran, and of course we continue running ads. Our ads are very effective, I guess, and, you know getting a very reliable connection with from the customer to the business. And so, you know, we've run all these studies. We support, I talked about two examples today. One of them is the largest black-owned, woman black-owned wine company, and how they needed to move to an online program and, you know, we gave them a grant, and supported them through their ads campaign and, you know, they saw 60% lift in purchases, or something like that. So, a lot of good stories, small stories, you know, on a scale of 200 million, that really sort of made me feel proud about the work we do. And you know, now more than ever before, I think people can connect so directly with businesses. You can WhatsApp them, I come from India, every business is on WhatsApp. And you can, you know, WhatsApp them, you can send them Facebook messages, and you can build this like direct connection with things that matter to you. >> We have this expectation that we can be connected anywhere. I was just at Mobile World Congress for MWC last week, where, obviously talking about connectivity. We want to be able to do any transaction, whether it's post on Facebook or call an Uber, or watch on Netflix if you're on the road, we expect that we're going to be connected. >> Yeah. >> And what we, I think a lot of us don't realize I mean, those of us in tech do, but how much data science is a facilitator of all of those interactions. >> Yeah! >> As we, Gayatree, as we talk about, like, any business, whether it is the black women-owned wine business, >> Yeah. >> great business, or a a grocer or a car dealer, everybody has to become data-driven. >> Yes. >> Because the consumer has the expectation. >> Yes. >> Talk about data science as a facilitator of just pretty much everything we are doing and conducting in our daily lives. >> Yeah, I think that's a great question. I think data science as a field wasn't really defined like maybe 15 years ago, right? So this is all in our lifetimes that we are seeing this. Even in data science today, People come from so many different backgrounds and bring their own expertise here. And I think we, you know, this conference, all of us get to define what that means and how we can bring data to do good in the world. Everything you do, as you said, there is a lot of data. Facebook has a lot of data, Meta has a lot of data, and how do we responsibly use this data? How do we use this data to make sure that we're, you know representing all diversity? You know, minorities? Like machine learning algorithms don't do well with small data, they do well with big data, but the small data matters. And how do you like, you know, bring that into algorithms? Yeah, so everything we do at Meta is very, very data-driven. I feel proud about that, to be honest, because while data gets a bad rap sometimes, having no data and making decisions in the blind is just the absolute worst thing you can do. And so, you know, we, the job as a data scientist at Facebook is to make sure that we use this data, use this responsibly, make sure that we are representing every aspect of the, you know, 3 billion users who come to our platform. Yeah, data serves all the products that we build here. >> The responsibility factor is, is huge. You know, we can't talk about AI without talking about ethics. One of the things that I was talking with Hannah and our other co-host, Tracy, about during our opening is something I just learned over the weekend. And that is that the CTO of ChatGPT is a woman. (Gayatree laughs) I didn't know that. And I thought, why isn't she getting more awareness? There's a lot of conversations with their CEO. >> Yeah. >> Everyone's using it, playing around with it. I actually asked it yesterday, "What's hot in Data Science?" (all laugh) I was like, should I have asked that to let itself in, what's hot? (Gayatree laughs) But it, I thought that was phenomenal, and we need to be talking about this more. >> Yeah. >> This is something that they're likening to the launch of the iPhone, which has transformed our lives. >> I know, it is. >> ChatGPT, and its chief technologist is a female, how great is that? >> And I don't know whether you, I don't know the stats around this, but I think CTO is even less, it's even more rare to have a woman there, like you have women CEOs because I mean, we are building upon years and years of women not choosing technical fields and not choosing STEM, and it's going to take some time, but yeah, yeah, she's a woman. Isn't it amazing? It's wonderful. >> Yes, there was a great, there's a great "Fast Company" article on her that I was looking at yesterday and I just thought, we need to do what we can to help spread, Mira Murati is her name, because what she's doing is, one of the biggest technological breakthroughs we may ever see in our lifetime. It gives me goosebumps just thinking about it. (Gayatree laughs) I also wanted to share some stats, oh, sorry, go ahead, Hannah. >> Yeah, I was going to follow up on the thing that you mentioned that we had many years with like not enough women choosing a career path in STEM and that we have to overcome this trend. What are some, like what is some advice you have like as the Vice-President Data Science? Like what can we do to make this feel more, you know, approachable and >> Yeah. >> accessible for women? >> Yeah, I, there's so much that we have done already and you know, want to continue, keep doing. Of course conferences like these were, you know and I think there are high school students here there are students from my Alma Mater's undergrad year. It's amazing to like get all these women together to get them to see what success could look like. >> Yeah. >> What being a woman leader in this space could look like. So that's, you know, that's one, at Meta I lead recruiting at Meta and we've done a bunch to sort of open up the thinking around data science and technical jobs for women. Simple things like what you write in your job description. I don't know whether you know this, or this is a story you've heard before, when you see, when you have a job description and there are like 10 things that you need to, you know be good at to apply to this job, a woman sees those 10 and says, okay, I don't meet the qualifications of one of them and she doesn't apply. And a man sees one that he meets the qualifications to and he applies. And so, you know, there's small things you can do, and just how you write your job description, what goals you set for diversity and inclusion for your own organization. We have goals, Facebook's always been pretty up there in like, you know, speaking out for diversity and Sheryl Sandberg has been our Chief Business Officer for a very long time and she's been, like, amazing at like pushing from more women. So yeah, every step of the way, I think, we made a lot of progress, to be honest. I do think women choose STEM fields a lot more than they did. When I did my Computer Science I was often one of one or two women in the Computer Science class. It takes some time to, for it to percolate all the way to like having more CTOs and CEOs, >> Yeah. >> but it's going to happen in our lifetime, and you know, three of us know this, women are going to rule the world, and it (laughs) >> Drop the mic, girl! >> And it's going to happen in our lifetime, so I'm excited about it. >> And we have responsibility in helping make that happen. You know, I'm curious, you were in STEM, you talked about Computer Science, being one of the only females. One of the things that the nadb.org data from 2022 showed, some good numbers, the number of women in technical roles is now 27.6%, I believe, so up from 25, it's up in '22, which is good, more hiring of women. >> Yeah. >> One of the biggest challenges is attrition. What keeps you motivated? >> Yeah. >> To stay what, where you are doing what you're doing, managing a family and helping to drive these experiences at Facebook that we all expect are just going to happen? >> Yeah, two things come to mind. It does take a village. You do need people around you. You know, I'm grateful for my husband. You talked about managing a family, I did the very Indian thing and my parents live with us, and they help take care of the kids. >> Right! (laughs) >> (laughs) My kids are young, six and four, and I definitely needed help over the last few years. It takes mentors, it takes other people that you look up to, who've gone through all of those same challenges and can, you know, advise you to sort of continue working in the field. I remember when my kid was born when he was six months old, I was considering quitting. And my husband's like, to be a good role model for your children, you need to continue working. Like, just being a mother is not enough. And so, you know, so that's one. You know, the village that you build around you your supporters, your mentors who keep encouraging you. Sheryl Sandberg said this to me in my second month at Facebook. She said that women drop out of technical fields, they become managers, they become sort of administrative more, in their nature of their work, and her advice was, "Don't do that, Don't stop the technical". And I think that's the other thing I'd say to a lot of women. Technical stuff is hard, but you know, keeping up with that and keeping sort of on top of it actually does help you in the long run. And it's definitely helped me in my career at Facebook. >> I think one of the things, and Hannah and I and Tracy talked about this in the open, and I think you'll agree with us, is the whole saying of you can't be what you can't see, and I like to way, "Well, you can be what you can see". That visibility, the great thing that WiDS did, of having you on the stage as a speaker this morning so people can understand, everyone, like I said, everyone knows Meta, >> Yeah. >> everyone uses Facebook. And so it's important to bring that connection, >> Yeah. >> of how data is driving the experiences, the fact that it's User First, but we need to be able to see women in positions, >> Yes. >> like you, especially with Sheryl stepping down moving on to something else, or people that are like YouTube influencers, that have no idea that the head of YouTube for a very long time, Susan Wojcicki is a woman. >> (laughs) Yes. Who pioneered streaming, and I mean how often do you are you on YouTube every day? >> Yep, every day. >> But we have to be able to see and and raise the profile of these women and learn from them and be inspired, >> Absolutely. >> to keep going and going. I like what I do, I'm making a difference here. >> Yeah, yeah, absolutely. >> And I can be the, the sponsor or the mentor for somebody down the road. >> Absolutely. >> Yeah, and then referring back to what we talked in the beginning, show that data science is so diverse and it doesn't mean if you're like in IT, you're like sitting in your dark room, >> Right. (laughs) >> coding all day, but you know, >> (laughs) Right! >> to show the different facets of this job and >> Right! >> make this appealing to women, >> Yeah. for sure. >> And I said this in my keynote too, you know, one of the things that helped me most is complimenting the data and the techniques and the algorithms with how you work with people, and you know, empathy and alignment building and leadership, strategic thinking. And I think honestly, I think women do a lot of this stuff really well. We know how to work with people and so, you know, I've seen this at Meta for sure, like, you know, all of these skills soft skills, as we call them, go a long way, and like, you know, doing the right things and having a lasting impact. And like I said, women are going to rule the world, you know, in our lifetimes. (laughs) >> Oh, I can't, I can't wait to see that happen. There's some interesting female candidates that are already throwing their hats in the ring for the next presidential election. >> Yes. >> So we'll have to see where that goes. But some of the things that are so interesting to me, here we are in California and Palo Alto, technically Stanford is its own zip code, I believe. And we're in California, we're freaking out because we've gotten so much rain, it's absolutely unprecedented. We need it, we had a massive drought, an extreme drought, technically, for many years. I've got friends that live up in Tahoe, I've been getting pictures this morning of windows that are >> (laughs) that are covered? >> Yes, actually, yes. (Gayatree laughs) That, where windows like second-story windows are covered in snow. >> Yeah. >> Climate change. >> Climate change. >> There's so much that data science is doing to power and power our understanding of climate change whether it's that, or police violence. >> Yeah. (all talk together) >> We had talk today on that it was amazing. >> Yes. So I want more people to know what data science is really facilitating, that impacts all of us, whether you're in a technical role or not. >> And data wins arguments. >> Yes, I love that! >> I said this is my slide today, like, you know, there's always going to be doubters and naysayers and I mean, but there's hard evidence, there's hard data like, yeah. In all of these fields, I mean the data that climate change, the data science that we have done in the environmental and climate change areas and medical, and you know, medicine professions just so much, so much more opportunity, and like, how much we can learn more about the world. >> Yeah. >> Yeah, it's a pretty exciting time to be a data scientist. >> I feel like, we're just scratching the surface. >> Yeah. >> With the potential and the global impact that we can make with data science. Gayatree, it's been so great having you on theCUBE, thank you. >> Right, >> Thank you so much, Gayatree. >> So much, I love, >> Thank you. >> I'm going to take Data WiD's arguments into my personal life. (Gayatree laughs) I was actually just, just a quick anecdote, funny story. I was listening to the radio this morning and there was a commercial from an insurance company and I guess the joke is, it's an argument between two spouses, and the the voiceover comes in and says, "Let's watch a replay". I'm like, if only they, then they got the data that helped the woman win the argument. (laughs) >> (laughs) I will warn you it doesn't always help with arguments I have with my husband. (laughs) >> Okay, I'm going to keep it in the middle of my mind. >> Yes! >> Gayatree, thank you so much. >> Thank you so much, >> for sharing, >> Thank you both for the opportunity. >> And being a great female that we can look up to, we really appreciate your insights >> Oh, likewise. >> and your time. >> Thank you. >> All right, for our guest, for Hannah Freitag, I'm Lisa Martin, live at Stanford University covering "Women in Data Science '23". Stick around, our next guest joins us in just a minute. (upbeat music) I have been in the software and technology industry for over 12 years now, so I've had the opportunity as a marketer to really understand and interact with customers across the entire buyer's journey. Hi, I'm Lisa Martin and I'm a host of theCUBE. (upbeat music) Being a host on theCUBE has been a dream of mine for the last few years. I had the opportunity to meet Jeff and Dave and John at EMC World a few years ago and got the courage up to say, "Hey, I'm really interested in this. I love talking with customers, gimme a shot, let me come into the studio and do an interview and see if we can work together". I think where I really impact theCUBE is being a female in technology. We interview a lot of females in tech, we do a lot of women in technology events and one of the things I.

(ambient music) >> Good morning, everyone. Lisa Martin with theCUBE, live at the eighth Annual Women in Data Science Conference. This is one of my absolute favorite events of the year. We engage with tons of great inspirational speakers, men and women, and what's happening with WiDS is a global movement. I've got two fabulous co-hosts with me today that you're going to be hearing and meeting. Please welcome Tracy Zhang and Hannah Freitag, who are both from the sata journalism program, master's program, at Stanford. So great to have you guys. >> So excited to be here. >> So data journalism's so interesting. Tracy, tell us a little bit about you, what you're interested in, and then Hannah we'll have you do the same thing. >> Yeah >> Yeah, definitely. I definitely think data journalism is very interesting, and in fact, I think, what is data journalism? Is definitely one of the big questions that we ask during the span of one year, which is the length of our program. And yeah, like you said, I'm in this data journalism master program, and I think coming in I just wanted to pivot from my undergrad studies, which is more like a traditional journalism, into data. We're finding stories through data, so that's why I'm also very excited about meeting these speakers for today because they're all, they have different backgrounds, but they all ended up in data science. So I think they'll be very inspirational and I can't wait to talk to them. >> Data in stories, I love that. Hannah, tell us a little bit about you. >> Yeah, so before coming to Stanford, I was a research assistant at Humboldt University in Berlin, so I was in political science research. And I love to work with data sets and data, but I figured that, for me, I don't want this story to end up in a research paper, which is only very limited in terms of the audience. And I figured, okay, data journalism is the perfect way to tell stories and use data to illustrate anecdotes, but to make it comprehensive and accessible for a broader audience. So then I found this program at Stanford and I was like, okay, that's the perfect transition from political science to journalism, and to use data to tell data-driven stories. So I'm excited to be in this program, I'm excited for the conference today and to hear from these amazing women who work in data science. >> You both brought up great points, and we were chatting earlier that there's a lot of diversity in background. >> Tracy: Definitely. >> Not everyone was in STEM as a young kid or studied computer science. Maybe some are engineering, maybe some are are philosophy or economic, it's so interesting. And what I find year after year at WiDS is it brings in so much thought diversity. And that's what being data-driven really demands. It demands that unbiased approach, that diverse, a spectrum of diverse perspectives, and we definitely get that at WiDS. There's about 350 people in person here, but as I mentioned in the opening, hundreds of thousands will engage throughout the year, tens of thousands probably today at local events going on across the globe. And it just underscores the importance of every organization, whether it's a bank or a grocer, has to be data-driven. We have that expectation as consumers in our consumer lives, and even in our business lives, that I'm going to engage with a business, whatever it is, and they're going to know about me, they're going to deliver me a personalized experience that's relevant to me and my history. And all that is powered by data science, which is I think it's fascinating. >> Yeah, and the great way is if you combine data with people. Because after all, large data sets, they oftentimes consist of stories or data that affects people. And to find these stories or advanced research in whatever fields, maybe in the financial business, or in health, as you mentioned, the variety of fields, it's very powerful, powerful tool to use. >> It's a very power, oh, go ahead Tracy. >> No, definitely. I just wanted to build off of that. It's important to put a face on data. So a dataset without a name is just some numbers, but if there's a story, then I think it means something too. And I think Margot was talking about how data science is about knowing or understanding the past, I think that's very interesting. That's a method for us to know who we are. >> Definitely. There's so many opportunities. I wanted to share some of the statistics from AnitaB.org that I was just looking at from 2022. We always talk at events like WiDS, and some of the other women in tech things, theCUBE is very much pro-women in tech, and has been for a very long, since the beginning of theCUBE. But we've seen the numbers of women technologists historically well below 25%, and we see attrition rates are high. And so we often talk about, well, what can we do? And part of that is raising the awareness. And that's one of the great things about WiDS, especially WiDS happening on International Women's Day, today, March 8th, and around event- >> Tracy: A big holiday. >> Exactly. But one of the nice things I was looking at, the AnitaB.org research, is that representation of tech women is on the rise, still below pre-pandemic levels, but it's actually nearly 27% of women in technical roles. And that's an increase, slow increase, but the needle is moving. We're seeing much more gender diversity across a lot of career levels, which is exciting. But some of the challenges remain. I mean, the representation of women technologists is growing, except at the intern level. And I thought that was really poignant. We need to be opening up that pipeline and going younger. And you'll hear a lot of those conversations today about, what are we doing to reach girls in grade school, 10 year olds, 12 year olds, those in high school? How do we help foster them through their undergrad studies- >> And excite them about science and all these fields, for sure. >> What do you think, Hannah, on that note, and I'll ask you the same question, what do you think can be done? The theme of this year's International Women's Day is Embrace Equity. What do you think can be done on that intern problem to help really dial up the volume on getting those younger kids interested, one, earlier, and two, helping them stay interested? >> Yeah. Yeah, that's a great question. I think it's important to start early, as you said, in school. Back in the day when I went to high school, we had this one day per year where we could explore as girls, explore a STEM job and go into the job for one day and see how it's like to work in a, I dunno, in IT or in data science, so that's a great first step. But as you mentioned, it's important to keep girls and women excited about this field and make them actually pursue this path. So I think conferences or networking is very powerful. Also these days with social media and technology, we have more ability and greater ways to connect. And I think we should even empower ourselves even more to pursue this path if we're interested in data science, and not be like, okay, maybe it's not for me, or maybe as a woman I have less chances. So I think it's very important to connect with other women, and this is what WiDS is great about. >> WiDS is so fantastic for that network effect, as you talked about. It's always such, as I was telling you about before we went live, I've covered five or six WiDS for theCUBE, and it's always such a day of positivity, it's a day of of inclusivity, which is exactly what Embrace Equity is really kind of about. Tracy, talk a little bit about some of the things that you see that will help with that hashtag Embrace Equity kind of pulling it, not just to tech. Because we're talking and we saw Meta was a keynote who's going to come to talk with Hannah and me in a little bit, we see Total Energies on the program today, we see Microsoft, Intuit, Boeing Air Company. What are some of the things you think that can be done to help inspire, say, little Tracy back in the day to become interested in STEM or in technology or in data? What do you think companies can and should be doing to dial up the volume for those youngsters? >> Yeah, 'cause I think somebody was talking about, one of the keynote speakers was talking about how there is a notion that girls just can't be data scientists. girls just can't do science. And I think representation definitely matters. If three year old me see on TV that all the scientists are women, I think I would definitely have the notion that, oh, this might be a career choice for me and I can definitely also be a scientist if I want. So yeah, I think representation definitely matters and that's why conference like this will just show us how these women are great in their fields. They're great data scientists that are bringing great insight to the company and even to the social good as well. So yeah, I think that's very important just to make women feel seen in this data science field and to listen to the great woman who's doing amazing work. >> Absolutely. There's a saying, you can't be what you can't see. >> Exactly. >> And I like to say, I like to flip it on its head, 'cause we can talk about some of the negatives, but there's a lot of positives and I want to share some of those in a minute, is that we need to be, that visibility that you talked about, the awareness that you talked about, it needs to be there but it needs to be sustained and maintained. And an organization like WiDS and some of the other women in tech events that happen around the valley here and globally, are all aimed at raising the profile of these women so that the younger, really, all generations can see what they can be. We all, the funny thing is, we all have this expectation whether we're transacting on Uber ride or we are on Netflix or we're buying something on Amazon, we can get it like that. They're going to know who I am, they're going to know what I want, they're going to want to know what I just bought or what I just watched. Don't serve me up something that I've already done that. >> Hannah: Yeah. >> Tracy: Yeah. >> So that expectation that everyone has is all about data, though we don't necessarily think about it like that. >> Hannah: Exactly. >> Tracy: Exactly. >> But it's all about the data that, the past data, the data science, as well as the realtime data because we want to have these experiences that are fresh, in the moment, and super relevant. So whether women recognize it or not, they're data driven too. Whether or not you're in data science, we're all driven by data and we have these expectations that every business is going to meet it. >> Exactly. >> Yeah. And circling back to young women, I think it's crucial and important to have role models. As you said, if you see someone and you're younger and you're like, oh I want to be like her. I want to follow this path, and have inspiration and a role model, someone you look up to and be like, okay, this is possible if I study the math part or do the physics, and you kind of have a goal and a vision in mind, I think that's really important to drive you. >> Having those mentors and sponsors, something that's interesting is, I always, everyone knows what a mentor is, somebody that you look up to, that can guide you, that you admire. I didn't learn what a sponsor was until a Women in Tech event a few years ago that we did on theCUBE. And I was kind of, my eyes were open but I didn't understand the difference between a mentor and a sponsor. And then it got me thinking, who are my sponsors? And I started going through LinkedIn, oh, he's a sponsor, she's a sponsor, people that help really propel you forward, your recommenders, your champions, and it's so important at every level to build that network. And we have, to your point, Hannah, there's so much potential here for data drivenness across the globe, and there's so much potential for women. One of the things I also learned recently , and I wanted to share this with you 'cause I'm not sure if you know this, ChatGPT, exploding, I was on it yesterday looking at- >> Everyone talking about it. >> What's hot in data science? And it was kind of like, and I actually asked it, what was hot in data science in 2023? And it told me that it didn't know anything prior to 2021. >> Tracy: Yes. >> Hannah: Yeah. >> So I said, Oh, I'm so sorry. But everyone's talking about ChatGPT, it is the most advanced AI chatbot ever released to the masses, it's on fire. They're likening it to the launch of the iPhone, 100 million-plus users. But did you know that the CTO of ChatGPT is a woman? >> Tracy: I did not know, but I learned that. >> I learned that a couple days ago, Mira Murati, and of course- >> I love it. >> She's been, I saw this great profile piece on her on Fast Company, but of course everything that we're hearing about with respect to ChatGPT, a lot on the CEO. But I thought we need to help dial up the profile of the CTO because she's only 35, yet she is at the helm of one of the most groundbreaking things in our lifetime we'll probably ever see. Isn't that cool? >> That is, yeah, I completely had no idea. >> I didn't either. I saw it on LinkedIn over the weekend and I thought, I have to talk about that because it's so important when we talk about some of the trends, other trends from AnitaB.org, I talked about some of those positive trends. Overall hiring has rebounded in '22 compared to pre-pandemic levels. And we see also 51% more women being hired in '22 than '21. So the data, it's all about data, is showing us things are progressing quite slowly. But one of the biggest challenges that's still persistent is attrition. So we were talking about, Hannah, what would your advice be? How would you help a woman stay in tech? We saw that attrition last year in '22 according to AnitaB.org, more than doubled. So we're seeing women getting into the field and dropping out for various reasons. And so that's still an extent concern that we have. What do you think would motivate you to stick around if you were in a technical role? Same question for you in a minute. >> Right, you were talking about how we see an increase especially in the intern level for women. And I think if, I don't know, this is a great for a start point for pushing the momentum to start growth, pushing the needle rightwards. But I think if we can see more increase in the upper level, the women representation in the upper level too, maybe that's definitely a big goal and something we should work towards to. >> Lisa: Absolutely. >> But if there's more representation up in the CTO position, like in the managing level, I think that will definitely be a great factor to keep women in data science. >> I was looking at some trends, sorry, Hannah, forgetting what this source was, so forgive me, that was showing that there was a trend in the last few years, I think it was Fast Company, of more women in executive positions, specifically chief operating officer positions. What that hasn't translated to, what they thought it might translate to, is more women going from COO to CEO and we're not seeing that. We think of, if you ask, name a female executive that you'd recognize, everyone would probably say Sheryl Sandberg. But I was shocked to learn the other day at a Women in Tech event I was doing, that there was a survey done by this organization that showed that 78% of people couldn't identify. So to your point, we need more of them in that visible role, in the executive suite. >> Tracy: Exactly. >> And there's data that show that companies that have women, companies across industries that have women in leadership positions, executive positions I should say, are actually more profitable. So it's kind of like, duh, the data is there, it's telling you this. >> Hannah: Exactly. >> Right? >> And I think also a very important point is work culture and the work environment. And as a woman, maybe if you enter and you work two or three years, and then you have to oftentimes choose, okay, do I want family or do I want my job? And I think that's one of the major tasks that companies face to make it possible for women to combine being a mother and being a great data scientist or an executive or CEO. And I think there's still a lot to be done in this regard to make it possible for women to not have to choose for one thing or the other. And I think that's also a reason why we might see more women at the entry level, but not long-term. Because they are punished if they take a couple years off if they want to have kids. >> I think that's a question we need to ask to men too. >> Absolutely. >> How to balance work and life. 'Cause we never ask that. We just ask the woman. >> No, they just get it done, probably because there's a woman on the other end whose making it happen. >> Exactly. So yeah, another thing to think about, another thing to work towards too. >> Yeah, it's a good point you're raising that we have this conversation together and not exclusively only women, but we all have to come together and talk about how we can design companies in a way that it works for everyone. >> Yeah, and no slight to men at all. A lot of my mentors and sponsors are men. They're just people that I greatly admire who saw raw potential in me 15, 18 years ago, and just added a little water to this little weed and it started to grow. In fact, theCUBE- >> Tracy: And look at you now. >> Look at me now. And theCUBE, the guys Dave Vellante and John Furrier are two of those people that are sponsors of mine. But it needs to be diverse. It needs to be diverse and gender, it needs to include non-binary people, anybody, shouldn't matter. We should be able to collectively work together to solve big problems. Like the propaganda problem that was being discussed in the keynote this morning with respect to China, or climate change. Climate change is a huge challenge. Here, we are in California, we're getting an atmospheric river tomorrow. And Californians and rain, we're not so friendly. But we know that there's massive changes going on in the climate. Data science can help really unlock a lot of the challenges and solve some of the problems and help us understand better. So there's so much real-world implication potential that being data-driven can really lead to. And I love the fact that you guys are studying data journalism. You'll have to help me understand that even more. But we're going to going to have great conversations today, I'm so excited to be co-hosting with both of you. You're going to be inspired, you're going to learn, they're going to learn from us as well. So let's just kind of think of this as a community of men, women, everything in between to really help inspire the current generations, the future generations. And to your point, let's help women feel confident to be able to stay and raise their hand for fast-tracking their careers. >> Exactly. >> What are you guys, last minute, what are you looking forward to most for today? >> Just meeting these great women, I can't wait. >> Yeah, learning from each other. Having this conversation about how we can make data science even more equitable and hear from the great ideas that all these women have. >> Excellent, girls, we're going to have a great day. We're so glad that you're here with us on theCUBE, live at Stanford University, Women in Data Science, the eighth annual conference. I'm Lisa Martin, my two co-hosts for the day, Tracy Zhang, Hannah Freitag, you're going to be seeing a lot of us, we appreciate. Stick around, our first guest joins Hannah and me in just a minute. (ambient music)

(bright upbeat music) >> Hello and welcome to theCUBE's coverage of International Women's Day. I'm your host, John Furrier here in Palo Alto, California Studio and remoting is a great guest CUBE alumni, co-founder, technical co-founder and she's also the VP of Product at Platform9 Systems. It's a company pioneering Kubernetes infrastructure, been doing it for a long, long time. Madhura Maskasky, thanks for coming on theCUBE. Appreciate you. Thanks for coming on. >> Thank you for having me. Always exciting. >> So I always... I love interviewing you for many reasons. One, you're super smart, but also you're a co-founder, a technical co-founder, so entrepreneur, VP of product. It's hard to do startups. (John laughs) Okay, so everyone who started a company knows how hard it is. It really is and the rewarding too when you're successful. So I want to get your thoughts on what's it like being an entrepreneur, women in tech, some things you've done along the way. Let's get started. How did you get into your career in tech and what made you want to start a company? >> Yeah, so , you know, I got into tech long, long before I decided to start a company. And back when I got in tech it was very clear to me as a direction for my career that I'm never going to start a business. I was very explicit about that because my father was an entrepreneur and I'd seen how rough the journey can be. And then my brother was also and is an entrepreneur. And I think with both of them I'd seen the ups and downs and I had decided to myself and shared with my family that I really want a very well-structured sort of job at a large company type of path for my career. I think the tech path, tech was interesting to me, not because I was interested in programming, et cetera at that time, to be honest. When I picked computer science as a major for myself, it was because most of what you would consider, I guess most of the cool students were picking that as a major, let's just say that. And it sounded very interesting and cool. A lot of people were doing it and that was sort of the top, top choice for people and I decided to follow along. But I did discover after I picked computer science as my major, I remember when I started learning C++ the first time when I got exposure to it, it was just like a light bulb clicking in my head. I just absolutely loved the language, the lower level nature, the power of it, and what you can do with it, the algorithms. So I think it ended up being a really good fit for me. >> Yeah, so it clicked for you. You tried it, it was all the cool kids were doing it. I mean, I can relate, I did the same thing. Next big thing is computer science, you got to be in there, got to be smart. And then you get hooked on it. >> Yeah, exactly. >> What was the next level? Did you find any blockers in your way? Obviously male dominated, it must have been a lot of... How many females were in your class? What was the ratio at that time? >> Yeah, so the ratio was was pretty, pretty, I would say bleak when it comes to women to men. I think computer science at that time was still probably better compared to some of the other majors like mechanical engineering where I remember I had one friend, she was the single girl in an entire class of about at least 120, 130 students or so. So ratio was better for us. I think there were maybe 20, 25 girls in our class. It was a large class and maybe the number of men were maybe three X or four X number of women. So relatively better. Yeah. >> How about the job when you got into the structured big company? How did that go? >> Yeah, so, you know, I think that was a pretty smooth path I would say after, you know, you graduated from undergrad to grad school and then when I got into Oracle first and VMware, I think both companies had the ratios were still, you know, pretty off. And I think they still are to a very large extent in this industry, but I think this industry in my experience does a fantastic job of, you know, bringing everybody and kind of embracing them and treating them at the same level. That was definitely my experience. And so that makes it very easy for self-confidence, for setting up a path for yourself to thrive. So that was it. >> Okay, so you got an undergraduate degree, okay, in computer science and a master's from Stanford in databases and distributed systems. >> That's right. >> So two degrees. Was that part of your pathway or you just decided, "I want to go right into school?" Did it go right after each other? How did that work out? >> Yeah, so when I went into school, undergrad there was no special major and I didn't quite know if I liked a particular subject or set of subjects or not. Even through grad school, first year it wasn't clear to me, but I think in second year I did start realizing that in general I was a fan of backend systems. I was never a front-end person. The backend distributed systems really were of interest to me because there's a lot of complex problems to solve, and especially databases and large scale distributed systems design in the context of database systems, you know, really started becoming a topic of interest for me. And I think luckily enough at Stanford there were just fantastic professors like Mendel Rosenblum who offered operating system class there, then started VMware and later on I was able to join the company and I took his class while at school and it was one of the most fantastic classes I've ever taken. So they really had and probably I think still do a fantastic curriculum when it comes to distributor systems. And I think that probably helped stoke that interest. >> How do you talk to the younger girls out there in elementary school and through? What's the advice as they start to get into computer science, which is changing and still evolving? There's backend, there's front-end, there's AI, there's data science, there's no code, low code, there's cloud. What's your advice when they say what's the playbook? >> Yeah, so I think two things I always say, and I share this with anybody who's looking to get into computer science or engineering for that matter, right? I think one is that it's, you know, it's important to not worry about what that end specialization's going to be, whether it's AI or databases or backend or front-end. It does naturally evolve and you lend yourself to a path where you will understand, you know, which systems, which aspect you like better. But it's very critical to start with getting the fundamentals well, right? Meaning all of the key coursework around algorithm, systems design, architecture, networking, operating system. I think it is just so crucial to understand those well, even though at times you make question is this ever going to be relevant and useful to me later on in my career? It really does end up helping in ways beyond, you know, you can describe. It makes you a much better engineer. So I think that is the most important aspect of, you know, I would think any engineering stream, but definitely true for computer science. Because there's also been a trend more recently, I think, which I'm not a big fan of, of sort of limited scoped learning, which is you decide early on that you're going to be, let's say a front-end engineer, which is fine, you know. Understanding that is great, but if you... I don't think is ideal to let that limit the scope of your learning when you are an undergrad phrase or grad school. Because later on it comes back to sort of bite you in terms of you not being able to completely understand how the systems work. >> It's a systems kind of thinking. You got to have that mindset of, especially now with cloud, you got distributed systems paradigm going to the edge. You got 5G, Mobile World Congress recently happened, you got now all kinds of IOT devices out there, IP of devices at the edge. Distributed computing is only getting more distributed. >> That's right. Yeah, that's exactly right. But the other thing is also happens... That happens in computer science is that the abstraction layers keep raising things up and up and up. Where even if you're operating at a language like Java, which you know, during some of my times of programming there was a period when it was popular, it already abstracts you so far away from the underlying system. So it can become very easier if you're doing, you know, Java script or UI programming that you really have no understanding of what's happening behind the scenes. And I think that can be pretty difficult. >> Yeah. It's easy to lean in and rely too heavily on the abstractions. I want to get your thoughts on blockers. In your career, have you had situations where it's like, "Oh, you're a woman, okay seat at the table, sit on the side." Or maybe people misunderstood your role. How did you deal with that? Did you have any of that? >> Yeah. So, you know, I think... So there's something really kind of personal to me, which I like to share a few times, which I think I believe in pretty strongly. And which is for me, sort of my personal growth began at a very early phase because my dad and he passed away in 2012, but throughout the time when I was growing up, I was his special little girl. And every little thing that I did could be a simple test. You know, not very meaningful but the genuine pride and pleasure that he felt out of me getting great scores in those tests sort of et cetera, and that I could see that in him, and then I wanted to please him. And through him, I think I build that confidence in myself that I am good at things and I can do good. And I think that just set the building blocks for me for the rest of my life, right? So, I believe very strongly that, you know, yes, there are occasions of unfair treatment and et cetera, but for the most part, it comes from within. And if you are able to be a confident person who is kind of leveled and understands and believes in your capabilities, then for the most part, the right things happen around you. So, I believe very strongly in that kind of grounding and in finding a source to get that for yourself. And I think that many women suffer from the biggest challenge, which is not having enough self-confidence. And I've even, you know, with everything that I said, I've myself felt that, experienced that a few times. And then there's a methodical way to get around it. There's processes to, you know, explain to yourself that that's actually not true. That's a fake feeling. So, you know, I think that is the most important aspect for women. >> I love that. Get the confidence. Find the source for the confidence. We've also been hearing about curiosity and building, you mentioned engineering earlier, love that term. Engineering something, like building something. Curiosity, engineering, confidence. This brings me to my next question for you. What do you think the key skills and qualities are needed to succeed in a technical role? And how do you develop to maintain those skills over time? >> Yeah, so I think that it is so critical that you love that technology that you are part of. It is just so important. I mean, I remember as an example, at one point with one of my buddies before we started Platform9, one of my buddies, he's also a fantastic computer scientists from VMware and he loves video games. And so he said, "Hey, why don't we try to, you know, hack up a video game and see if we can take it somewhere?" And so, it sounded cool to me. And then so we started doing things, but you know, something I realized very quickly is that I as a person, I absolutely hate video games. I've never liked them. I don't think that's ever going to change. And so I was miserable. You know, I was trying to understand what's going on, how to build these systems, but I was not enjoying it. So, I'm glad that I decided to not pursue that. So it is just so important that you enjoy whatever aspect of technology that you decide to associate yourself with. I think that takes away 80, 90% of the work. And then I think it's important to inculcate a level of discipline that you are not going to get sort of... You're not going to get jaded or, you know, continue with happy path when doing the same things over and over again, but you're not necessarily challenging yourself, or pushing yourself, or putting yourself in uncomfortable situation. I think a combination of those typically I think works pretty well in any technical career. >> That's a great advice there. I think trying things when you're younger, or even just for play to understand whether you abandon that path is just as important as finding a good path because at least you know that skews the value in favor of the choices. Kind of like math probability. So, great call out there. So I have to ask you the next question, which is, how do you keep up to date given all the changes? You're in the middle of a world where you've seen personal change in the past 10 years from OpenStack to now. Remember those days when I first interviewed you at OpenStack, I think it was 2012 or something like that. Maybe 10 years ago. So much changed. How do you keep up with technologies in your field and resources that you rely on for personal development? >> Yeah, so I think when it comes to, you know, the field and what we are doing for example, I think one of the most important aspect and you know I am product manager and this is something I insist that all the other product managers in our team also do, is that you have to spend 50% of your time talking to prospects, customers, leads, and through those conversations they do a huge favor to you in that they make you aware of the other things that they're keeping an eye on as long as you're doing the right job of asking the right questions and not just, you know, listening in. So I think that to me ends up being one of the biggest sources where you get tidbits of information, new things, et cetera, and then you pursue. To me, that has worked to be a very effective source. And then the second is, you know, reading and keeping up with all of the publications. You guys, you know, create a lot of great material, you interview a lot of people, making sure you are watching those for us you know, and see there's a ton of activities, new projects keeps coming along every few months. So keeping up with that, listening to podcasts around those topics, all of that helps. But I think the first one I think goes in a big way in terms of being aware of what matters to your customers. >> Awesome. Let me ask you a question. What's the most rewarding aspect of your job right now? >> So, I think there are many. So I think I love... I've come to realize that I love, you know, the high that you get out of being an entrepreneur independent of, you know, there's... In terms of success and failure, there's always ups and downs as an entrepreneur, right? But there is this... There's something really alluring about being able to, you know, define, you know, path of your products and in a way that can potentially impact, you know, a number of companies that'll consume your products, employees that work with you. So that is, I think to me, always been the most satisfying path, is what kept me going. I think that is probably first and foremost. And then the projects. You know, there's always new exciting things that we are working on. Even just today, there are certain projects we are working on that I'm super excited about. So I think it's those two things. >> So now we didn't get into how you started. You said you didn't want to do a startup and you got the big company. Your dad, your brother were entrepreneurs. How did you get into it? >> Yeah, so, you know, it was kind of surprising to me as well, but I think I reached a point of VMware after spending about eight years or so where I definitely packed hold and I could have pushed myself by switching to a completely different company or a different organization within VMware. And I was trying all of those paths, interviewed at different companies, et cetera, but nothing felt different enough. And then I think I was very, very fortunate in that my co-founders, Sirish Raghuram, Roopak Parikh, you know, Bich, you've met them, they were kind of all at the same journey in their careers independently at the same time. And so we would all eat lunch together at VMware 'cause we were on the same team and then we just started brainstorming on different ideas during lunchtime. And that's kind of how... And we did that almost for a year. So by the time that the year long period went by, at the end it felt like the most logical, natural next step to leave our job and to, you know, to start off something together. But I think I wouldn't have done that had it not been for my co-founders. >> So you had comfort with the team as you knew each other at VMware, but you were kind of a little early, (laughing) you had a vision. It's kind of playing out now. How do you feel right now as the wave is hitting? Distributed computing, microservices, Kubernetes, I mean, stuff you guys did and were doing. I mean, it didn't play out exactly, but directionally you were right on the line there. How do you feel? >> Yeah. You know, I think that's kind of the challenge and the fun part with the startup journey, right? Which is you can never predict how things are going to go. When we kicked off we thought that OpenStack is going to really take over infrastructure management space and things kind of went differently, but things are going that way now with Kubernetes and distributed infrastructure. And so I think it's been interesting and in every path that you take that does end up not being successful teaches you so much more, right? So I think it's been a very interesting journey. >> Yeah, and I think the cloud, certainly AWS hit that growth right at 2013 through '17, kind of sucked all the oxygen out. But now as it reverts back to this abstraction layer essentially makes things look like private clouds, but they're just essentially DevOps. It's cloud operations, kind of the same thing. >> Yeah, absolutely. And then with the edge things are becoming way more distributed where having a single large cloud provider is becoming even less relevant in that space and having kind of the central SaaS based management model, which is what we pioneered, like you said, we were ahead of the game at that time, is becoming sort of the most obvious choice now. >> Now you look back at your role at Stanford, distributed systems, again, they have world class program there, neural networks, you name it. It's really, really awesome. As well as Cal Berkeley, there was in debates with each other, who's better? But that's a separate interview. Now you got the edge, what are some of the distributed computing challenges right now with now the distributed edge coming online, industrial 5G, data? What do you see as some of the key areas to solve from a problem statement standpoint with edge and as cloud goes on-premises to essentially data center at the edge, apps coming over the top AI enabled. What's your take on that? >> Yeah, so I think... And there's different flavors of edge and the one that we focus on is, you know, what we call thick edge, which is you have this problem of managing thousands of as we call it micro data centers, rather than managing maybe few tens or hundreds of large data centers where the problem just completely shifts on its head, right? And I think it is still an unsolved problem today where whether you are a retailer or a telecommunications vendor, et cetera, managing your footprints of tens of thousands of stores as a retailer is solved in a very archaic way today because the tool set, the traditional management tooling that's designed to manage, let's say your data centers is not quite, you know, it gets retrofitted to manage these environments and it's kind of (indistinct), you know, round hole kind of situation. So I think the top most challenges are being able to manage this large footprint of micro data centers in the most effective way, right? Where you have latency solved, you have the issue of a small footprint of resources at thousands of locations, and how do you fit in your containerized or virtualized or other workloads in the most effective way? To have that solved, you know, you need to have the security aspects around these environments. So there's a number of challenges that kind of go hand-in-hand, like what is the most effective storage which, you know, can still be deployed in that compact environment? And then cost becomes a related point. >> Costs are huge 'cause if you move data, you're going to have cost. If you move compute, it's not as much. If you have an operating system concept, is the data and state or stateless? These are huge problems. This is an operating system, don't you think? >> Yeah, yeah, absolutely. It's a distributed operating system where it's multiple layers, you know, of ways of solving that problem just in the context of data like you said having an intermediate caching layer so that you know, you still do just in time processing at those edge locations and then send some data back and that's where you can incorporate some AI or other technologies, et cetera. So, you know, just data itself is a multi-layer problem there. >> Well, it's great to have you on this program. Advice final question for you, for the folks watching technical degrees, most people are finding out in elementary school, in middle school, a lot more robotics programs, a lot more tech exposure, you know, not just in Silicon Valley, but all around, you're starting to see that. What's your advice for young girls and people who are getting either coming into the workforce re-skilled as they get enter, it's easy to enter now as they stay in and how do they stay in? What's your advice? >> Yeah, so, you know, I think it's the same goal. I have two little daughters and it's the same principle I try to follow with them, which is I want to give them as much exposure as possible without me having any predefined ideas about what you know, they should pursue. But it's I think that exposure that you need to find for yourself one way or the other, because you really never know. Like, you know, my husband landed into computer science through a very, very meandering path, and then he discovered later in his career that it's the absolute calling for him. It's something he's very good at, right? But so... You know, it's... You know, the reason why he thinks he didn't pick that path early is because he didn't quite have that exposure. So it's that exposure to various things, even things you think that you may not be interested in is the most important aspect. And then things just naturally lend themselves. >> Find your calling, superpower, strengths. Know what you don't want to do. (John chuckles) >> Yeah, exactly. >> Great advice. Thank you so much for coming on and contributing to our program for International Women's Day. Great to see you in this context. We'll see you on theCUBE. We'll talk more about Platform9 when we go KubeCon or some other time. But thank you for sharing your personal perspective and experiences for our audience. Thank you. >> Fantastic. Thanks for having me, John. Always great. >> This is theCUBE's coverage of International Women's Day, I'm John Furrier. We're talking to the leaders in the industry, from developers to the boardroom and everything in between and getting the stories out there making an impact. Thanks for watching. (bright upbeat music)

>> Hello everyone. Welcome to theCUBE's coverage of International Women's Day. I'm John Furrier, host of theCUBE been profiling the leaders in the technology world, women in technology from developers to the boardroom, everything in between. We have two great guests promoting in from Malaysia. Nancy Wang is the general manager, also CUBE alumni from AWS Data Protection, and founder and board chair of Advancing Women in Tech, awit.org. And of course Kate Watts who's the executive director of Advancing Women in Tech.org. So it's awit.org. Nancy, Kate, thanks for coming all the way across remotely from Malaysia. >> Of course, we're coming to you as fast as our internet bandwidth will allow us. And you know, I'm just thrilled today that you get to see a whole nother aspect of my life, right? Because typically we talk about AWS, and here we're talking about a topic near and dear to my heart. >> Well, Nancy, I love the fact that you're spending a lot of time taking the empowerment to go out and help the industries and helping with the advancement of women in tech. Kate, the executive director it's a 501C3, it's nonprofit, dedicating to accelerating the careers of women in groups in tech. Can you talk about the organization? >> Yes, I can. So Advancing Women in Tech was founded in 2017 in order to fix some of the pathway problems that we're seeing on the rise to leadership in the industry. And so we specifically focus on supporting mid-level women in technical roles, get into higher positions. We do that in a few different ways through mentorship programs through building technical skills and by connecting people to a supportive community. So you have your peer network and then a vertical sort of relationships to help you navigate the next steps in your career. So to date we've served about 40,000 individuals globally and we're just looking to expand our reach and impact and be able to better support women in the industry. >> Nancy, talk about the creation, the origination story. How'd this all come together? Obviously the momentum, everyone in the industry's been focused on this for a long time. Where did AWIT come from? Advancing Women in Technology, that's the acronym. Advancing Women in Technology.org, where'd it come from? What's the origination story? >> Yeah, so AWIT really originated from this desire that I had, to Kate's point around, well if you look around right and you know, don't take my word for it, right? Look at stats, look at news reports, or just frankly go on your LinkedIn and see how many women in underrepresented groups are in senior technical leadership roles right out in the companies whose names we all know. And so that was my case back in 2016. And so when I first got the idea and back then I was actually at Google, just another large tech company in the valley, right? It was about how do we get more role models, how we get more, for example, women into leadership roles so they can bring up the next generation, right? And so this is actually part of a longer speech that I'm about to give on Wednesday and part of the US State Department speaker program. In fact, that's why Kate and I are here in Malaysia right now is working with over 200 women entrepreneurs from all over in Southeast Asia, including Malaysia Philippines, Vietnam, Borneo, you know, so many countries where having more women entrepreneurs can help raise the GDP right, and that fits within our overall mission of getting more women into top leadership roles in tech. >> You know, I was talking about Teresa Carlson she came on the program as well for this year this next season we're going to do. And she mentioned the decision between the US progress and international. And she's saying as much as it's still bad numbers, it's worse than outside the United States and needs to get better. Can you comment on the global aspect? You brought that up. I think it's super important to highlight that it's just not one area, it's a global evolution. >> Absolutely, so let me start, and I'd love to actually have Kate talk about our current programs and all of the international groups that we're working with. So as Teresa aptly mentioned there is so much work to be done not just outside the US and North Americas where typically tech nonprofits will focus, but rather if you think about the one to end model, right? For example when I was doing the product market fit workshop for the US State Department I had women dialing in from rice fields, right? So let me just pause there for a moment. They were holding their cell phones up near towers near trees just so that they can get a few minutes of time with me to do a workshop and how to accelerate their business. So if you don't call that the desire to propel oneself or accelerate oneself, not sure what is, right. And so it's really that passion that drove me to spend the next week and a half here working with local entrepreneurs working with policy makers so we can take advantage and really leverage that passion that people have, right? To accelerate more business globally. And so that's why, you know Kate will be leading our contingent with the United Nations Women Group, right? That is focused on women's economic empowerment because that's super important, right? One aspect can be sure, getting more directors, you know vice presidents into companies like Google and Amazon. But another is also how do you encourage more women around the world to start businesses, right? To reach economic and freedom independence, right? To overcome some of the maybe social barriers to becoming a leader in their own country. >> Yes, and if I think about our own programs and our model of being very intentional about supporting the learning development and skills of women and members of underrepresented groups we focused very much on providing global access to a number of our programs. For instance, our product management certification on Coursera or engineering management our upcoming women founders accelerator. We provide both access that you can get from anywhere. And then also very intentional programming that connects people into the networks to be able to further their networks and what they've learned through the skills online, so. >> Yeah, and something Kate just told me recently is these courses that Kate's mentioning, right? She was instrumental in working with the American Council on Education and so that our learners can actually get up to six college credits for taking these courses on product management engineering management, on cloud product management. And most recently we had our first organic one of our very first organic testimonials was from a woman's tech bootcamp in Nigeria, right? So if you think about the worldwide impact of these upskilling courses where frankly in the US we might take for granted right around the world as I mentioned, there are women dialing in from rice patties from other, you know, for example, outside the, you know corporate buildings in order to access this content. >> Can you think about the idea of, oh sorry, go ahead. >> Go ahead, no, go ahead Kate. >> I was going to say, if you can't see it, you can't become it. And so we are very intentional about ensuring that we have we're spotlighting the expertise of women and we are broadcasting that everywhere so that anybody coming up can gain the skills and the networks to be able to succeed in this industry. >> We'll make sure we get those links so we can promote them. Obviously we feel the same way getting the word out. I think a couple things I'd like to ask you guys cause I think you hit a great point. One is the economic advantage the numbers prove that diverse teams perform better number one, that's clear. So good point there. But I want to get your thoughts on the entrepreneurial equation. You mentioned founders and startups and there's also different makeups in different countries. It's not like the big corporations sometimes it's smaller business in certain areas the different cultures have different business sizes and business types. How do you guys see that factoring in outside the United States, say the big tech companies? Okay, yeah. The easy lower the access to get in education than stay with them, in other countries is it the same or is it more diverse in terms of business? >> So what really actually got us started with the US State Department was around our work with women founders. And I love for Kate to actually share her experience working with AWS startups in that capacity. But frankly, you know, we looked at the content and the mentor programs that were providing women who wanted to be executives, you know, quickly realize a lot of those same skills such as finding customers, right? Scaling your product and building channels can also apply to women founders, not just executives. And so early supporters of our efforts from firms such as Moderna up in Seattle, Emergence Ventures, Decibel Ventures in, you know, the Bay Area and a few others that we're working with right now. Right, they believed in the mission and really helped us scale out what is now our existing platform and offerings for women founders. >> Those are great firms by the way. And they also are very founder friendly and also understand the global workforce. I mean, that's a whole nother dimension. Okay, what's your reaction to all that? >> Yes, we have been very intentional about taking the product expertise and the learnings of women and in our network, we first worked with AWS startups to support the development of the curriculum for the recent accelerator for women founders that was held last spring. And so we're able to support 25 founders and also brought in the expertise of about 20 or 30 women from Advancing Women in Tech to be able to be the lead instructors and mentors for that. And so we have really realized that with this network and this individual sort of focus on product expertise building strong teams, we can take that information and bring it to folks everywhere. And so there is very much the intentionality of allowing founders allowing individuals to take the lessons and bring it to their individual circumstances and the cultures in which they are operating. But the product sense is a skill that we can support the development of and we're proud to do so. >> That's awesome. Nancy, I want to ask you some never really talk about data storage and AWS cloud greatness and goodness, here's different and you also work full-time at AWS and you're the founder or the chairman of this great organization. How do you balance both and do you get, they're getting behind you on this, Amazon is getting behind you on this. >> Well, as I say it's always easier to negotiate on the way in. But jokes aside, I have to say the leadership has been tremendously supportive. If you think about, for example, my leaders Wayne Duso who's also been on the show multiple times, Bill Vaas who's also been on the show multiple times, you know they're both founders and also operators entrepreneurs at heart. So they understand that it is important, right? For all of us, it's really incumbent on all of us who are in positions to do so, to create a pathway for more people to be in leadership roles for more people to be successful entrepreneurs. So, no, I mean if you just looked at LinkedIn they're always uploading my vote so they reach to more audiences. And frankly they're rooting for us back home in the US while we're in Malaysia this week. >> That's awesome. And I think that's a good culture to have that empowerment and I think that's very healthy. What's next for you guys? What's on the agenda? Take us through the activities. I know that you got a ton of things happening. You got your event out there, which is why you're out there. There's a bunch of other activities. I think you guys call it the Advancing Women in Tech week. >> Yes, this week we are having a week of programming that you can check out at Advancing Women in Tech.org. That is spotlighting the expertise of a number of women in our space. So it is three days of programming Tuesday, Wednesday and Thursday if you are in the US so the seventh through the ninth, but available globally. We are also going to be in New York next week for the event at the UN and are looking to continue to support our mentorship programs and also our work supporting women founders throughout the year. >> All right. I have to ask you guys if you don't mind get a little market data so you can share with us here at theCUBE. What are you hearing this year that's different in the conversation space around the topics, the interests? Obviously I've seen massive amounts of global acceleration around conversations, more video, things like this more stories are scaling, a lot more LinkedIn activity. It just seems like it's a lot different this year. Can you guys share any kind of current trends you're seeing relative to the conversations and topics being discussed across the the community? >> Well, I think from a needle moving perspective, right? I think due to the efforts of wonderful organizations including the Q for spotlighting all of these awesome women, right? Trailblazing women and the nonprofits the government entities that we work with there's definitely more emphasis on creating access and creating pathways. So that's probably one thing that you're seeing is more women, more investors posting about their activities. Number two, from a global trend perspective, right? The rise of women in security. I noticed that on your agenda today, you had Lena Smart who's a good friend of mine chief information security officer at MongoDB, right? She and I are actually quite involved in helping founders especially early stage founders in the security space. And so globally from a pure technical perspective, right? There's right more increasing regulations around data privacy, data sovereignty, right? For example, India's in a few weeks about to get their first data protection regulation there locally. So all of that is giving rise to yet another wave of opportunity and we want women founders uniquely positioned to take advantage of that opportunity. >> I love it. Kate, reaction to that? I mean founders, more pathways it sounds like a neural network, it sounds like AI enabled. >> Yes, and speaking of AI, with the rise of that we are also hearing from many community members the importance of continuing to build their skills upskill learn to be able to keep up with the latest trends. There's a lot of people wondering what does this mean for my own career? And so they're turning to organizations like Advancing Women in Tech to find communities to both learn the latest information, but also build their networks so that they are able to move forward regardless of what the industry does. >> I love the work you guys are doing. It's so impressive. I think the economic angle is new it's more amplified this year. It's always kind of been there and continues to be. What do you guys hope for by next year this time what do you hope to see different from a needle moving perspective, to use your word Nancy, for next year? What's the visual output in your mind? >> I want to see real effort made towards 50-50 representation in all tech leadership roles. And I'd like to see that happen by 2050. >> Kate, anything on your end? >> I love that. I'm going to go a little bit more touchy-feely. I want everybody in our space to understand that the skills that they build and that the networks they have carry with them regardless of wherever they go. And so to be able to really lean in and learn and continue to develop the career that you want to have. So whether that be at a large organization or within your own business, that you've got the potential to move forward on that within you. >> Nancy, Kate, thank you so much for your contribution. I'll give you the final word. Put a plug in for the organization. What are you guys looking for? Any kind of PSA you want to share with the folks watching? >> Absolutely, so if you're in a position to be a mentor, join as a mentor, right? Help elevate and accelerate the next generation of women leaders. If you're an investor help us invest in more women started companies, right? Women founded startups and lastly, if you are women looking to accelerate your career, come join our community. We have resources, we have mentors and who we have investors who are willing to come in on the ground floor and help you accelerate your business. >> Great work. Thank you so much for participating in our International Women's Day 23 program and we'd look to keep this going quarterly. We'll see you next year, next time. Thanks for coming on. Appreciate it. >> Thanks so much John. >> Thank you. >> Okay, women leaders here. >> Nancy: Thanks for having us >> All over the world, coming together for a great celebration but really highlighting the accomplishments, the pathways the investment, the mentoring, everything in between. It's theCUBE. Bring as much as we can. I'm John Furrier, your host. Thanks for watching.

(upbeat intro music) >> Hello everyone. Welcome to theCUBE's coverage of International Women's Day. I'm your host, John Furrier, here in Palo Alto, California. Got a special remote guest coming in. Teresa Carlson, President and Chief Commercial Officer at Flexport, theCUBE alumni, one of the first, let me go back to 2013, Teresa, former AWS. Great to see you. Thanks for coming on. >> Oh my gosh, almost 10 years. That is unbelievable. It's hard to believe so many years of theCUBE. I love it. >> It's been such a great honor to interview you and follow your career. You've had quite the impressive run, executive level woman in tech. You've done such an amazing job, not only in your career, but also helping other women. So I want to give you props to that before we get started. Thank you. >> Thank you, John. I, it's my, it's been my honor and privilege. >> Let's talk about Flexport. Tell us about your new role there and what it's all about. >> Well, I love it. I'm back working with another Amazonian, Dave Clark, who is our CEO of Flexport, and we are about 3,000 people strong globally in over 90 countries. We actually even have, we're represented in over 160 cities and with local governments and places around the world, which I think is super exciting. We have over 100 network partners and growing, and we are about empowering the global supply chain and trade and doing it in a very disruptive way with the use of platform technology that allows our customers to really have visibility and insight to what's going on. And it's a lot of fun. I'm learning new things, but there's a lot of technology in this as well, so I feel right at home. >> You quite have a knack from mastering growth, technology, and building out companies. So congratulations, and scaling them up too with the systems and processes. So I want to get into that. Let's get into your personal background. Then I want to get into the work you've done and are doing for empowering women in tech. What was your journey about, how did it all start? Like, I know you had a, you know, bumped into it, you went Microsoft, AWS. Take us through your career, how you got into tech, how it all happened. >> Well, I do like to give a shout out, John, to my roots and heritage, which was a speech and language pathologist. So I did start out in healthcare right out of, you know, university. I had an undergraduate and a master's degree. And I do tell everyone now, looking back at my career, I think it was super helpful for me because I learned a lot about human communication, and it has done me very well over the years to really try to understand what environments I'm in and what kind of individuals around the world culturally. So I'm really blessed that I had that opportunity to work in healthcare, and by the way, a shout out to all of our healthcare workers that has helped us get through almost three years of COVID and flu and neurovirus and everything else. So started out there and then kind of almost accidentally got into technology. My first small company I worked for was a company called Keyfile Corporation, which did workflow and document management out of Nashua, New Hampshire. And they were a Microsoft goal partner. And that is actually how I got into big tech world. We ran on exchange, for everybody who knows that term exchange, and we were a large small partner, but large in the world of exchange. And those were the days when you would, the late nineties, you would go and be in the same room with Bill Gates and Steve Ballmer. And I really fell in love with Microsoft back then. I thought to myself, wow, if I could work for a big tech company, I got to hear Bill on stage about saving, he would talk about saving the world. And guess what my next step was? I actually got a job at Microsoft, took a pay cut and a job downgrade. I tell this story all the time. Took like three downgrades in my role. I had been a SVP and went to a manager, and it's one of the best moves I ever made. And I shared that because I really didn't know the world of big tech, and I had to start from the ground up and relearn it. I did that, I just really loved that job. I was at Microsoft from 2000 to 2010, where I eventually ran all of the U.S. federal government business, which was a multi-billion dollar business. And then I had the great privilege of meeting an amazing man, Andy Jassy, who I thought was just unbelievable in his insights and knowledge and openness to understanding new markets. And we talked about government and how government needed the same great technology as every startup. And that led to me going to work for Andy in 2010 and starting up our worldwide public sector business. And I pinch myself some days because we went from two people, no offices, to the time I left we had over 10,000 people, billions in revenue, and 172 countries and had done really amazing work. I think changing the way public sector and government globally really thought about their use of technology and Cloud computing in general. And that kind of has been my career. You know, I was there till 2020, 21 and then did a small stint at Splunk, a small stint back at Microsoft doing a couple projects for Microsoft with CEO, Satya Nadella, who is also an another amazing CEO and leader. And then Dave called me, and I'm at Flexport, so I couldn't be more honored, John. I've just had such an amazing career working with amazing individuals. >> Yeah, I got to say the Amazon One well-documented, certainly by theCUBE and our coverage. We watched you rise and scale that thing. And like I said at a time, this will when we look back as a historic run because of the build out. I mean as a zero to massive billions at a historic time where government was transforming, I would say Microsoft had a good run there with Fed, but it was already established stuff. Federal business was like, you know, blocking and tackling. The Amazon was pure build out. So I have to ask you, what was your big learnings? Because one, you're a Seattle big tech company kind of entrepreneurial in the sense of you got, here's some working capital seed finance and go build that thing, and you're in DC and you're a woman. What did you learn? >> I learned that you really have to have a lot of grit. You, my mom and dad, these are kind of more southern roots words, but stick with itness, you know. you can't give up and no's not in your vocabulary. I found no is just another way to get to yes. That you have to figure out what are all the questions people are going to ask you. I learned to be very patient, and I think one of the things John, for us was our secret sauce was we said to ourselves, if we're going to do something super transformative and truly disruptive, like Cloud computing, which the government really had not utilized, we had to be patient. We had to answer all their questions, and we could not judge in any way what they were thinking because if we couldn't answer all those questions and prove out the capabilities of Cloud computing, we were not going to accomplish our goals. And I do give so much credit to all my colleagues there from everybody like Steve Schmidt who was there, who's still there, who's the CISO, and Charlie Bell and Peter DeSantis and the entire team there that just really helped build that business out. Without them, you know, we would've just, it was a team effort. And I think that's the thing I loved about it was it was not just sales, it was product, it was development, it was data center operations, it was legal, finance. Everybody really worked as a team and we were on board that we had to make a lot of changes in the government relations team. We had to go into Capitol Hill. We had to talk to them about the changes that were required and really get them to understand why Cloud computing could be such a transformative game changer for the way government operates globally. >> Well, I think the whole world and the tech world can appreciate your work and thank you later because you broke down those walls asking those questions. So great stuff. Now I got to say, you're in kind of a similar role at Flexport. Again, transformative supply chain, not new. Computing wasn't new when before Cloud came. Supply chain, not a new concept, is undergoing radical change and transformation. Online, software supply chain, hardware supply chain, supply chain in general, shipping. This is a big part of our economy and how life is working. Similar kind of thing going on, build out, growth, scale. >> It is, it's very much like that, John, I would say, it's, it's kind of a, the model with freight forwarding and supply chain is fairly, it's not as, there's a lot of technology utilized in this global supply chain world, but it's not integrated. You don't have a common operating picture of what you're doing in your global supply chain. You don't have easy access to the information and visibility. And that's really, you know, I was at a conference last week in LA, and it was, the themes were so similar about transparency, access to data and information, being able to act quickly, drive change, know what was happening. I was like, wow, this sounds familiar. Data, AI, machine learning, visibility, common operating picture. So it is very much the same kind of themes that you heard even with government. I do believe it's an industry that is going through transformation and Flexport has been a group that's come in and said, look, we have this amazing idea, number one to give access to everyone. We want every small business to every large business to every government around the world to be able to trade their goods, think about supply chain logistics in a very different way with information they need and want at their fingertips. So that's kind of thing one, but to apply that technology in a way that's very usable across all systems from an integration perspective. So it's kind of exciting. I used to tell this story years ago, John, and I don't think Michael Dell would mind that I tell this story. One of our first customers when I was at Keyfile Corporation was we did workflow and document management, and Dell was one of our customers. And I remember going out to visit them, and they had runners and they would run around, you know, they would run around the floor and do their orders, right, to get all those computers out the door. And when I think of global trade, in my mind I still see runners, you know, running around and I think that's moved to a very digital, right, world that all this stuff, you don't need people doing this. You have machines doing this now, and you have access to the information, and you know, we still have issues resulting from COVID where we have either an under-abundance or an over-abundance of our supply chain. We still have clogs in our shipping, in the shipping yards around the world. So we, and the ports, so we need to also, we still have some clearing to do. And that's the reason technology is important and will continue to be very important in this world of global trade. >> Yeah, great, great impact for change. I got to ask you about Flexport's inclusion, diversity, and equity programs. What do you got going on there? That's been a big conversation in the industry around keeping a focus on not making one way more than the other, but clearly every company, if they don't have a strong program, will be at a disadvantage. That's well reported by McKinsey and other top consultants, diverse workforces, inclusive, equitable, all perform better. What's Flexport's strategy and how are you guys supporting that in the workplace? >> Well, let me just start by saying really at the core of who I am, since the day I've started understanding that as an individual and a female leader, that I could have an impact. That the words I used, the actions I took, the information that I pulled together and had knowledge of could be meaningful. And I think each and every one of us is responsible to do what we can to make our workplace and the world a more diverse and inclusive place to live and work. And I've always enjoyed kind of the thought that, that I could help empower women around the world in the tech industry. Now I'm hoping to do my little part, John, in that in the supply chain and global trade business. And I would tell you at Flexport we have some amazing women. I'm so excited to get to know all. I've not been there that long yet, but I'm getting to know we have some, we have a very diverse leadership team between men and women at Dave's level. I have some unbelievable women on my team directly that I'm getting to know more, and I'm so impressed with what they're doing. And this is a very, you know, while this industry is different than the world I live in day to day, it's also has a lot of common themes to it. So, you know, for us, we're trying to approach every day by saying, let's make sure both our interviewing cycles, the jobs we feel, how we recruit people, how we put people out there on the platforms, that we have diversity and inclusion and all of that every day. And I can tell you from the top, from Dave and all of our leaders, we just had an offsite and we had a big conversation about this is something. It's a drum beat that we have to think about and live by every day and really check ourselves on a regular basis. But I do think there's so much more room for women in the world to do great things. And one of the, one of the areas, as you know very well, we lost a lot of women during COVID, who just left the workforce again. So we kind of went back unfortunately. So we have to now move forward and make sure that we are giving women the opportunity to have great jobs, have the flexibility they need as they build a family, and have a workplace environment that is trusted for them to come into every day. >> There's now clear visibility, at least in today's world, not withstanding some of the setbacks from COVID, that a young girl can look out in a company and see a path from entry level to the boardroom. That's a big change. A lot than even going back 10, 15, 20 years ago. What's your advice to the folks out there that are paying it forward? You see a lot of executive leaderships have a seat at the table. The board still underrepresented by most numbers, but at least you have now kind of this solidarity at the top, but a lot of people doing a lot more now than I've seen at the next levels down. So now you have this leveled approach. Is that something that you're seeing more of? And credit compare and contrast that to 20 years ago when you were, you know, rising through the ranks? What's different? >> Well, one of the main things, and I honestly do not think about it too much, but there were really no women. There were none. When I showed up in the meetings, I literally, it was me or not me at the table, but at the seat behind the table. The women just weren't in the room, and there were so many more barriers that we had to push through, and that has changed a lot. I mean globally that has changed a lot in the U.S. You know, if you look at just our U.S. House of Representatives and our U.S. Senate, we now have the increasing number of women. Even at leadership levels, you're seeing that change. You have a lot more women on boards than we ever thought we would ever represent. While we are not there, more female CEOs that I get an opportunity to see and talk to. Women starting companies, they do not see the barriers. And I will share, John, globally in the U.S. one of the things that I still see that we have that many other countries don't have, which I'm very proud of, women in the U.S. have a spirit about them that they just don't see the barriers in the same way. They believe that they can accomplish anything. I have two sons, I don't have daughters. I have nieces, and I'm hoping someday to have granddaughters. But I know that a lot of my friends who have granddaughters today talk about the boldness, the fortitude, that they believe that there's nothing they can't accomplish. And I think that's what what we have to instill in every little girl out there, that they can accomplish anything they want to. The world is theirs, and we need to not just do that in the U.S., but around the world. And it was always the thing that struck me when I did all my travels at AWS and now with Flexport, I'm traveling again quite a bit, is just the differences you see in the cultures around the world. And I remember even in the Middle East, how I started seeing it change. You've heard me talk a lot on this program about the fact in both Saudi and Bahrain, over 60% of the tech workers were females and most of them held the the hardest jobs, the security, the architecture, the engineering. But many of them did not hold leadership roles. And that is what we've got to change too. To your point, the middle, we want it to get bigger, but the top, we need to get bigger. We need to make sure women globally have opportunities to hold the most precious leadership roles and demonstrate their capabilities at the very top. But that's changed. And I would say the biggest difference is when we show up, we're actually evaluated properly for those kind of roles. We have a ways to go. But again, that part is really changing. >> Can you share, Teresa, first of all, that's great work you've done and I wan to give you props of that as well and all the work you do. I know you champion a lot of, you know, causes in in this area. One question that comes up a lot, I would love to get your opinion 'cause I think you can contribute heavily here is mentoring and sponsorship is huge, comes up all the time. What advice would you share to folks out there who were, I won't say apprehensive, but maybe nervous about how to do the networking and sponsorship and mentoring? It's not just mentoring, it's sponsorship too. What's your best practice? What advice would you give for the best way to handle that? >> Well yeah, and for the women out there, I would say on the mentorship side, I still see mentorship. Like, I don't think you can ever stop having mentorship. And I like to look at my mentors in different parts of my life because if you want to be a well-rounded person, you may have parts of your life every day that you think I'm doing a great job here and I definitely would like to do better there. Whether it's your spiritual life, your physical life, your work life, you know, your leisure life. But I mean there's, and there's parts of my leadership world that I still seek advice from as I try to do new things even in this world. And I tried some new things in between roles. I went out and asked the people that I respected the most. So I just would say for sure have different mentorships and don't be afraid to have that diversity. But if you have mentorships, the second important thing is show up with a real agenda and questions. Don't waste people's time. I'm very sensitive today. If you're, if you want a mentor, you show up and you use your time super effectively and be prepared for that. Sponsorship is a very different thing. And I don't believe we actually do that still in companies. We worked, thank goodness for my great HR team. When I was at AWS, we worked on a few sponsorship programs where for diversity in general, where we would nominate individuals in the company that we felt that weren't, that had a lot of opportunity for growth, but they just weren't getting a seat at the table. And we brought 'em to the table. And we actually kind of had a Chatham House rules where when they came into the meetings, they had a sponsor, not a mentor. They had a sponsor that was with them the full 18 months of this program. We would bring 'em into executive meetings. They would read docs, they could ask questions. We wanted them to be able to open up and ask crazy questions without, you know, feeling wow, I just couldn't answer this question in a normal environment or setting. And then we tried to make sure once they got through the program that we found jobs and support and other special projects that they could go do. But they still had that sponsor and that group of individuals that they'd gone through the program with, John, that they could keep going back to. And I remember sitting there and they asked me what I wanted to get out of the program, and I said two things. I want you to leave this program and say to yourself, I would've never had that experience if I hadn't gone through this program. I learned so much in 18 months. It would probably taken me five years to learn. And that it helped them in their career. The second thing I told them is I wanted them to go out and recruit individuals that look like them. I said, we need diversity, and unless you all feel that we are in an inclusive environment sponsoring all types of individuals to be part of this company, we're not going to get the job done. And they said, okay. And you know, but it was really one, it was very much about them. That we took a group of individuals that had high potential and a very diverse with diverse backgrounds, held 'em up, taught 'em things that gave them access. And two, selfishly I said, I want more of you in my business. Please help me. And I think those kind of things are helpful, and you have to be thoughtful about these kind of programs. And to me that's more sponsorship. I still have people reach out to me from years ago, you know, Microsoft saying, you were so good with me, can you give me a reference now? Can you talk to me about what I should be doing? And I try to, I'm not pray 100%, some things pray fall through the cracks, but I always try to make the time to talk to those individuals because for me, I am where I am today because I got some of the best advice from people like Don Byrne and Linda Zecker and Andy Jassy, who were very honest and upfront with me about my career. >> Awesome. Well, you got a passion for empowering women in tech, paying it forward, but you're quite accomplished and that's why we're so glad to have you on the program here. President and Chief Commercial Officer at Flexport. Obviously storied career and your other jobs, specifically Amazon I think, is historic in my mind. This next chapter looks like it's looking good right now. Final question for you, for the few minutes you have left. Tell us what you're up to at Flexport. What's your goals as President, Chief Commercial Officer? What are you trying to accomplish? Share a little bit, what's on your mind with your current job? >> Well, you kind of said it earlier. I think if I look at my own superpowers, I love customers, I love partners. I get my energy, John, from those interactions. So one is to come in and really help us build even a better world class enterprise global sales and marketing team. Really listen to our customers, think about how we interact with them, build the best executive programs we can, think about new ways that we can offer services to them and create new services. One of my favorite things about my career is I think if you're a business leader, it's your job to come back around and tell your product group and your services org what you're hearing from customers. That's how you can be so much more impactful, that you listen, you learn, and you deliver. So that's one big job. The second job for me, which I am so excited about, is that I have an amazing group called flexport.org under me. And flexport.org is doing amazing things around the world to help those in need. We just announced this new funding program for Tech for Refugees, which brings assistance to millions of people in Ukraine, Pakistan, the horn of Africa, and those who are affected by earthquakes. We just took supplies into Turkey and Syria, and Flexport, recently in fact, just did sent three air shipments to Turkey and Syria for these. And I think we did over a hundred trekking shipments to get earthquake relief. And as you can imagine, it was not easy to get into Syria. But you know, we're very active in the Ukraine, and we are, our goal for flexport.org, John, is to continue to work with our commercial customers and team up with them when they're trying to get supplies in to do that in a very cost effective, easy way, as quickly as we can. So that not-for-profit side of me that I'm so, I'm so happy. And you know, Ryan Peterson, who was our founder, this was his brainchild, and he's really taken this to the next level. So I'm honored to be able to pick that up and look for new ways to have impact around the world. And you know, I've always found that I think if you do things right with a company, you can have a beautiful combination of commercial-ity and giving. And I think Flexport does it in such an amazing and unique way. >> Well, the impact that they have with their system and their technology with logistics and shipping and supply chain is a channel for societal change. And I think that's a huge gift that you have that under your purview. So looking forward to finding out more about flexport.org. I can only imagine all the exciting things around sustainability, and we just had Mobile World Congress for Big Cube Broadcast, 5Gs right around the corner. I'm sure that's going to have a huge impact to your business. >> Well, for sure. And just on gas emissions, that's another thing that we are tracking gas, greenhouse gas emissions. And in fact we've already reduced more than 300,000 tons and supported over 600 organizations doing that. So that's a thing we're also trying to make sure that we're being climate aware and ensuring that we are doing the best job we can at that as well. And that was another thing I was honored to be able to do when we were at AWS, is to really cut out greenhouse gas emissions and really go global with our climate initiatives. >> Well Teresa, it's great to have you on. Security, data, 5G, sustainability, business transformation, AI all coming together to change the game. You're in another hot seat, hot roll, big wave. >> Well, John, it's an honor, and just thank you again for doing this and having women on and really representing us in a big way as we celebrate International Women's Day. >> I really appreciate it, it's super important. And these videos have impact, so we're going to do a lot more. And I appreciate your leadership to the industry and thank you so much for taking the time to contribute to our effort. Thank you, Teresa. >> Thank you. Thanks everybody. >> Teresa Carlson, the President and Chief Commercial Officer of Flexport. I'm John Furrier, host of theCUBE. This is International Women's Day broadcast. Thanks for watching. (upbeat outro music)