(upbeat music) >> Welcome back. Day two of theCUBE's coverage of re:MARS in Las Vegas. Amazon re:MARS, it's part of the Re Series they call it at Amazon. re:Invent is their big show, re:Inforce is a security show, re:MARS is the new emerging machine learning automation, robotics, and space. The confluence of machine learning powering a new industrial age and inflection point. I'm John Furrier, host of theCUBE. We're here to break it down for another wall to wall coverage. We've got a great guest here, CUBE alumni from our AWS startup showcase, Krishna Gade, founder and CEO of fiddler.ai. Welcome back to theCUBE. Good to see you. >> Great to see you, John. >> In person. We did the remote one before. >> Absolutely, great to be here, and I always love to be part of these interviews and love to talk more about what we're doing. >> Well, you guys have a lot of good street cred, a lot of good word of mouth around the quality of your product, the work you're doing. I know a lot of folks that I admire and trust in the AI machine learning area say great things about you. A lot going on, you guys are growing companies. So you're kind of like a startup on a rocket ship, getting ready to go, pun intended here at the space event. What's going on with you guys? You're here. Machine learning is the centerpiece of it. Swami gave the keynote here at day two and it really is an inflection point. Machine learning is now ready, it's scaling, and some of the examples that they were showing with the workloads and the data sets that they're tapping into, you know, you've got CodeWhisperer, which they announced, you've got trust and bias now being addressed, we're hitting a level, a new level in ML, ML operations, ML modeling, ML workloads for developers. >> Yep, yep, absolutely. You know, I think machine learning now has become an operational software, right? Like you know a lot of companies are investing millions and billions of dollars and creating teams to operationalize machine learning based products. And that's the exciting part. I think the thing that that is very exciting for us is like we are helping those teams to observe how those machine learning applications are working so that they can build trust into it. Because I believe as Swami was alluding to this today, without actually building trust into AI, it's really hard to actually have your business users use it in their business workflows. And that's where we are excited about bringing their trust and visibility factor into machine learning. >> You know, a lot of us all know what you guys are doing here in the ecosystem of AWS. And now extending here, take a minute to explain what Fiddler is doing for the folks that are in the space, that are in discovery mode, trying to understand who's got what, because like Swami said on stage, it's a full-time job to keep up on all the machine learning activities and tool sets and platforms. Take a minute to explain what Fiddler's doing, then we can get into some, some good questions. >> Absolutely. As the enterprise is taking on operationalization of machine learning models, one of the key problems that they run into is lack of visibility into how those models perform. You know, for example, let's say if I'm a bank, I'm trying to introduce credit risk scoring models using machine learning. You know, how do I know when my model is rejecting someone's loan? You know, when my model is accepting someone's loan? And why is it doing it? And I think this is basically what makes machine learning a complex thing to implement and operationalize. Without this visibility, you cannot build trust and actually use it in your business. With Fiddler, what we provide is we actually open up this black box and we help our customers to really understand how those models work. You know, for example, how is my model doing? Is it accurately working or not? You know, why is it actually rejecting someone's loan application? We provide these both fine grain as well as coarse grain insights. So our customers can actually deploy machine learning in a safe and trustworthy manner. >> Who is your customer? Who you're targeting? What persona is it, the data engineer, is it data science, is it the CSO, is it all the above? >> Yeah, our customer is the data scientist and the machine learning engineer, right? And we usually talk to teams that have a few models running in production, that's basically our sweet spot, where they're trying to look for a single pane of glass to see like what models are running in their production, how they're performing, how they're affecting their business metrics. So we typically engage with like head of data science or head of machine learning that has a few machine learning engineers and data scientists. >> Okay, so those people that are watching, you're into this, you can go check it out. It's good to learn. I want to get your thoughts on some trends that I see emerging, and I want to get your reaction to those. Number one, we're seeing the cloud scale now and integration a big part of things. So the time to value was brought up on stage today, Swami kind of mentioned time to value, showed some benchmark where they got four hours, some other teams were doing eight weeks. Where are we on the progression of value, time to value, and on the scale side. Can you scope that for me? >> I mean, it depends, right? You know, depending upon the company. So for example, when we work with banks, for them to time to operationalize a model can take months actually, because of all the regulatory procedures that they have to go through. You know, they have to get the models reviewed by model validators, model risk management teams, and then they audit those models, they have to then ship those models and constantly monitor them. So it's a very long process for them. And even for non-regulated sectors, if you do not have the right tools and processes in place, operationalizing machine learning models can take a long time. You know, with tools like Fiddler, what we are enabling is we are basically compressing that life cycle. We are helping them automate like model monitoring and explainability so that they can actually ship models more faster. Like you get like velocity in terms of shipping models. For example, one of the growing fintech companies that started with us last year started with six models in production, now they're running about 36 models in production. So it's within a year, they were able to like grow like 10x. So that is basically what we are trying to do. >> At other things, we at re:MARS, so first of all, you got a great product and a lot of markets that grow onto, but here you got space. I mean, anyone who's coming out of college or university PhD program, and if they're into aero, they're going to be here, right? This is where they are. Now you have a new core companies with machine learning, not just the engineering that you see in the space or aerospace area, you have a new engineering. Now I go back to the old days where my parents, there was Fortran, you used Fortran was Lingua Franca to manage the equipment. Little throwback to the old school. But now machine learning is companion, first class citizen, to the hardware. And in fact, and some will say more important. >> Yep, I mean, machine learning model is the new software artifact. It is going into production in a big way. And I think it has two different things that compare to traditional software. Number one, unlike traditional software, it's a black box. You cannot read up a machine learning model score and see why it's making those predictions. Number two, it's a stochastic entity. What that means is it's predictive power can wane over time. So it needs to be constantly monitored and then constantly refreshed so that it's actually working in tech. So those are the two main things you need to take care. And if you can do that, then machine learning can give you a huge amount of ROI. >> There is some practitioner kind of like craft to it. >> Correct. >> As you said, you got to know when to refresh, what data sets to bring in, which to stay away from, certainly when you get to the bias, but I'll get to that in a second. My next question is really along the lines of software. So if you believe that open source will dominate the software business, which I do, I mean, most people won't argue. I think you would agree with that, right? Open source is driving everything. If everything's open source, where's the differentiation coming from? So if I'm a startup entrepreneur or I'm a project manager working on the next Artemis mission, I got to open source. Okay, there's definitely security issues here. I don't want to talk about shift left right now, but like, okay, open source is everything. Where's the differentiation, where do I have the proprietary edge? >> It's a great question, right? So I used to work in tech companies before Fiddler. You know, when I used to work at Facebook, we would build everything in house. We would not even use a lot of open source software. So there are companies like that that build everything in house. And then I also worked at companies like Twitter and Pinterest, which are actually used a lot of open source, right? So now, like the thing is, it depends on the maturity of the organization. So if you're a Facebook or a Google, you can build a lot of things in house. Then if you're like a modern tech company, you would probably leverage open source, but there are lots of other companies in the world that still don't have the talent pool to actually build, take things from open source and productionize it. And that's where the opportunity for startups comes in so that we can commercialize these things, create a great enterprise experience, so actually operationalize things for them so that they don't have to do it in house for them. And that's the advantage working with startups. >> I don't want to get all operating systems with you on theory here on the stage here, but I will have to ask you the next question, which I totally agree with you, by the way, that's the way to go. There's not a lot of people out there that are peaked. And that's just statistical and it'll get better. Data engineering is really narrow. That is like the SRE of data. That's a new role emerging. Okay, all the things are happening. So if open source is there, integration is a huge deal. And you start to see the rise of a lot of MSPs, managed service providers. I run Kubernetes clusters, I do this, that, and the other thing. So what's your reaction to the growth of the integration side of the business and this role of new services coming from third parties? >> Yeah, absolutely. I think one of the big challenges for a chief data officer or someone like a CTO is how do they devise this infrastructure architecture and with components, either homegrown components or open source components or some vendor components, and how do they integrate? You know, when I used to run data engineering at Pinterest, we had to devise a data architecture combining all of these things and create something that actually flows very nicely, right? >> If you didn't do it right, it would break. >> Absolutely. And this is why it's important for us, like at Fiddler, to really make sure that Fiddler can integrate to all varies of ML platforms. Today, a lot of our customers use machine learning, build machine learning models on SageMaker. So Fiddler nicely integrate with SageMaker so that data, they get a seamless experience to monitor their models. >> Yeah, I mean, this might not be the right words for it, but I think data engineering as a service is really what I see you guys doing, as well other things, you're providing all that. >> And ML engineering as a service. >> ML engineering as a- Well it's hard. I mean, it's like the hard stuff. >> Yeah, yeah. >> Hear, hear. But that has to enable. So you as a business entrepreneur, you have to create a multiple of value proposition to your customers. What's your vision on that? What is that value? It has to be a multiple, at least 5 to 10. >> I mean, the value is simple, right? You know, if you have to operationize machine learning, you need visibility into how these things work. You know, if you're CTO or like chief data officer is asking how is my model working and how is it affecting my business? You need to be able to show them a dashboard, how it's working, right? And so like a data scientist today struggles to do this. They have to manually generate a report, manually do this analysis. What Fiddler is doing them is basically reducing their work so that they can automate these things and they can still focus on the core aspect of model building and data preparation and this boring aspect of monitoring the model and creating reports around the models is automated for them. >> Yeah, you guys got a great business. I think it's a lot of great future there and it's only going to get bigger. Again, the TAM's going to expand as the growth rising tide comes in. I want to ask you on while we're on that topic of rising tides, Dave Malik and I, since re:Invent last year have been kind of kicked down around this term that we made up called supercloud. And supercloud was a word that came out of these clouds that were not Amazon hyperscalers. So Snowflake, Buildman Sachs, Capital One, you name it, they're building massive proprietary value on top of the CapEx of Amazon. Jerry Chen at Greylock calls it castles in the cloud. You can create these moats. >> Yeah, right. >> So this is a phenomenon, right? And you land on one, and then you go to the others. So the strategies, everyone goes to Amazon first, and then hits Azure and GCP. That then creates this kind of multicloud so, okay, so super cloud's kind of happening, it's a thing. Charles Fitzgerald will disagree, he's a platformer, he says he's against the term. I get why, but he's off base a little. We can't wait to debate him on that. So superclouds are happening, but now what do I do about multicloud, because now I understand multicloud, I have this on that cloud, integrating across clouds is a very difficult thing. >> Krishna: Right, right, right. >> If I'm Snowflake or whatever, hey, I'll go to Azure, more TAM expansion, more market. But are people actually working together? Are we there yet? Where it's like, okay, I'm going to re-operationalize this code base over here. >> I mean, the reality of it, enterprise wants optionality, right? I think they don't want to be locked in into one particular cloud vendor on one particular software. And therefore you actually have in a situation where you have a multicloud scenario where they want to have some workloads in Amazon, some workloads in Azure. And this is an opportunity for startups like us because we are cloud agnostic. We can monitor models wherever you have. So this is where a lot of our customers, they have some of their models are running in their data centers and some of their models running in Amazon. And so we can provide a universal single pan of glass, right? So we can basically connect all of those data and actually showcase. I think this is an opportunity for startups to combine the data streams come from various different clouds and give them a single pain of experience. That way, the sort of the where is your data, where are my models running, which cloud are there, is all abstracted out from the customer. Because at the end of the day, enterprises will want optionality. And we are in this multicloud. >> Yeah, I mean, this reminds me of the interoperability days back when I was growing into the business. Everything was interoperability and OSI and the standards came out, but what's your opinion on openness, okay? There's a kneejerk reaction right now in the market to go silo on your data for governance or whatever reasons, but yet machine learning gurus and experts will say, "Hey, you want to horizon horizontal scalability and have the best machine learning models, you've got to have access to data and fast in real time or near real time." And the antithesis is siloing. >> Krishna: Right, right, right. >> So what's the solution? Customers control the data plane and have a control plane that's... What do customers do? It's a big challenge. >> Yeah, absolutely. I think there are multiple different architectures of ML, right, you know? We've seen like where vendors like us used to deploy completely on-prem, right? And they still do it, we still do it in some customers. And then you had this managed cloud experience where you just abstract out the entire operations from the customer. And then now you have this hybrid experience where you split the control plane and data plane. So you preserve the privacy of the customer from the data perspective, but you still control the infrastructure, right? I don't think there's a right answer. It depends on the product that you're trying to solve. You know, Databricks is able to solve this control plane, data plane split really well. I've seen some other tools that have not done this really well. So I think it all depends upon- >> What about Snowflake? I think they a- >> Sorry, correct. They have a managed cloud service, right? So predominantly that's their business. So I think it all depends on what is your go to market? You know, which customers you're talking to? You know, what's your product architecture look like? You know, from Fiddler's perspective today, we actually have chosen, we either go completely on-prem or we basically provide a managed cloud service and that's actually simpler for us instead of splitting- >> John: So it's customer choice. >> Exactly. >> That's your position. >> Exactly. >> Whoever you want to use Fiddler, go on-prem, no problem, or cloud. >> Correct, or cloud, yeah. >> You'll deploy and you'll work across whatever observability space you want to. >> That's right, that's right. >> Okay, yeah. So that's the big challenge, all right. What's the big observation from your standpoint? You've been on the hyperscaler side, your journey, Facebook, Pinterest, so back then you built everything, because no one else had software for you, but now everybody wants to be a hyperscaler, but there's a huge CapEx advantage. What should someone do? If you're a big enterprise, obviously I could be a big insurance, I could be financial services, oil and gas, whatever vertical, I want a supercloud, what do I do? >> I think like the biggest advantage enterprise today have is they have a plethora of tools. You know, when I used to work on machine learning way back in Microsoft on Bing Search, we had to build everything. You know, from like training platforms, deployment platforms, experimentation platforms. You know, how do we monitor those models? You know, everything has to be homegrown, right? A lot of open source also did not exist at the time. Today, the enterprise has this advantage, they're sitting on this gold mine of tools. You know, obviously there's probably a little bit of tool fatigue as well. You know, which tools to select? >> There's plenty of tools available. >> Exactly, right? And then there's like services available for you. So now you need to make like smarter choices to cobble together this, to create like a workflow for your engineers. And you can really get started quite fast, and actually get on par with some of these modern tech companies. And that is the advantage that a lot of enterprises see. >> If you were going to be the CTO or CEO of a big transformation, knowing what you know, 'cause you just brought up the killer point about why it's such a great time right now, you got platform as a service and the tooling essentially reset everything. So if you're going to throw everything out and start fresh, you're basically brewing the system architecture. It's a complete reset. That's doable. How fast do you think you could do that for say a large enterprise? >> See, I think if you set aside the organization processes and whatever kind of comes in the friction, from a technology perspective, it's pretty fast, right? You can devise a data architecture today with like tools like Kafka, Snowflake and Redshift, and you can actually devise a data architecture very clearly right from day one and actually implement it at scale. And then once you have accumulated enough data and you can extract more value from it, you can go and implement your MLOps workflow as well on top of it. And I think this is where tools like Fiddler can help as well. So I would start with looking at data, do we have centralization of data? Do we have like governance around data? Do we have analytics around data? And then kind of get into machine learning operations. >> Krishna, always great to have you on theCUBE. You're great masterclass guest. Obviously great success in your company. Been there, done that, and doing it again. I got to ask you, since you just brought that up about the whole reset, what is the superhero persona right now? Because it used to be the full stack developer, you know? And then it's like, then I call them, it didn't go over very well in theCUBE, the half stack developer, because nobody wants to be a half stack anything, a half sounds bad, worse than full. But cloud is essentially half a stack. I mean, you got infrastructure, you got tools. Now you're talking about a persona that's going to reset, look at tools, make selections, build an architecture, build an operating environment, distributed computing operating. Who is that person? What's that persona look like? >> I mean, I think the superhero persona today is ML engineering. I'm usually surprised how much is put on an ML engineer to do actually these days. You know, when I entered the industry as a software engineer, I had three or four things in my job to do, I write code, I test it, I deploy it, I'm done. Like today as an ML engineer, I need to worry about my data. How do I collect it? I need to clean the data, I need to train my models, I need to experiment with what it is, and to deploy them, I need to make sure that they're working once they're deployed. >> Now you got to do all the DevOps behind it. >> And all the DevOps behind it. And so I'm like working halftime as a data scientist, halftime as a software engineer, halftime as like a DevOps cloud. >> Cloud architect. >> It's like a heroic job. And I think this is why this is why obviously these jobs are like now really hard jobs and people want to be more and more machine learning >> And they get paid. >> engineering. >> Commensurate with the- >> And they're paid commensurately as well. And this is where I think an opportunity for tools like Fiddler exists as well because we can help those ML engineers do their jobs better. >> Thanks for coming on theCUBE. Great to see you. We're here at re:MARS. And great to see you again. And congratulations on being on the AWS startup showcase that we're in year two, episode four, coming up. We'll have to have you back on. Krishna, great to see you. Thanks for coming on. Okay, This is theCUBE's coverage here at re:MARS. I'm John Furrier, bringing all the signal from all the noise here. Not a lot of noise at this event, it's very small, very intimate, a little bit different, but all on point with space, machine learning, robotics, the future of industrial. We'll back with more coverage after the short break. >> Man: Thank you John. (upbeat music)

(upbeat pop music) >> Well, hi everyone, John Walls here on "theCUBE" as we continue our CUBE conversations as part of the "AWS Startup Showcase". And we welcome in today Krishna Gade who is the founder and the CEO of Fiddler AI. and Krishna, good to see you today. Thanks for joining us here on the "theCUBE". >> Hey John, thanks so much for inviting us and I'm glad to be here, and looking forward to our conversation. >> Yeah me two, and first off, I want to say congratulations as I look at your company's, this tremendous roster, this list of awards that just keep coming your way. Most recently recognized by "Forbes" as one of the Top 50 AI Companies To Watch here in 2021. I know Gartner called you one of their Cool Companies not too long ago. World Economic Forum also giving you a shout out. So whatever it is you're doing, you're doing it very well, but it's got to feel good I would think, some validation to get all this kind of recognition. >> Absolutely, I know we've been very fortunate to get all the recognition. You know, part of it is also because of the space we are playing in, right? A lot of companies are, you know, operationalizing AI and therefore, you know, this whole point of, you know, explainability monitoring and governance of AI is like forefront and it's in the news for various different reasons. So there's a lot of, you know, good sort of talk that is going on in the press around how one should bear responsible AI. And we are very fortunate to be, you know, in the space and pioneering, you know, some of the technologies here. >> Right. And talking about machine learning monitoring, obviously, in the AI space, and you mentioned explainability. So let's just talk about that concept broadly first off and explain to our viewers what you mean by explainability in this particular context. >> Yeah, that's a good question. So if you think about an AI system, one of the main differences between it and a traditional software system is that it's a black box in the sense that you cannot open it up and read it's code like a traditional software system. The reason is, you know, the AI systems that are built using data and training models which are represented in this non-human readable format. And you cannot really understand how a model is actually making a prediction at any given point of time. So therefore what happens is when you are deploying these AI systems at scale for a variety of use cases, let's say credit underwriting or, you know, screening resumes, or clinical diagnosis which are extremely, you know, important for general human beings. There is a need to understand how the AI system is working. You know, why did it approve a positive person's loan or reject someone's loan? Or why did it reject someone's, you know, resume from, you know, a job screening pipeline? How is it working overall? Right? And so this is where explainability becomes important because you need to understand the AI system, you need a way to probe it, to interrogate it, to understand how the system is making predictions, how is it being influenced by various inputs you're supplying to the system. And so this gamut of technologies or the algorithms that have come across in the last, you know, few years have really matured to a point where, you know, products like Fiddler are developing them and productizing them for the general enterprise to you know, put it in their machine learning and AI workflows. >> So you're talking about context basically, right? I mean, trying to give everybody an idea. This is, you know, kind of where this inputs coming, this is where the problem is, this is where the bottleneck might be, whatever it is, and and doing that in real time. Very efficient operation here. Well, let's talk about the ML world right now and in terms of how it relates to artificial intelligence and this interaction you know, that we're seeing and the, I guess, the problem that you are trying to fix, if you will, in terms of machine learning monitoring. So let's just deal with that first off. When you look at somebody's architecture and somebody set up, what do you see? What are you looking for? And what kind of problems are you trying to solve for your clients? >> Yeah. So just following up what I said. The two main problems with operationalizing AI is one is the black box nature of AI, which I already talked about. The other problem is that the AI system is fundamentally a stochastic system or a probabilistic system. By that, I mean that its performance, you know, its predictions can change over time based on the data it is receiving. So it's not a deterministic system like traditional software systems where you expect the same output all the time, right? So when you have a system that is stochastic in nature where its performance can vary based on the data it is receiving, then you are in a situation where you have uncertainty, right? You know, you let's say you have an AI system that is deployed for serving a credit underwriting model or a fraud, you know, detection use case. And you see that, okay, sometimes accuracy is up, sometimes accuracy is down. You know, when do you want, when do you trust your predictions, when you're not. How do you know if the model is actually performing in the same manner that you trained it? All of these issues open up the need for continuous monitoring of these AI systems, because without which you may have AI systems making bad predictions for your users, hurting your business metrics, potentially making biased decisions that can put your company into a compliance or a brand reputation risk scenario. To avoid all of these things you can actually monitor these AI systems continuously so that you know exactly if they're performing the way you expect them to be. Do you to retrain them right now, right? Or do you need to shut them down because they are actually not predicting the way that you expect them to be? So this is actually very important. And so that's what Fiddler tries to solve for our customers by helping them operationalize AI with full visibility and explainability, right? So you can essentially install Fiddler in your workflow to continuously monitor your AI systems and analyze and explain them when you have questions about how they're working. >> I mean, you talked about governance earlier a little bit, you know, compliance, obviously a great critical issue, big concern, fraud detection. Security, just in general here, as we know, I mean, we keep almost every day it seems like we're hearing about some kinds of security intrusion. So, in terms of identifying vulnerabilities or in terms of identifying anomalies, whatever it might be, what kind of work are you doing in that space to give your client base the kind of comfort and the peace of mind that everybody's searching for these days? >> Right, I mean, if you step back a little bit, John, we are truly living in the age of algorithms, right? So everything that we interact with on a day-to-day basis, the movies we watch, or when we request an Uber driver, or when we go to a financial institution and request for a loan application or a mortgage, there are algorithms behind the scenes that are processing our requests and delivering the experiences that we have. Now, increasingly these algorithms are becoming AI based algorithms. And when you have these AI based algorithms, they're trained on this data that's available, that an institution may collect from their users, or they may buy from other third parties. And when you develop these AI systems based on this data, if this data is not equally distributed amongst all different ethnicity backgrounds, people coming from different cultures, different religions, different races, different genders, you may actually build systems that can make very different decisions for different individuals based on like this bias that could creep into them. And so this actually needs, this means that at the end of the day, you can actually create a dystopian world where, you know, some people get like really great decisions from your systems, where some people are left out, right? So therefore, you know, this aspect of governing your AI systems so that you're validating what you're building upfront. You're validating the data that you're using to train the systems. You're continuously monitoring the systems there so that they're actually producing the right outcomes for your users. And then you can actually explain if some customer asks you or some regulator or a third party asks you how your system is working. It's very very important. This is an emerging area in industry, certain sectors already have this, for example, financial services. It's in companies like banks, where it is mandated to have model governance, so that every model that they are deploying needs to be validated and needs to be monitored. And we are seeing the emergence of generally AI governance creeping into other sectors as well. And so this is like a broader topic that covers explainability, covers monitoring, covers detecting bias in your AI systems and ensuring that you're building safe and responsible AI for your customers and your organization. >> Yeah, I find the bias point really interesting, actually, because I hadn't really thought about these prejudices or subjectivities, you know, it might bring to our work with us in terms of what we look at, what we ignore, what we process, how we don't. But it's a really interesting point you just raised. So thank you for that. And then there's also the kind of issue with data drift too a little bit, right? It's like, where did it go (laughing)? >> Right. >> What are we doing here? What happened to it? So maybe if you could talk about that a little bit in terms of all this data that's coming in and corralling it, right? Making sure that it stays organized and stays in a way that you can analyze and process it, and then glean insight from. >> Yeah, data drift is one of the main reasons why AI systems deteriorate in performance. So for example, let's say I'm trying to build a recommendation system that predicts the items that you want to buy when you go to an E-commerce website. Now, if I have used data pre-COVID, then the user behavior was very different, right? That kind of items people were probably buying before you know, February, 2020 was like probably much different than the kind of items that people were buying after it. So what happens is when you train your AI systems on datasets that are older but then that data has changed ever since because of an event like COVID-19 has happened, or some other seasonality has kicked in, then your AI systems are seeing different distribution data. For example, you may see that suddenly, you know, people who were shopping, let's say, in March or April last year, people were shopping for all kinds of, you know, toilet paper and all kinds of things to stock up, you know, to be ready for lockdown, right? And maybe they were not buying similar amounts in there previously. So therefore, if you have an inventory management system based on AI or an E-commerce recommendation system based on AI, you know, they would see data drift, because the buying patterns are different. The amount of stuff that people are buying in terms of toilet paper has completely shifted. And so their model is actually, may not be predicting as accurately as it would, right? So therefore identifying this data drift and alerting your AI engineer so that they can be prepared for this is very important. Otherwise, what you would see is if you're an E-commerce company, this has actually happened, you know? Instacart, a grocery delivery company and another company www.etsy.com, they blogged about it where they have seen their models go down in accuracy from 90% to 65% when this data shift happened, you know, especially during COVID-19. And so you need the ability to continuously monitor for drift so that when you can catch these things earlier, and then, you know, save your business from losing, you know, in terms of business metrics like such as number of sales that you may be making, number of bad recommendations that your systems are making to your users. >> So we've talked a lot about these various components of monitoring of which, you know, all of which you do extremely well. And I was reading earlier, just a little bit about the company, and we talked about accountability. We've already talked about that. We talked about fraud detection, we talked about reliability. There was also a point about ethical considerations, you know, and so I was interested in that, hearing from you about that in terms of why that's a pillar of your service or what exactly that was pointed toward in terms of monitoring, and what you can do. >> Right. So, I guess I'll just go back to like a famous quote from Marc Andreessen. He mentioned, you know, a few years ago that software is eating the world, right? Now, what's happening is AI is eating software. All the software that we are consuming is becoming AI based software, because basically at the end of the day some intelligence is being baked into the software to make it, you know, predict more interesting things for you to make those decisions. Instead of rule-based decisions, make it more AI based decisions. And so therefore it is very important that when we are building the software, we need to use ethical practices. You know, we need to know how, where you're collecting the data from. It can be very dangerous if you don't do it and you can land into trouble. And we have seen these incidents many times, right? For example, in 2019, when Apple and Goldman Sachs came up with a credit card, a lot of customers complained about gender bias with respect to the credit card limits that the algorithm was setting. You know, in the same household, the husband and wife were getting 10 times in terms of a difference between the credit limit between a male and a female, right? Even though they probably had similar salary ranges, similar FICO scores, right? So if you do not actually make sure that, you know, you're collecting data from the right sources that your datasets are not outbalanced. If your models, if your algorithms are tested for bias you know, before hand, before you deploy them and then you're continuously monitoring them, these are all ethical practices. These are all the responsible ways of building your AI. You can actually, you know, land into trouble. Your customers will complain about it. You know, you would lose your brand reputation. And at the end of the day you'll be essentially, and instead of actually adding value to the customers, you may be actually hurting them, right? And so this is actually why it's so important, and it's become more important when the more stakes, the higher the stakes are, right? You know, for example, when it's being used for criminal justice scenarios or when it's being used for clinical diagnosis scenarios. Being able to ensure that the system is making unbiased decisions is very, very important. >> Well, before I let you go, too, I like you to touch base on your AWS relationship about, you know, what was the Genesis of that. And currently what it is that you're working on together to provide this great value to your customers. >> Absolutely. So the follow-up to this ethical AI is like Amazon as a company is interested in pursuing, you know, the responsible AI but, you know, they have a lot of AI products. So they are looking for, you know, fostering a community and ecosystem of AI technologies. And in that hypothesis they actually invested in Fiddler last year in terms of enabling us to develop this explainable AI and ethical AI technology. And so we are working with Alexa Fund and also like AWS ecosystem in terms of partnering with how effectively Fiddler can be delivered to other AWS customers through, like, through their marketplace and other sort of areas that we can distribute the software. So it's a great partnership. We are very, very excited about the opportunity to work with Alexa Fund as well as the AWS ecosystem. It increases another opportunity for us to enable a lot more customers than we than we can otherwise. So this is a great win-win situation for both Amazon and Fiddler. >> Well, it sure is. And congratulations on that and developing that partnership. I know it's working well for your clients and it's working well for Fiddler AI obviously by the number of recognitions that have been coming your way. So Krishna, we wish you continued success and thanks for the time here today on "theCUBE". >> Yep. Thank you so much, John. It was a pleasure talking to you today. >> I enjoyed it. Thank you. John Walls here wrapping up our conversation with Fiddler AI's Krishna Gade, talking today about machine learning monitoring on the "AWS Startup Showcase". (upbeat pop music)