>>from >>around the globe. It's the Cube with digital coverage of AWS reinvent 2020 sponsored by Intel, AWS and our community partners. >>Hi. >>And welcome to the Cube. Virtual in our coverage of aws reinvent 2020. We have a cube virtual, and I'm your host, Justin Warren. And today, my guest is Shauna Wolverton, executive vice president of product at ZENDESK. And she's coming to us from Oakland, California. Shauna, welcome to the >>Cube. Thanks so much for having me. It is >>It is lovely to be here. How's the weather over there? In Oakland, >>we just suddenly went from summer to winter, which, uh, after the weather we've had is no complaints. >>All right, Well, as as a resident of Melbourne, where we have four seasons in one day, I am very familiar with rapid weather changes. So, uh, hopefully it's not too cold for you, and you get a little bit of nicer weather just before you go fully into winter. Absolutely. Now Zendesk and Amazon have a pretty close relationship is my understanding, and we know that Amazon is famous for its customer center at attitude. Wonderful thing about customers, of course, is that they're never really happy with everything that we have. So zendesk fit in with that with that relationship with Amazon. And how is your approach to customer? >>Yeah. I mean, the relationship we have with them is I'm really excited. Really Have gone all in on our move to the cloud. There are sole provider on DWI run all of our services, um, on AWS. And in addition, we have some great partnerships with, uh, Jacob Amazon Connect, which allows us to provide great telephony and call center services to our customers. We have a great partnership around event bridge and a zwelling app connect. So I think there is a fantastic relationship that we have where we're able to deliver not just our basic services, but to really take advantage of a lot of the services that Amazon on AWS provide s so that we can sort of accelerate our own roadmap and deliver great new features to our customers. >>Now, a lot of people have gone through a pretty similar adoption of the cloud of the moment. Unfortunate reason for doing so. But it certainly has driven the adoption very, very quickly. Uh, zendesk, of course, as you say, has been has been doing this for quite some time. So what have you noticed that stayed the same eso from last year to this year? What were you already doing that you're now noticing? Everyone else's discovering. Actually, this is pretty good. >>Well, you know, I think you know the rumors of of the call center and and the telephone as a channel. Their demise are greatly exactly. I think, um, for us. Much as we're all excited about chat and messaging and all of the different ways that we can connect with our customers, there's something about having a phone number and allowing people to pick up the phone and talk to a human that refuses to go out of style. And so I think, um, you know, our partnership with, uh with Amazon connection has been hugely powerful and even, you know, recently when a lot of this sort of acceleration has picked up, we've seen, um, you know, we saw a customer who had a power failure kind of massive failure of their own phone system. Be able thio, come to us, get, get, connect up and running incredibly quickly and start taking thousands of calls a day and that kind of sort of quick time to value fast start ability for our customers. Just this hugely important. Um, now. But really, you know, that's always been true, right? >>Yeah. I mean, when people want to call you and they want to talk to you, then they're not really happy If they can't get through that and particularly right now, being able to make that human human connection for me, I know that that that's been a really important part of getting through this. I work remotely most of the time. So actually, speaking to humans as we're doing now is is really refreshing change from just seeing everything on on a text screen. Um, so yeah, so it's It's interesting that the phone has actually has been so resilient, even though we were here from Ah, lot of young people say, Oh, we never answer the phone when someone calls, uh, but a lot of people are actually calling into businesses when they wanna make contact or when they when they don't see things on the website. So >>how does >>zendesk help, too, to integrate with what people are doing in their online and digital channels through to what they're doing with phone system. >>Yeah, but I think fundamentally people want their questions answered. One of my favorite studies that we did was around our benchmark study and we talked to Millennials. They said the first place they go to get help to their phone, but when you push it a little deeper, it was clear that they actually didn't know that the phone was for making phone calls. It was just all of the other help centers like like the first way that a lot of people today are looking for. Answers is, you know I wanna google it. And for that you need a really great help center has all that information out there and then you want toe have, you know, communities where people can talk to each other and get help. And then, you know, Mawr and Mawr. We're seeing the rise of messaging as a channel, both through the social channels like WhatsApp and Facebook Messenger Aziz Well, Azaz native messaging kind of ongoing conversations. He you ordered your dinner. It hasn't arrived. It's so great to be able to go into those applications and just message to the business and figure out what's what's going on and get that sort of instantaneous response as well, >>right? And you shared some stats with this regarding how much has moved across to some of these things phone based messaging channels. So tickets coming in has risen about 50% on DCA, paired to some gains on on live chat. So people are really embracing the idea of being about a message, not just individual talking to your friends in the group chat, but actually using that to engage with with the companies that they would normally use websites or or phone. It's like text chat is a thing. >>Yeah, I mean, it was funny to me. You know, I think we're still, uh, in the U. S. Not quite as far along as a lot of our international friends. When I when traveling was a thing that we did, you know, I was always like it was cool to see that there were billboards and ads that had what that phone numbers on them is a really, you know, way that businesses were wanting to engage. I mean, you think about be wanting to be where your customers are today. So many of us, um do have you know what's happened? Wechat and line and vibrant. They're all in our pocket. And being able to provide all of those two businesses is a new way to engage. I think we're finding is hugely powerful, >>right? So with with all of these dynamic changes that have been happening, and it sounds like it's actually just sort of riding the wave of what customers were already doing, we're just doing it just that little bit mawr. But have you noticed any other larger changes? Possibly ones that aren't related thio a pandemic, Just general shifts that have been happening that you've seen in your customer base? >>Yeah. I mean, like I said, I think so much of what we're seeing is that people, uh, in general want answers quickly, and whether it's a phone call is great. And like I said, people are not going to stop calling. But I think people want to make sure less than like, I need a human to have a conversation I want. I want the answer quickly, and that's where we're really focused in both thinking about how we provide tools around automating some of getting those answers using, uh, a i N m l so that people can come to us, ask questions and we can get them the best answer very quickly without, um, having Thio engage a person. I think things idea of quick resolution is clearly becoming one of the most important things in customer sentiment. I think we know that, um, Mawr and Mawr. This idea of how quickly I can get my question's resolved or how easy it is for me to do business with you is a huge differentiator in how people make buying >>choices. Mm. On that. That automation has long been a new track tive idea. I mean, I'm I'm old enough to remember expert systems and and having a go at doing this kind of heavily automated way of resolving particularly common issues. And I mean, we were familiar with Coulson, a chat scripts. Where there's here are the top three issues and or it will be in the I V. R. Where it's like we're currently experiencing this particular problems, so that resolves your question quite quickly. But there's been a big rise in things like chatbots and and the use of AI. How far advanced. Is that because I still remember some of the early forays into that were a little bit flaky, and that could actually exacerbate the poor customer experience. I'm already having a problem, and and now you're chatbots getting in the way. Have they gotten a lot better? Are they Are they up to the challenge? >>Yeah. I mean, I think what's really critical when you're thinking about automation? Um, in the conversations you're having with customers, it's it's two things. One Don't try to hide that. That you're a computer. No, no, my name is Chad. I am. I am a human. Um, you're not in the vault. Yeah, there's not anyone. Um, so I think being really clear. And then, um e think surfacing how thio very easily opt out of those flows. I think, um, you know, automation is great, but it's not away. You shouldn't think of it as a way to frustrate your users to keep them tied up until you can get to them. It really is. Give them some quick options. And if they don't? If those don't solve their problems, really make sure that your you've got an escape valve, right? We were putting out a new sort of flow build their product zendesk. And we have all of the different, uh, words that someone could say that air like smashing the zero button. That means please transfer me to a person, right? You're driving me crazy. Let me connect you to an agent. Eso We're really making sure that it's easy, um, for customers to provide the solution where their customers can get the help they need rather than I >>really like that. That's That's something I think that gets a little bit lost in the focus on computers and and on automation is that the reason we do this is to help the humans. So when we have these AI systems, it's not actually to replace. The human interaction is to make it better. It's to make mean that we can then get to that genuine connection. Computers a fabulous and when they work, it's when they don't when they frustrate things that that bothers us. And that's generally why we're calling is that something has already gone wrong and we're a bit frustrated. So adding more frustration, doesn't it? Sounds like a good approach. It sounds like zendesk really got that? That dolled in very, very well. Is that something that you've you've always had? Is it something that you've refined over time? And can you teach it to a bunch of other companies? >>Way would love to teach each other. People know, I think e think we have always thought about how the machines can help the humans. And I think one it's how can they help the customers, of course. But the other side that I don't think people talk about quite a much is how can we use computers to help agents? Right. So you're talking to a person, and how can we take sort of the best answers that they've given Thio other customers and surface those, um, when When a new agent is coming on board, how do we suggest, um, you know, the different kinds of work flows that they might want to use to solve this problem in a more dynamic way. So I really like to think of the computers never as a replacement but really as a sort of hidden superpower, Um, that organizations have to make every agent one of their best >>agents, right? Yes, it is a kind of external cyborg thing. I mean, I can't remember anything these days. I constantly right less and they all live in computers. But they are. That's the kind of society that we live with today. And I think we should remember to embrace that side of things. That ah, lot of life has actually gotten a lot better through the use of these computing systems. It's not all terrible. It's, um, and I think more companies could probably learn from zendesk. And the approach that you've taken to center the humans, both the customers and and your internal staff, the call center and and the people who are providing this service. No one enjoys it when things are breaking and and things have gone wrong being able to resolve that quickly. Thanks a better experience for everybody. >>Yeah. I mean, I think we find over and over again sometimes you know, if you can handle an issue that's gone wrong, Um well, you can actually induce more loyalty than you know. If someone never contacted. You'd also if you could really take advantage of the times you have, unfortunately, maybe messed up on bake those customers happy. You really do you know, put so much in the sort of loyalty piggy bank for later. It's really great. >>So for some of the companies that have maybe struggled with this a little bit and particularly under very trying conditions, is there's some advice that you could give to them. Is there some places that they should should start to investigate this when they want to improve the way that they handle customer service, perhaps with things like Zendesk. >>Yeah, I mean, I think a lot of what what we're focused on right now is the this channel that's coming. Like I said, we think a lot about social messaging, but also in native messaging. Andi, how you can have a sort of ongoing long term conversation for a long time customer service, sort of Holy Grail was chat, and you could have a agent online and a human online, and you could solve their problem and then move on right And and sometimes those things take a little longer to solve. Or, you know, you might have a big issue and a whole bunch of people who have an issue and maybe not enough agents to solve them. And so, with messaging. We've really changed the dynamic. So chat was this completely synchronous, Almost like a phone call. Kind of experience and more messaging. You're able to live in this sort of duality where we can have a conversation if we're both here. But just like with your friends, right? Sometimes you throw a message out to offend you. Put it in your pocket, you pick it up, and you could pick up the conversation right where you left off. So bring that paradigm into your customer support experience really allows you to take some of that fear out of handling the volume that might come from chat. To be able to sort of have these ongoing sort of back and forth conversations over time. Andi also and give that that persistent so that we're always both in the same place when we show up again together >>embracing what the technology does well and avoiding what it doesn't do. Well, that that sounds like a plan. >>Shawna, >>this has been fabulous. It is. It is always very edifying for me. Thio here, when companies are doing well and centering the humans to make the technology improve all of our lives. Um It has been wonderful to have you here on the Cube. >>Thanks so much. It was a lot of fun, right? >>And thank you for joining in and and watching us here of the Cube virtual and our special coverage off AWS reinvent 2020. Do come back and look for more coverage off. Reinvent 2020 right here on the Cube. Next time I've been your host, Justin Warren, and we'll see you again soon.

>>from Las Vegas. It's the Q covering. Next work. 2019 America's Do You buy Juniper Networks? Come back already. Jeffrey here with the Cube were in Las Vegas at Caesar's at the Juniper. Next work event. About 1000 people kind of going over a lot of new cool things. 400 gigs. Who knew that was coming out of new information for me? But that's not what we're here today. We're here for the fourth installment of around the Cube unpacking. I were happy to have all the winners of the three previous rounds here at the same place. We don't have to do it over the phone s so we're happy to have him. Let's jump into it. So winner of Round one was Bob Friday. He is the VP and CTO at Missed the Juniper Company. Bob, Great to see you. Good to be back. Absolutely. All the way from Seattle. Sharna Parky. She's a VP applied scientist at Tech CEO could see Sharna and, uh, from Google. We know a lot of a I happen to Google. Rajan's chef. He is the V p ay ay >>product management on Google. Welcome. Thank you, Christy. Here >>All right, so let's jump into it. So just warm everybody up and we'll start with you. Bob, What are some When you're talking to someone at a cocktail party Friday night talking to your mom And they say, What is a I What >>do you >>give him? A Zen examples of where a eyes of packing our lives today? >>Well, I think we all know the examples of the south driving car, you know? Aye, aye. Starting to help our health care industry being diagnosed cancer for me. Personally, I had kind of a weird experience last week at a retail technology event where basically had these new digital mirrors doing facial recognition. Right? And basically, you start to have little mirrors were gonna be a skeevy start guessing. Hey, you have a beard, you have some glasses, and they start calling >>me old. So this is kind >>of very personal. I have a something for >>you, Camille, but eh? I go walking >>down a mall with a bunch of mirrors, calling me old. >>That's a little Illinois. Did it bring you out like a cane or a walker? You know, you start getting some advertising's >>that were like Okay, you guys, this is a little bit over the top. >>Alright, Charlotte, what about you? What's your favorite example? Share with people? >>Yeah, E think one of my favorite examples of a I is, um, kind of accessible in on your phone where the photos you take on an iPhone. The photos you put in Google photos, they're automatically detecting the faces and their labeling them for you. They're like, Here's selfies. Here's your family. Here's your Children. And you know, that's the most successful one of the ones that I think people don't really think about a lot or things like getting loan applications right. We actually have a I deciding whether or not we get loans. And that one is is probably the most interesting one to be right now. >>Roger. So I think the father's example is probably my favorite as well. And what's interesting to me is that really a I is actually not about the Yeah, it's about the user experience that you can create as a result of a I. What's cool about Google photos is that and my entire family uses Google photos and they don't even know actually that the underlying in some of the most powerful a I in the world. But what they know is they confined every picture of our kids on the beach whenever they whenever they want to. Or, you know, we had a great example where we were with our kids. Every time they like something in the store, we take a picture of it, Um, and we can look up toy and actually find everything that they've taken picture. >>It's interesting because I think most people don't even know the power that they have. Because if you search for beach in your Google photos or you search for, uh, I was looking for an old bug picture from my high school there it came right up until you kind of explore. You know, it's pretty tricky, Raja, you know, I think a lot of conversation about A They always focus the general purpose general purpose, general purpose machines and robots and computers. But people don't really talk about the applied A that's happening all around. Why do you think that? >>So it's a good question. There's there's a lot more talk about kind of general purpose, but the reality of where this has an impact right now is, though, are those specific use cases. And so, for example, things like personalizing customer interaction or, ah, spotting trends that did that you wouldn't have spotted for turning unstructured data like documents into structure data. That's where a eyes actually having an impact right now. And I think it really boils down to getting to the right use cases where a I right? >>Sharon, I want ask you. You know, there's a lot of conversation. Always has A I replace people or is it an augmentation for people? And we had Gary Kasparov on a couple years ago, and he talked about, you know, it was the combination if he plus the computer made the best chess player, but that quickly went away. Now the computer is actually better than Garry Kasparov. Plus the computer. How should people think about a I as an augmentation tool versus a replacement tool? And is it just gonna be specific to the application? And how do you kind of think about those? >>Yeah, I would say >>that any application where you're making life and death decisions where you're making financial decisions that disadvantage people anything where you know you've got u A. V s and you're deciding whether or not to actually dropped the bomb like you need a human in the loop. If you're trying to change the words that you are using to get a different group of people to apply for jobs, you need a human in the loop because it turns out that for the example of beach, you type sheep into your phone and you might get just a field, a green field and a I doesn't know that, uh, you know, if it's always seen sheep in a field that when the sheep aren't there, that that isn't a sheep like it doesn't have that kind of recognition to it. So anything were we making decisions about parole or financial? Anything like that needs to have human in the loop because those types of decisions are changing fundamentally the way we live. >>Great. So shift gears. The team are Jeff Saunders. Okay, team, your mind may have been the liquid on my bell, so I'll be more active on the bell. Sorry about that. Everyone's even. We're starting a zero again, so I want to shift gears and talk about data sets. Um Bob, you're up on stage. Demo ing some some of your technology, the Miss Technology and really, you know, it's interesting combination of data sets A I and its current form needs a lot of data again. Kind of the classic Chihuahua on blue buried and photos. You got to run a lot of them through. How do you think about data sets? In terms of having the right data in a complete data set to drive an algorithm >>E. I think we all know data sets with one The tipping points for a I to become more real right along with cloud computing storage. But data is really one of the key points of making a I really write my example on stage was wine, right? Great wine starts a great grape street. Aye, aye. Starts a great data for us personally. L s t M is an example in our networking space where we have data for the last three months from our customers and rule using the last 30 days really trained these l s t m algorithms to really get that tsunami detection the point where we don't have false positives. >>How much of the training is done. Once you once you've gone through the data a couple times in a just versus when you first started, you're not really sure how it's gonna shake out in the algorithm. >>Yeah. So in our case right now, right, training happens every night. So every night, we're basically retraining those models, basically, to be able to predict if there's gonna be an anomaly or network, you know? And this is really an example. Where you looking all these other cat image thinks this is where these neural networks there really were one of the transformational things that really moved a I into the reality calling. And it's starting to impact all our different energy. Whether it's text imaging in the networking world is an example where even a I and deep learnings ruling starting to impact our networking customers. >>Sure, I want to go to you. What do you do if you don't have a big data set? You don't have a lot of pictures of chihuahuas and blackberries, and I want to apply some machine intelligence to the problem. >>I mean, so you need to have the right data set. You know, Big is a relative term on, and it depends on what you're using it for, right? So you can have a massive amount of data that represents solar flares, and then you're trying to detect some anomaly, right? If you train and I what normal is based upon a massive amount of data and you don't have enough examples of that anomaly you're trying to detect, then it's never going to say there's an anomaly there, so you actually need to over sample. You have to create a population of data that allows you to detect images you can't say, Um oh, >>I'm going to reflect in my data set the percentage of black women >>in Seattle, which is something below 6% and say it's fair. It's not right. You have to be able thio over sample things that you need, and in some ways you can get this through surveys. You can get it through, um, actually going to different sources. But you have to boot, strap it in some way, and then you have to refresh it, because if you leave that data set static like Bob mentioned like you, people are changing the way they do attacks and networks all the time, and so you may have been able to find the one yesterday. But today it's a completely different ball game >>project to you, which comes first, the chicken or the egg. You start with the data, and I say this is a ripe opportunity to apply some. Aye, aye. Or do you have some May I objectives that you want to achieve? And I got to go out and find the >>data. So I actually think what starts where it starts is the business problem you're trying to solve. And then from there, you need to have the right data. What's interesting about this is that you can actually have starting points. And so, for example, there's techniques around transfer, learning where you're able to take an an algorithm that's already been trained on a bunch of data and training a little bit further with with your data on DSO, we've seen that such that people that may have, for example, only 100 images of something, but they could use a model that's trained on millions of images and only use those 100 thio create something that's actually quite accurate. >>So that's a great segue. Wait, give me a ring on now. And it's a great Segway into talking about applying on one algorithm that was built around one data set and then applying it to a different data set. Is that appropriate? Is that correct? Is air you risking all kinds of interesting problems by taking that and applying it here, especially in light of when people are gonna go to outweigh the marketplace, is because I've got a date. A scientist. I couldn't go get one in the marketplace and apply to my data. How should people be careful not to make >>a bad decision based on that? So I think it really depends. And it depends on the type of machine learning that you're doing and what type of data you're talking about. So, for example, with images, they're they're they're well known techniques to be able to do this, but with other things, there aren't really and so it really depends. But then the other inter, the other really important thing is that no matter what at the end, you need to test and generate based on your based on your data sets and on based on sample data to see if it's accurate or not, and then that's gonna guide everything. Ultimately, >>Sharon has got to go to you. You brought up something in the preliminary rounds and about open A I and kind of this. We can't have this black box where stuff goes into the algorithm. That stuff comes out and we're not sure what the result was. Sounds really important. Is that Is that even plausible? Is it feasible? This is crazy statistics, Crazy math. You talked about the business objective that someone's trying to achieve. I go to the data scientist. Here's my data. You're telling this is the output. How kind of where's the line between the Lehman and the business person and the hard core data science to bring together the knowledge of Here's what's making the algorithm say this. >>Yeah, there's a lot of names for this, whether it's explainable. Aye, aye. Or interpret a belay. I are opening the black box. Things like that. Um, the algorithms that you use determine whether or not they're inspect herbal. Um, and the deeper your neural network gets, the harder it is to inspect, actually. Right. So, to your point, every time you take an aye aye and you use it in a different scenario than what it was built for. For example, um, there is a police precinct in New York that had a facial recognition software, and, uh, victim said, Oh, it looked like this actor. This person looked like Bill Cosby or something like that, and you were never supposed to take an image of an actor and put it in there to find people that look like them. But that's how people were using it. So the Russians point yes, like it. You can transfer learning to other a eyes, but it's actually the humans that are using it in ways that are unintended that we have to be more careful about, right? Um, even if you're a, I is explainable, and somebody tries to use it in a way that it was never intended to be used. The risk is much higher >>now. I think maybe I had, You know, if you look at Marvis kind of what we're building for the networking community Ah, good examples. When Marvis tries to do estimate your throughput right, your Internet throughput. That's what we usually call decision tree algorithm. And that's a very interpretive algorithm. and we predict low throughput. We know how we got to that answer, right? We know what features God, is there? No. But when we're doing something like a NAMI detection, that's a neural network. That black box it tells us yes, there's a problem. There's some anomaly, but that doesn't know what caused the anomaly. But that's a case where we actually used neural networks, actually find the anomie, and then we're using something else to find the root cause, eh? So it really depends on the use case and where the night you're going to use an interpreter of model or a neural network which is more of a black box model. T tell her you've got a cat or you've got a problem >>somewhere. So, Bob, that's really interested. So can you not unpacking? Neural network is just the nature of the way that the communication and the data flows and the inferences are made that you can't go in and unpack it, that you have to have the >>separate kind of process too. Get to the root cause. >>Yeah, assigned is always hard to say. Never. But inherently s neural networks are very complicated. Saito set of weights, right? It's basically usually a supervised training model, and we're feeding a bunch of data and trying to train it to detect a certain features, sir, an output. But that is where they're powerful, right? And that's why they basically doing such good, Because they are mimicking the brain, right? That neural network is a very complex thing. Can't like your brain, right? We really don't understand how your brain works right now when you have a problem, it's really trialling there. We try to figure out >>right going right. So I want to stay with you, bought for a minute. So what about when you change what you're optimizing? Four? So you just said you're optimizing for throughput of the network. You're looking for problems. Now, let's just say it's, uh, into the end of the quarter. Some other reason we're not. You're changing your changing what you're optimizing for, Can you? You have to write separate algorithm. Can you have dynamic movement inside that algorithm? How do you approach a problem? Because you're not always optimizing for the same things, depending on the market conditions. >>Yeah, I mean, I think a good example, you know, again, with Marvis is really with what we call reinforcement. Learning right in reinforcement. Learning is a model we use for, like, radio resource management. And there were really trying to optimize for the user experience in trying to balance the reward, the models trying to reward whether or not we have a good balance between the network and the user. Right, that reward could be changed. So that algorithm is basically reinforcement. You can finally change hell that Algren works by changing the reward you give the algorithm >>great. Um, Rajan back to you. A couple of huge things that have come into into play in the marketplace and get your take one is open source, you know, kind of. What's the impact of open source generally on the availability, desire and more applications and then to cloud and soon to be edge? You know, the current next stop. How do you guys incorporate that opportunity? How does it change what you can do? How does it open up the lens of >>a I Yeah, I think open source is really important because I think one thing that's interesting about a I is that it's a very nascent field and the more that there's open source, the more that people could build on top of each other and be able to utilize what what others others have done. And it's similar to how we've seen open source impact operating systems, the Internet, things like things like that with Cloud. I think one of the big things with cloud is now you have the processing power and the ability to access lots of data to be able to t create these thes networks. And so the capacity for data and the capacity for compute is much higher. Edge is gonna be a very important thing, especially going into next few years. You're seeing Maur things incorporated on the edge and one exciting development is around Federated learning where you can train on the edge and then combine some of those aspects into a cloud side model. And so that I think will actually make EJ even more powerful. >>But it's got to be so dynamic, right? Because the fundamental problem used to always be the move, the computer, the data or the date of the computer. Well, now you've got on these edge devices. You've got Tanya data right sensor data all kinds of machining data. You've got potentially nasty hostile conditions. You're not in a nice, pristine data center where the environmental conditions are in the connective ity issues. So when you think about that problem yet, there's still great information. There you got latent issues. Some I might have to be processed close to home. How do you incorporate that age old thing of the speed of light to still break the break up? The problem to give you a step up? Well, we see a lot >>of customers do is they do a lot of training on the cloud, but then inference on the on the edge. And so that way they're able to create the model that they want. But then they get fast response time by moving the model to the edge. The other thing is that, like you said, lots of data is coming into the edge. So one way to do it is to efficiently move that to the cloud. But the other way to do is filter. And to try to figure out what data you want to send to the clouds that you can create the next days. >>Shawna, back to you let's shift gears into ethics. This pesky, pesky issue that's not not a technological issue at all, but right. We see it often, especially in tech. Just cause you should just cause you can doesn't mean that you should. Um so and this is not a stem issue, right? There's a lot of different things that happened. So how should people be thinking about ethics? How should they incorporate ethics? Um, how should they make sure that they've got kind of a, you know, a standard kind of overlooking kind of what they're doing? The decisions are being made. >>Yeah, One of the more approachable ways that I have found to explain this is with behavioral science methodologies. So ethics is a massive field of study, and not everyone shares the same ethics. However, if you try and bring it closer to behavior change because every product that we're building is seeking to change of behavior. We need to ask questions like, What is the gap between the person's intention and the goal we have for them? Would they choose that goal for themselves or not? If they wouldn't, then you have an ethical problem, right? And this this can be true of the intention, goal gap or the intention action up. We can see when we regulated for cigarettes. What? We can't just make it look cool without telling them what the cigarettes are doing to them, right so we can apply the same principles moving forward. And they're pretty accessible without having to know. Oh, this philosopher and that philosopher in this ethicist said these things, it can be pretty human. The challenge with this is that most people building these algorithms are not. They're not trained in this way of thinking, and especially when you're working at a start up right, you don't have access to massive teams of people to guide you down this journey, so you need to build it in from the beginning, and you need to be open and based upon principles. Um, and it's going to touch every component. It should touch your data, your algorithm, the people that you're using to build the product. If you only have white men building the product, you have a problem you need to pull in other people. Otherwise, there are just blind spots that you are not going to think of in order to still that product for a wider audience, but it seems like >>they were on such a razor sharp edge. Right with Coca Cola wants you to buy Coca Cola and they show ads for Coca Cola, and they appeal to your let's all sing together on the hillside and be one right. But it feels like with a I that that is now you can cheat. Right now you can use behavioral biases that are hardwired into my brain is a biological creature against me. And so where is where is the fine line between just trying to get you to buy Coke? Which somewhat argues Probably Justus Bad is Jule cause you get diabetes and all these other issues, but that's acceptable. But cigarettes are not. And now we're seeing this stuff on Facebook with, you know, they're coming out. So >>we know that this is that and Coke isn't just selling Coke anymore. They're also selling vitamin water so they're they're play isn't to have a single product that you can purchase, but it is to have a suite of products that if you weren't that coke, you can buy it. But if you want that vitamin water you can have that >>shouldn't get vitamin water and a smile that only comes with the coat. Five. You want to jump in? >>I think we're going to see ethics really break into two different discussions, right? I mean, ethics is already, like human behavior that you're already doing right, doing bad behavior, like discriminatory hiring, training, that behavior. And today I is gonna be wrong. It's wrong in the human world is gonna be wrong in the eye world. I think the other component to this ethics discussion is really round privacy and data. It's like that mirror example, right? No. Who gave that mirror the right to basically tell me I'm old and actually do something with that data right now. Is that my data? Or is that the mirrors data that basically recognized me and basically did something with it? Right. You know, that's the Facebook. For example. When I get the email, tell me, look at that picture and someone's take me in the pictures Like, where was that? Where did that come from? Right? >>What? I'm curious about to fall upon that as social norms change. We talked about it a little bit for we turn the cameras on, right? It used to be okay. Toe have no black people drinking out of a fountain or coming in the side door of a restaurant. Not that long ago, right in the 60. So if someone had built an algorithm, then that would have incorporated probably that social norm. But social norms change. So how should we, you know, kind of try to stay ahead of that or at least go back reflectively after the fact and say kind of back to the black box, That's no longer acceptable. We need to tweak this. I >>would have said in that example, that was wrong. 50 years ago. >>Okay, it was wrong. But if you ask somebody in Alabama, you know, at the University of Alabama, Matt Department who have been born Red born, bred in that culture as well, they probably would have not necessarily agreed. But so generally, though, again, assuming things change, how should we make sure to go back and make sure that we're not again carrying four things that are no longer the right thing to do? >>Well, I think I mean, as I said, I think you know what? What we know is wrong, you know is gonna be wrong in the eye world. I think the more subtle thing is when we start relying on these Aye. Aye. To make decisions like no shit in my car, hit the pedestrian or save my life. You know, those are tough decisions to let a machine take off or your balls decision. Right when we start letting the machines Or is it okay for Marvis to give this D I ps preference over other people, right? You know, those type of decisions are kind of the ethical decision, you know, whether right or wrong, the human world, I think the same thing will apply in the eye world. I do think it will start to see more regulation. Just like we see regulation happen in our hiring. No, that regulation is going to be applied into our A I >>right solutions. We're gonna come back to regulation a minute. But, Roger, I want to follow up with you in your earlier session. You you made an interesting comment. You said, you know, 10% is clearly, you know, good. 10% is clearly bad, but it's a soft, squishy middle at 80% that aren't necessarily super clear, good or bad. So how should people, you know, kind of make judgments in this this big gray area in the middle? >>Yeah, and I think that is the toughest part. And so the approach that we've taken is to set us set out a set of AI ai principles on DDE. What we did is actually wrote down seven things that we will that we think I should do and four things that we should not do that we will not do. And we now have to actually look at everything that we're doing against those Aye aye principles. And so part of that is coming up with that governance process because ultimately it boils down to doing this over and over, seeing lots of cases and figuring out what what you should do and so that governments process is something we're doing. But I think it's something that every company is going to need to do. >>Sharon, I want to come back to you, so we'll shift gears to talk a little bit about about law. We've all seen Zuckerberg, unfortunately for him has been, you know, stuck in these congressional hearings over and over and over again. A little bit of a deer in a headlight. You made an interesting comment on your prior show that he's almost like he's asking for regulation. You know, he stumbled into some really big Harry nasty areas that were never necessarily intended when they launched Facebook out of his dorm room many, many moons ago. So what is the role of the law? Because the other thing that we've seen, unfortunately, a lot of those hearings is a lot of our elected officials are way, way, way behind there, still printing their e mails, right? So what is the role of the law? How should we think about it? What shall we What should we invite from fromthe law to help sort some of this stuff out? >>I think as an individual, right, I would like for each company not to make up their own set of principles. I would like to have a shared set of principles that were following the challenge. Right, is that with between governments, that's impossible. China is never gonna come up with same regulations that we will. They have a different privacy standards than we D'oh. Um, but we are seeing locally like the state of Washington has created a future of work task force. And they're coming into the private sector and asking companies like text you and like Google and Microsoft to actually advise them on what should we be regulating? We don't know. We're not the technologists, but they know how to regulate. And they know how to move policies through the government. What will find us if we don't advise regulators on what we should be regulating? They're going to regulate it in some way, just like they regulated the tobacco industry. Just like they regulated. Sort of, um, monopolies that tech is big enough. Now there is enough money in it now that it will be regularly. So we need to start advising them on what we should regulate because just like Mark, he said. While everyone else was doing it, my competitors were doing it. So if you >>don't want me to do it, make us all stop. What >>can I do? A negative bell and that would not for you, but for Mark's responsibly. That's crazy. So So bob old man at the mall. It's actually a little bit more codified right, There's GDP are which came through May of last year and now the newness to California Extra Gatorade, California Consumer Protection Act, which goes into effect January 1. And you know it's interesting is that the hardest part of the implementation of that I think I haven't implemented it is the right to be for gotten because, as we all know, computers, air, really good recording information and cloud. It's recorded everywhere. There's no there there. So when these types of regulations, how does that impact? Aye, aye, because if I've got an algorithm built on a data set in in person, you know, item number 472 decides they want to be forgotten How that too I deal with that. >>Well, I mean, I think with Facebook, I can see that as I think. I suspect Mark knows what's right and wrong. He's just kicking ball down tires like >>I want you guys. >>It's your problem, you know. Please tell me what to do. I see a ice kind of like any other new technology, you know, it could be abused and used in the wrong waste. I think legally we have a constitution that protects our rights. And I think we're going to see the lawyers treat a I just like any other constitutional things and people who are building products using a I just like me build medical products or other products and actually harmful people. You're gonna have to make sure that you're a I product does not harm people. You're a product does not include no promote discriminatory results. So I >>think we're going >>to see our constitutional thing is going applied A I just like we've seen other technologies work. >>And it's gonna create jobs because of that, right? Because >>it will be a whole new set of lawyers >>the holdings of lawyers and testers, even because otherwise of an individual company is saying. But we tested. It >>works. Trust us. Like, how are you gonna get the independent third party verification of that? So we're gonna start to see a whole terrorist proliferation of that type of fields that never had to exist before. >>Yeah, one of my favorite doctor room. A child. Grief from a center. If you don't follow her on Twitter Follower. She's fantastic and a great lady. So I want to stick with you for a minute, Bob, because the next topic is autonomous. And Rahman up on the keynote this morning, talked about missed and and really, this kind of shifting workload of fixing things into an autonomous set up where the system now is, is finding problems, diagnosing problems, fixing problems up to, I think, he said, even generating return authorizations for broken gear, which is amazing. But autonomy opens up all kinds of crazy, scary things. Robert Gates, we interviewed said, You know, the only guns that are that are autonomous in the entire U. S. Military are the ones on the border of North Korea. Every single other one has to run through a person when you think about autonomy and when you can actually grant this this a I the autonomy of the agency toe act. What are some of the things to think about in the word of the things to keep from just doing something bad, really, really fast and efficiently? >>Yeah. I mean, I think that what we discussed, right? I mean, I think Pakal purposes we're far, you know, there is a tipping point. I think eventually we will get to the CP 30 Terminator day where we actually build something is on par with the human. But for the purposes right now, we're really looking at tools that we're going to help businesses, doctors, self driving cars and those tools are gonna be used by our customers to basically allow them to do more productive things with their time. You know, whether it's doctor that's using a tool to actually use a I to predict help bank better predictions. They're still gonna be a human involved, you know, And what Romney talked about this morning and networking is really allowing our I T customers focus more on their business problems where they don't have to spend their time finding bad hard were bad software and making better experiences for the people. They're actually trying to serve >>right, trying to get your take on on autonomy because because it's a different level of trust that we're giving to the machine when we actually let it do things based on its own. But >>there's there's a lot that goes into this decision of whether or not to allow autonomy. There's an example I read. There's a book that just came out. Oh, what's the title? You look like a thing. And I love you. It was a book named by an A I, um if you want to learn a lot about a I, um and you don't know much about it, Get it? It's really funny. Um, so in there there is in China. Ah, factory where the Aye Aye. Is optimizing um, output of cockroaches now they just They want more cockroaches now. Why do they want that? They want to grind them up and put them in a lotion. It's one of their secret ingredients now. It depends on what parameters you allow that I to change, right? If you decide Thio let the way I flood the container, and then the cockroaches get out through the vents and then they get to the kitchen to get food, and then they reproduce the parameters in which you let them be autonomous. Over is the challenge. So when we're working with very narrow Ai ai, when use hell the Aye. Aye. You can change these three things and you can't just change anything. Then it's a lot easier to make that autonomous decision. Um and then the last part of it is that you want to know what is the results of a negative outcome, right? There was the result of a positive outcome. And are those results something that we can take actually? >>Right, Right. Roger, don't give you the last word on the time. Because kind of the next order of step is where that machines actually write their own algorithms, right? They start to write their own code, so they kind of take this next order of thought and agency, if you will. How do you guys think about that? You guys are way out ahead in the space, you have huge data set. You got great technology. Got tensorflow. When will the machines start writing their own A their own out rhythms? Well, and actually >>it's already starting there that, you know, for example, we have we have a product called Google Cloud. Ottawa. Mel Village basically takes in a data set, and then we find the best model to be able to match that data set. And so things like that that that are there already, but it's still very nascent. There's a lot more than that that can happen. And I think ultimately with with how it's used I think part of it is you have to start. Always look at the downside of automation. And what is what is the downside of a bad decision, whether it's the wrong algorithm that you create or a bad decision in that model? And so if the downside is really big, that's where you need to start to apply Human in the loop. And so, for example, in medicine. Hey, I could do amazing things to detect diseases, but you would want a doctor in the loop to be able to actually diagnose. And so you need tohave have that place in many situations to make sure that it's being applied well. >>But is that just today? Or is that tomorrow? Because, you know, with with exponential growth and and as fast as these things are growing, will there be a day where you don't necessarily need maybe need the doctor to communicate the news? Maybe there's some second order impacts in terms of how you deal with the family and, you know, kind of pros and cons of treatment options that are more emotional than necessarily mechanical, because it seems like eventually that the doctor has a role. But it isn't necessarily in accurately diagnosing a problem. >>I think >>I think for some things, absolutely over time the algorithms will get better and better, and you can rely on them and trust them more and more. But again, I think you have to look at the downside consequence that if there's a bad decision, what happens and how is that compared to what happens today? And so that's really where, where that is. So, for example, self driving cars, we will get to the point where cars are driving by themselves. There will be accidents, but the accident rate is gonna be much lower than what's there with humans today, and so that will get there. But it will take time. >>And there was a day when will be illegal for you to drive. You have manslaughter, right? >>I I believe absolutely there will be in and and I don't think it's that far off. Actually, >>wait for the day when I have my car take me up to Northern California with me. Sleepy. I've only lived that long. >>That's right. And work while you're while you're sleeping, right? Well, I want to thank everybody Aton for being on this panel. This has been super fun and these air really big issues. So I want to give you the final word will just give everyone kind of a final say and I just want to throw out their Mars law. People talk about Moore's law all the time. But tomorrow's law, which Gardner stolen made into the hype cycle, you know, is that we tend to overestimate in the short term, which is why you get the hype cycle and we turn. Tend to underestimate, in the long term the impacts of technology. So I just want it is you look forward in the future won't put a year number on it, you know, kind of. How do you see this rolling out? What do you excited about? What are you scared about? What should we be thinking about? We'll start with you, Bob. >>Yeah, you know, for me and, you know, the day of the terminus Heathrow. I don't know if it's 100 years or 1000 years. That day is coming. We will eventually build something that's in part of the human. I think the mission about the book, you know, you look like a thing and I love >>you. >>Type of thing that was written by someone who tried to train a I to basically pick up lines. Right? Cheesy pickup lines. Yeah, I'm not for sure. I'm gonna trust a I to help me in my pickup lines yet. You know I love you. Look at your thing. I love you. I don't know if they work. >>Yeah, but who would? Who would have guessed online dating is is what it is if you had asked, you know, 15 years ago. But I >>think yes, I think overall, yes, we will see the Terminator Cp through It was probably not in our lifetime, but it is in the future somewhere. A. I is definitely gonna be on par with the Internet cell phone, radio. It's gonna be a technology that's gonna be accelerating if you look where technology's been over last. Is this amazing to watch how fast things have changed in our lifetime alone, right? Yeah, we're just on this curve of technology accelerations. This in the >>exponential curves China. >>Yeah, I think the thing I'm most excited about for a I right now is the addition of creativity to a lot of our jobs. So ah, lot of we build an augmented writing product. And what we do is we look at the words that have happened in the world and their outcomes. And we tell you what words have impacted people in the past. Now, with that information, when you augment humans in that way, they get to be more creative. They get to use language that have never been used before. To communicate an idea. You can do this with any field you can do with composition of music. You can if you can have access as an individual, thio the data of a bunch of cultures the way that we evolved can change. So I'm most excited about that. I think I'm most concerned currently about the products that we're building Thio Give a I to people that don't understand how to use it or how to make sure they're making an ethical decision. So it is extremely easy right now to go on the Internet to build a model on a data set. And I'm not a specialist in data, right? And so I have no idea if I'm adding bias in or not, um and so it's It's an interesting time because we're in that middle area. Um, and >>it's getting loud, all right, Roger will throw with you before we have to cut out, or we're not gonna be able to hear anything. So I actually start every presentation out with a picture of the Mosaic browser, because what's interesting is I think that's where >>a eyes today compared to kind of weather when the Internet was around 1994 >>were just starting to see how a I can actually impact the average person. As a result, there's a lot of hype, but what I'm actually finding is that 70% of the company's I talked to the first question is, Why should I be using this? And what benefit does it give me? Why 70% ask you why? Yeah, and and what's interesting with that is that I think people are still trying to figure out what is this stuff good for? But to your point about the long >>run, and we underestimate the longer I think that every company out there and every product will be fundamentally transformed by eye over the course of the next decade, and it's actually gonna have a bigger impact on the Internet itself. And so that's really what we have to look forward to. >>All right again. Thank you everybody for participating. There was a ton of fun. Hope you had fun. And I look at the score sheet here. We've got Bob coming in and the bronze at 15 points. Rajan, it's 17 in our gold medal winner for the silver Bell. Is Sharna at 20 points. Again. Thank you. Uh, thank you so much and look forward to our next conversation. Thank Jeffrey Ake signing out from Caesar's Juniper. Next word unpacking. I Thanks for watching.