(inspirational music) >> theCUBE's live coverage is made possible by funding from Dell Technologies. Creating technologies that drive human progress. (uplifting electronic music) (crowd chattering in background) >> Hi everybody. We're back at the Fira in Barcelona. Winding up our four day wall-to-wall coverage of MWC23 theCUBE has been thrilled to cover the telco transformation. Dave Vellante with Dave Nicholson. Really excited to have NTT on. Shahid Ahmed is the Group EVP of New Ventures and Innovation at NTT in from Chicago. Welcome to Barcelona. Welcome to theCUBE. >> Thank you for having me over. >> So, really interesting title. You have, you know, people might not know NTT you know, huge Japan telco but a lot of other businesses, explain your business. >> So we do a lot of things. Most of us are known for our Docomo business in Japan. We have one of the largest wireless cellular carriers in the world. We serve most of Japan. Outside of Japan, we are B2B systems, integration, professional services company. So we offer managed services. We have data centers, we have undersea cables. We offer all kinds of outsourcing services. So we're a big company. >> So there's a narrative out there that says, you know, 5G, it's a lot of hype, not a lot of adoption. Nobody's ever going to make money at 5G. You have a different point of view, I understand. You're like leaning into 5G and you've actually got some traction there. Explain that. >> So 5G can be viewed from two lenses. One is just you and I using our cell phones and we get 5G coverage over it. And the other one is for businesses to use 5G, and we call that private 5G or enterprise grade 5G. Two very separate distinct things, but it is 5G in the end. Now the big debate here in Europe and US is how to monetize 5G. As a consumer, you and I are not going to pay extra for 5G. I mean, I haven't. I just expect the carrier to offer faster, cheaper services. And so would I pay extra? Not really. I just want a reliable network from my carrier. >> Paid up for the good camera though, didn't you? >> I did. (Dave and Dave laughing) >> I'm waiting for four cameras now. >> So the carriers are in this little bit of a pickle at the moment because they've just spent billions of dollars, not only on spectrum but the infrastructure needed to upgrade to 5G, yet nobody's willing to pay extra for that 5G service. >> Oh, right. >> So what do they do? And one idea is to look at enterprises, companies, industrial companies, manufacturing companies who want to build their own 5G networks to support their own use cases. And these use cases could be anything from automating the surveyor belt to cameras with 5G in it to AGVs. These are little carts running around warehouses picking up products and goods, but they have to be connected all the time. Wifi doesn't work all the time there. And so those businesses are willing to pay for 5G. So your question is, is there a business case for 5G? Yes. I don't think it's in the consumer side. I think it's in the business side. And that's where NTT is finding success. >> So you said, you know, how they going to make money, right? You very well described the telco dilemma. We heard earlier this week, you know, well, we could tax the OTT vendors, like Netflix of course shot back and said, "Well, we spent a lot of money on content. We're driving a lot of value. Why don't you help us pay for the content development?" Which is incredibly expensive. I think I heard we're going to tax the developers for API calls on the network. I'm not sure how well that's going to work out. Look at Twitter, you know, we'll see. And then yeah, there's the B2B piece. What's your take on, we heard the Orange CEO say, "We need help." You know, maybe implying we're going to tax the OTT vendors, but we're for net neutrality, which seems like it's completely counter-posed. What's your take on, you know, fair share in the network? >> Look, we've seen this debate unfold in the US for the last 10 years. >> Yeah. >> Tom Wheeler, the FCC chairman started that debate and he made great progress and open internet and net neutrality. The thing is that if you create a lane, a tollway, where some companies have to pay toll and others don't have to, you create an environment where the innovation could be stifled. Content providers may not appear on the scene anymore. And with everything happening around AI, we may see that backfire. So creating a toll for rich companies to be able to pay that toll and get on a faster speed internet, that may work some places may backfire in others. >> It's, you know, you're bringing up a great point. It's one of those sort of unintended consequences. You got to be be careful because the little guy gets crushed in that environment, and then what? Right? Then you stifle innovation. So, okay, so you're a fan of net neutrality. You think the balance that the US model, for a change, maybe the US got it right instead of like GDPR, who sort of informed the US on privacy, maybe the opposite on net neutrality. >> I think so. I mean, look, the way the US, particularly the FCC and the FTC has mandated these rules and regulation. I think it's a nice balance. FTC is all looking at big tech at the moment, but- >> Lena Khan wants to break up big tech. I mean for, you know, you big tech, boom, break 'em up, right? So, but that's, you know- >> That's a whole different story. >> Yeah. Right. We could talk about that too, if you want. >> Right. But I think that we have a balanced approach, a measured approach. Asking the content providers or the developers to pay for your innovative creative application that's on your phone, you know, that's asking for too much in my opinion. >> You know, I think you're right though. Government did do a good job with net neutrality in the US and, I mean, I'm just going to go my high horse for a second, so forgive me. >> Go for it. >> Market forces have always done a better job at adjudicating, you know, competition. Now, if a company's a monopoly, in my view they should be, you know, regulated, or at least penalized. Yeah, but generally speaking, you know the attack on big tech, I think is perhaps misplaced. I sat through, and the reason it's relevant to Mobile World Congress or MWC, is I sat through a Nokia presentation this week and they were talking about Bell Labs when United States broke up, you know, the US telcos, >> Yeah. >> Bell Labs was a gem in the US and now it's owned by Nokia. >> Yeah. >> Right? And so you got to be careful about, you know what you wish for with breaking up big tech. You got AI, you've got, you know, competition with China- >> Yeah, but the upside to breaking up Ma Bell was not just the baby Bells and maybe the stranded orphan asset of Bell Labs, but I would argue it led to innovation. I'm old enough to remember- >> I would say it made the US less competitive. >> I know. >> You were in junior high school, but I remember as an adult, having a rotary dial phone and having to pay for that access, and there was no such- >> Yeah, but they all came back together. The baby Bells are all, they got all acquired. And the cable company, it was no different. So I don't know, do you have a perspective of this? Because you know this better than I do. >> Well, I think look at Nokia, just they announced a whole new branding strategy and new brand. >> I like the brand. >> Yeah. And- >> It looks cool. >> But guess what? It's B2B oriented. >> (laughs) Yeah. >> It's no longer consumer, >> Right, yeah. >> because they felt that Nokia brand phone was sort of misleading towards a lot of business to business work that they do. And so they've oriented themselves to B2B. Look, my point is, the carriers and the service providers, network operators, and look, I'm a network operator, too, in Japan. We need to innovate ourselves. Nobody's stopping us from coming up with a content strategy. Nobody's stopping a carrier from building a interesting, new, over-the-top app. In fact, we have better control over that because we are closer to the customer. We need to innovate, we need to be more creative. I don't think taxing the little developer that's building a very innovative application is going to help in the long run. >> NTT Japan, what do they have a content play? I, sorry, I'm not familiar with it. Are they strong in content, or competitive like Netflix-like, or? >> We have relationships with them, and you remember i-mode? >> Yeah. Oh yeah, sure. >> Remember in the old days. I mean, that was a big hit. >> Yeah, yeah, you're right. >> Right? I mean, that was actually the original app marketplace. >> Right. >> And the application store. So, of course we've evolved from that and we should, and this is an evolution and we should look at it more positively instead of looking at ways to regulate it. We should let it prosper and let it see where- >> But why do you think that telcos generally have failed at content? I mean, AT&T is sort of the exception that proves the rule. I mean, they got some great properties, obviously, CNN and HBO, but generally it's viewed as a challenging asset and others have had to diversify or, you know, sell the assets. Why do you think that telcos have had such trouble there? >> Well, Comcast owns also a lot of content. >> Yeah. Yeah, absolutely. >> And I think, I think that is definitely a strategy that should be explored here in Europe. And I think that has been underexplored. I, in my opinion, I believe that every large carrier must have some sort of content strategy at some point, or else you are a pipe. >> Yeah. You lose touch with a customer. >> Yeah. And by the way, being a dump pipe is okay. >> No, it's a lucrative business. >> It's a good business. You just have to focus. And if you start to do a lot of ancillary things around it then you start to see the margins erode. But if you just focus on being a pipe, I think that's a very good business and it's very lucrative. Everybody wants bandwidth. There's insatiable demand for bandwidth all the time. >> Enjoy the monopoly, I say. >> Yeah, well, capital is like an organism in and of itself. It's going to seek a place where it can insert itself and grow. Do you think that the questions around fair share right now are having people wait in the wings to see what's going to happen? Because especially if I'm on the small end of creating content, creating services, and there's possibly a death blow to my fixed costs that could be coming down the line, I'm going to hold back and wait. Do you think that the answer is let's solve this sooner than later? What are your thoughts? >> I think in Europe the opinion has been always to go after the big tech. I mean, we've seen a lot of moves either through antitrust, or other means. >> Or the guillotine! >> That's right. (all chuckle) A guillotine. Yes. And I've heard those directly. I think, look, in the end, EU has to decide what's right for their constituents, the countries they operate, and the economy. Frankly, with where the economy is, you got recession, inflation pressures, a war, and who knows what else might come down the pipe. I would be very careful in messing with this equilibrium in this economy. Until at least we have gone through this inflation and recessionary pressure and see what happens. >> I, again, I think I come back to markets, ultimately, will adjudicate. I think what we're seeing with chatGPT is like a Netscape moment in some ways. And I can't predict what's going to happen, but I can predict that it's going to change the world. And there's going to be new disruptors that come about. That just, I don't think Amazon, Google, Facebook, Apple are going to rule the world forever. They're just, I guarantee they're not, you know. They'll make it through. But there's going to be some new companies. I think it might be open AI, might not be. Give us a plug for NTT at the show. What do you guys got going here? Really appreciate you coming on. >> Thank you. So, you know, we're showing off our private 5G network for enterprises, for businesses. We see this as a huge opportunities. If you look around here you've got Rohde & Schwarz, that's the industrial company. You got Airbus here. All the big industrial companies are here. Automotive companies and private 5G. 5G inside a factory, inside a hospital, a warehouse, a mining operation. That's where the dollars are. >> Is it a meaningful business for you today? >> It is. We just started this business only a couple of years ago. We're seeing amazing growth and I think there's a lot of good opportunities there. >> Shahid Ahmed, thanks so much for coming to theCUBE. It was great to have you. Really a pleasure. >> Thanks for having me over. Great questions. >> Oh, you're welcome. All right. For David Nicholson, Dave Vellante. We'll be back, right after this short break, from the Fira in Barcelona, MWC23. You're watching theCUBE. (uplifting electronic music)

(vibrant electronic music) >> Welcome back to SuperCloud2. You know, this event, and the Super Cloud initiative in general, it's an open industry-wide collaboration. Last August at SuperCloud22, we really honed in on the definition, which of course we've published. And there's this shared doc, which folks are still adding to and refining, in fact, just recently, Dr. Nelu Mihai added some critical points that really advanced some of the community's initial principles, and today at SuperCloud2, we're digging further into the topic with input from real world practitioners, and we're exploring that intersection of data, data mesh, and cloud, and importantly, the realities and challenges of deploying technology to drive new business capability, and I'm pleased to welcome Ash Naseer to the program. He's a Senior Director of Data Engineering at Warner Bros. Discovery. Ash, great to see you again, thanks so much for taking time with us. >> It's great to be back, these conversations are always very fun. >> I was so excited when we met last spring, I guess, so before we get started I wanted to play a clip from that conversation, it was June, it was at the Snowflake Summit in Las Vegas. And it's a comment that you made about your company but also data mesh. Guys, roll the clip. >> Yeah, so, when people think of Warner Bros., you always think of the movie studio. But we're more than that, right, I mean, you think of HBO, you think of TNT, you think of CNN. We have 30 plus brands in our portfolio, and each have their own needs. So the idea of a data mesh really helps us because what we can do is we can federate access across the company, so that CNN can work at their own pace, you know, when there's election season, they can ingest their own data. And they don't have to bump up against, as an example, HBO, if Game of Thrones is goin' on. >> So-- Okay, so that's pretty interesting, so you've got these sort of different groups that have different data requirements inside of your organization. Now data mesh, it's a relatively new concept, so you're kind of ahead of the curve. So Ash, my question is, when you think about getting value from data, and how that's changed over the past decade, you've had pre-Hadoop, Hadoop, what do you see that's changed, now you got the cloud coming in, what's changed? What had to be sort of fixed? What's working now, and where do you see it going? >> Yeah, so I feel like in the last decade, we've gone through quite a maturity curve. I actually like to say that we're in the golden age of data, because the tools and technology in the data space, particularly and then broadly in the cloud, they allow us to do things that we couldn't do way back when, like you suggested, back in the Hadoop era or even before that. So there's certainly a lot of maturity, and a lot of technology that has come about. So in terms of the good, bad, and ugly, so let me kind of start with the good, right? In terms of bringing value from the data, I really feel like we're in this place where the folks that are charged with unlocking that value from the data, they're actually spending the majority of their time actually doing that. And what do I mean by that? If you think about it, 10 years ago, the data scientist was the person that was going to sort of solve all of the data problems in a company. But what happened was, companies asked these data scientists to come in and do a multitude of things. And what these data scientists found out was, they were spending most of their time on, really, data wrangling, and less on actually getting the value out of the data. And in the last decade or so, I feel like we've made the shift, and we realize that data engineering, data management, data governance, those are as important practices as data science, which is sort of getting the value out of the data. And so what that has done is, it has freed up the data scientist and the business analyst and the data analyst, and the BI expert, to really focus on how to get value out of the data, and spend less time wrangling data. So I really think that that's the good. In terms of the bad, I feel like, there's a lot of legacy data platforms out there, and I feel like there's going to be a time where we'll be in that hybrid mode. And then the ugly, I feel like, with all the data and all the technology, creates another problem of itself. Because most companies don't have arms around their data, and making sure that they know who's using the data, what they're using for, and how can the company leverage the collective intelligence. That is a bigger problem to solve today than 10 years ago. And that's where technologies like the data mesh come in. >> Yeah, so when I think of data mesh, and I say, you're an early practitioner of data mesh, you mentioned legacy technology, so the concept of data mesh is inclusive. In theory anyway, you're supposed to be including the legacy technologies. Whether it's a data lake or data warehouse or Oracle or Snowflake or whatever it is. And when you think about Jamak Dagani's principles, it's domain-centric ownership, data as product. And that creates challenges around self-serve infrastructure and automated governance, and then when you start to combine these different technologies. You got legacy, you got cloud. Everything's different. And so you have to figure out how to deal with that, so my question is, how have you dealt with that, and what role has the cloud played in solving those problems, in particular, that self-serve infrastructure, and that automated governance, and where are we in terms of solving that problem from a practitioner's standpoint? >> Yeah, I always like to say that data is a team sport, and we should sort of think of it as such, and that's, I feel like, the key of the data mesh concept, is treating it as a team sport. A lot of people ask me, they're like, "Oh hey, Ash, I've heard about this thing called data mesh. "Where can I buy one?" or, "what's the technology that I use to get a data mesh? And the reality is that there isn't one technology, you can't really buy a data mesh. It's really a way of life, it's how organizations decide to approach data, like I said, back to a team sport analogy, making sure that everyone has the seat on the table, making sure that we embrace the fact that we have a lot of data, we have a lot of data problems to solve. And the way we'll be successful is to make everyone inclusive. You know, you think about the old days, Data silos or shadow IT, some might call it. That's been around for decades. And what hasn't changed was this notion that, hey, everything needs to be sort of managed centrally. But with the cloud and with the technologies that we have today, we have the right technology and the tooling to democratize that data, and democratize not only just the access, but also sort of building building blocks and sort of taking building blocks which are relevant to your product or your business. And adding to the overall data mesh. We've got all that technology. The challenge is for us to really embrace it, and make sure that we implement it from an organizational standpoint. >> So, thinking about super cloud, there's a layer that lives above the clouds and adds value. And you think about your brands you got 30 brands, you mentioned shadow IT. If, let's say, one of those brands, HBO or TNT, whatever. They want to go, "Hey, we really like Google's analytics tools," and they maybe go off and build something, I don't know if that's even allowed, maybe it's not. But then you build this data mesh. My question is around multi-cloud, cross cloud, super cloud if you will. Is that a advantage for you as a practitioner, or does that just make things more complicated? >> I really love the idea of a multi-cloud. I think it's great, I think that it should have been the norm, not the exception, I feel like people talk about it as if it's the exception. That should have been the case. I will say, though, I feel like multi-cloud should evolve organically, so back to your point about some of these different brands, and, you know, different brands or different business units. Or even in a merger and acquisitions situation, where two different companies or multiple different companies come together with different technology stacks. You know, I feel like that's an organic evolution, and making sure that we use the concepts and the technologies around the multi-cloud to bring everyone together. That's where we need to be, and again, it talks to the fact that each of those business units and each of those groups have their own unique needs, and we need to make sure that we embrace that and we enable that, rather than stifling everything. Now where I have a little bit of a challenge with the multi-cloud is when technology leaders try to build it by design. So there's a notion there that, "Hey, you need to sort of diversify "and don't put all your eggs in one basket." And so we need to have this multi-cloud thing. I feel like that is just sort of creating more complexity where it doesn't need to be, we can all sort of simplify our lives, but where it evolves organically, absolutely, I think that's the right way to go. >> But, so Ash, if it evolves organically don't you need some kind of cloud interpreter, to create a common experience across clouds, does that exist today? What are your thoughts on that? >> There is a lot of technology that exists today, and that helps go between these different clouds, a lot of these sort of cloud agnostic technologies that you talked about, the Snowflakes and the Databricks and so forth of the world, they operate in multiple clouds, they operate in multiple regions, within a given cloud and multiple clouds. So they span all of that, and they have the tools and technology, so, I feel like the tooling is there. There does need to be more of an evolution around the tooling and I think the market's need are going to dictate that, I feel like the market is there, they're asking for it, so, there's definitely going to be that evolution, but the technology is there, I think just making sure that we embrace that and we sort of embrace that as a challenge and not try to sort of shut all of that down and box everything into one. >> What's the biggest challenge, is it governance or security? Or is it more like you're saying, adoption, cultural? >> I think it's a combination of cultural as well as governance. And so, the cultural side I've talked about, right, just making sure that we give these different teams a seat at the table, and they actually bring that technology into the mix. And we use the modern tools and technologies to make sure that everybody sort of plays nice together. That is definitely, we have ways to go there. But then, in terms of governance, that is another big problem that most companies are just starting to wrestle with. Because like I said, I mean, the data silos and shadow IT, that's been around there, right? The only difference is that we're now sort of bringing everything together in a cloud environment, the collective organization has access to that. And now we just realized, oh we have quite a data problem at our hands, so how do we sort of organize this data, make sure that the quality is there, the trust is there. When people look at that data, a lot of those questions are now coming to the forefront because everything is sort of so transparent with the cloud, right? And so I feel like, again, putting in the right processes, and the right tooling to address that is going to be critical in the next years to come. >> Is sharing data across clouds, something that is valuable to you, or even within a single cloud, being able to share data. And my question is, not just within your organization, but even outside your organization, is that something that has sort of hit your radar or is it mature or is that something that really would add value to your business? >> Data sharing is huge, and again, this is another one of those things which isn't new. You know, I remember back in the '90s, when we had to share data externally, with our partners or our vendors, they used to physically send us stacks of these tapes, or physical media on some truck. And we've evolved since then, right, I mean, it went from that to sharing files online and so forth. But data sharing as a concept and as a concept which is now very frictionless, through these different technologies that we have today, that is very new. And that is something, like I said, it's always been going on. But that needs to be really embraced more as well. We as a company heavily leverage data sharing between our own different brands and business units, that helps us make that data mesh, so that when CNN, as an example, builds their own data model based on election data and the kinds of data that they need, compare that with other data in the rest of the company, sports, entertainment, and so forth and so on. Everyone has their unique data, but that data sharing capability brings it together wherever there is a need. So you think about having a Tiger Woods documentary, as an example, on HBO Max and making sure that you reach the audiences that are interested in golf and interested in sports and so forth, right? That all comes through the magic of data sharing, so, it's really critical, internally, for us. And then externally as well, because just understanding how our products are doing on our partners' networks and different distribution channels, that's important, and then just understanding how our consumers are consuming it off properties, right, I mean, we have brands that transcend just the screen, right? We have a lot of physical merchandise that you can buy in the store. So again, understanding who's buying the Batman action figures after the Batman movie was released, that's another critical insight. So it all gets enabled through data sharing, and something we rely heavily on. >> So I wanted to get your perspective on this. So I feel like the nirvana of data mesh is if I want to use Google BigQuery, an Oracle database, or a Microsoft database, or Snowflake, Databricks, Amazon, whatever. That that's a node on the mesh. And in the perfect world, you can share that data, it can be governed, I don't think we're quite there today, so. But within a platform, maybe it's within Google or within Amazon or within Snowflake or Databricks. If you're in that world, maybe even Oracle. You actually can do some levels of data sharing, maybe greater with some than others. Do you mandate as an organization that you have to use this particular data platform, or are you saying "Hey, we are architecting a data mesh for the future "where we believe the technology will support that," or maybe you've invented some technology that supports that today, can you help us understand that? >> Yeah, I always feel like mandate is a strong area, and it breeds the shadow IT and the data silos. So we don't mandate, we do make sure that there's a consistent set of governance rules, policies, and tooling that's there, so that everyone is on the same page. However, at the same time our focus is really operating in a federated way, that's been our solution, right? Is to make sure that we work within a common set of tooling, which may be different technologies, which in some cases may be different clouds. Although we're not that multi-cloud. So what we're trying to do is making sure that everyone who has that technology already built, as long as it sort of follows certain standards, it's modern, it has the capabilities that will eventually allow us to be successful and eventually allow for that data sharing, amongst those different nodes, as you put it. As long as that's the case, and as long as there's a governance layer, a master governance layer, where we know where all that data is and who has access to what and we can sort of be really confident about the quality of the data, as long as that case, our approach to that is really that federated approach. >> Sorry, did I hear you correctly, you're not multi-cloud today? >> Yeah, that's correct. There are certain spots where we use that, but by and large, we rely on a particular cloud, and that's just been, like I said, it's been the evolution, it was our evolution. We decided early on to focus on a single cloud, and that's the direction we've been going in. >> So, do you want to go to a multi-cloud, or, you mentioned organic before, if a business unit wants to go there, as long as they're adhering to those standards that you put out, maybe recommendations, that that's okay? I guess my question is, does that bring benefit to your business that you'd like to tap, or do you feel like it's not necessary? >> I'll go back to the point of, if it happens organically, we're going to be open about it. Obviously we'll have to look at every situations, not all clouds are created equal as well, so there's a number of different considerations. But by and large, when it happens organically, the key is time to value, right? How do you quickly bring those technologies in, as long as you could share the data, they're interconnected, they're secured, they're governed, we are confident on the quality, as long as those principles are met, we could definitely go in that direction. But by and large, we're sort of evolving in a singular direction, but even within a singular cloud, we're a global company. And we have audiences around the world, so making sure that even within a single cloud, those different regions interoperate as one, that's a bigger challenge that we're having to solve as well. >> Last question is kind of to the future of data and cloud and how it's going to evolve, do you see a day when companies like yours are increasingly going to be offering data, their software, services, and becoming more of a technology company, sort of pointing your tooling and your proprietary knowledge at the external world, as an opportunity, as a business opportunity? >> That's a very interesting concept, and I know companies have done that, and some of them have been extremely successful, I mean, Amazon is the biggest example that comes to mind, right-- >> Yeah. >> When they launched AWS, something that they had that expertise they had internally, and they offered it to the world as a product. But by and large, I think it's going to be far and few between, especially, it's going to be focused on companies that have technology as their DNA, or almost like in the technology sector, building technology. Most other companies have different markets that they are addressing. And in my opinion, a lot of these companies, what they're trying to do is really focus on the problems that we can solve for ourselves, I think there are more problems than we have people and expertise. So my guess is that most large companies, they're going to focus on solving their own problems. A few, like I said, more tech-focused companies, that would want to be in that business, would probably branch out, but by and large, I think companies will continue to focus on serving their customers and serving their own business. >> Alright, Ash, we're going to leave it there, Ash Naseer. Thank you so much for your perspectives, it was great to see you, I'm sure we'll see you face-to-face later on this year. >> This is great, thank you for having me. >> Ah, you're welcome, alright. Keep it right there for more great content from SuperCloud2. We'll be right back. (gentle percussive music)

>>From the Cube Studios in Palo Alto in Boston, bringing you data-driven insights from the cube and E T R. This is breaking analysis with Dave Valante. >>Making technology predictions in 2022 was tricky business, especially if you were projecting the performance of markets or identifying I P O prospects and making binary forecast on data AI and the macro spending climate and other related topics in enterprise tech 2022, of course was characterized by a seesaw economy where central banks were restructuring their balance sheets. The war on Ukraine fueled inflation supply chains were a mess. And the unintended consequences of of forced march to digital and the acceleration still being sorted out. Hello and welcome to this week's weekly on Cube Insights powered by E T R. In this breaking analysis, we continue our annual tradition of transparently grading last year's enterprise tech predictions. And you may or may not agree with our self grading system, but look, we're gonna give you the data and you can draw your own conclusions and tell you what, tell us what you think. >>All right, let's get right to it. So our first prediction was tech spending increases by 8% in 2022. And as we exited 2021 CIOs, they were optimistic about their digital transformation plans. You know, they rushed to make changes to their business and were eager to sharpen their focus and continue to iterate on their digital business models and plug the holes that they, the, in the learnings that they had. And so we predicted that 8% rise in enterprise tech spending, which looked pretty good until Ukraine and the Fed decided that, you know, had to rush and make up for lost time. We kind of nailed the momentum in the energy sector, but we can't give ourselves too much credit for that layup. And as of October, Gartner had it spending growing at just over 5%. I think it was 5.1%. So we're gonna take a C plus on this one and, and move on. >>Our next prediction was basically kind of a slow ground ball. The second base, if I have to be honest, but we felt it was important to highlight that security would remain front and center as the number one priority for organizations in 2022. As is our tradition, you know, we try to up the degree of difficulty by specifically identifying companies that are gonna benefit from these trends. So we highlighted some possible I P O candidates, which of course didn't pan out. S NQ was on our radar. The company had just had to do another raise and they recently took a valuation hit and it was a down round. They raised 196 million. So good chunk of cash, but, but not the i p O that we had predicted Aqua Securities focus on containers and cloud native. That was a trendy call and we thought maybe an M SS P or multiple managed security service providers like Arctic Wolf would I p o, but no way that was happening in the crummy market. >>Nonetheless, we think these types of companies, they're still faring well as the talent shortage in security remains really acute, particularly in the sort of mid-size and small businesses that often don't have a sock Lacework laid off 20% of its workforce in 2022. And CO C e o Dave Hatfield left the company. So that I p o didn't, didn't happen. It was probably too early for Lacework. Anyway, meanwhile you got Netscope, which we've cited as strong in the E T R data as particularly in the emerging technology survey. And then, you know, I lumia holding its own, you know, we never liked that 7 billion price tag that Okta paid for auth zero, but we loved the TAM expansion strategy to target developers beyond sort of Okta's enterprise strength. But we gotta take some points off of the failure thus far of, of Okta to really nail the integration and the go to market model with azero and build, you know, bring that into the, the, the core Okta. >>So the focus on endpoint security that was a winner in 2022 is CrowdStrike led that charge with others holding their own, not the least of which was Palo Alto Networks as it continued to expand beyond its core network security and firewall business, you know, through acquisition. So overall we're gonna give ourselves an A minus for this relatively easy call, but again, we had some specifics associated with it to make it a little tougher. And of course we're watching ve very closely this this coming year in 2023. The vendor consolidation trend. You know, according to a recent Palo Alto network survey with 1300 SecOps pros on average organizations have more than 30 tools to manage security tools. So this is a logical way to optimize cost consolidating vendors and consolidating redundant vendors. The E T R data shows that's clearly a trend that's on the upswing. >>Now moving on, a big theme of 2020 and 2021 of course was remote work and hybrid work and new ways to work and return to work. So we predicted in 2022 that hybrid work models would become the dominant protocol, which clearly is the case. We predicted that about 33% of the workforce would come back to the office in 2022 in September. The E T R data showed that figure was at 29%, but organizations expected that 32% would be in the office, you know, pretty much full-time by year end. That hasn't quite happened, but we were pretty close with the projection, so we're gonna take an A minus on this one. Now, supply chain disruption was another big theme that we felt would carry through 2022. And sure that sounds like another easy one, but as is our tradition, again we try to put some binary metrics around our predictions to put some meat in the bone, so to speak, and and allow us than you to say, okay, did it come true or not? >>So we had some data that we presented last year and supply chain issues impacting hardware spend. We said at the time, you can see this on the left hand side of this chart, the PC laptop demand would remain above pre covid levels, which would reverse a decade of year on year declines, which I think started in around 2011, 2012. Now, while demand is down this year pretty substantially relative to 2021, I D C has worldwide unit shipments for PCs at just over 300 million for 22. If you go back to 2019 and you're looking at around let's say 260 million units shipped globally, you know, roughly, so, you know, pretty good call there. Definitely much higher than pre covid levels. But so what you might be asking why the B, well, we projected that 30% of customers would replace security appliances with cloud-based services and that more than a third would replace their internal data center server and storage hardware with cloud services like 30 and 40% respectively. >>And we don't have explicit survey data on exactly these metrics, but anecdotally we see this happening in earnest. And we do have some data that we're showing here on cloud adoption from ET R'S October survey where the midpoint of workloads running in the cloud is around 34% and forecast, as you can see, to grow steadily over the next three years. So this, well look, this is not, we understand it's not a one-to-one correlation with our prediction, but it's a pretty good bet that we were right, but we gotta take some points off, we think for the lack of unequivocal proof. Cause again, we always strive to make our predictions in ways that can be measured as accurate or not. Is it binary? Did it happen, did it not? Kind of like an O K R and you know, we strive to provide data as proof and in this case it's a bit fuzzy. >>We have to admit that although we're pretty comfortable that the prediction was accurate. And look, when you make an hard forecast, sometimes you gotta pay the price. All right, next, we said in 2022 that the big four cloud players would generate 167 billion in IS and PaaS revenue combining for 38% market growth. And our current forecasts are shown here with a comparison to our January, 2022 figures. So coming into this year now where we are today, so currently we expect 162 billion in total revenue and a 33% growth rate. Still very healthy, but not on our mark. So we think a w s is gonna miss our predictions by about a billion dollars, not, you know, not bad for an 80 billion company. So they're not gonna hit that expectation though of getting really close to a hundred billion run rate. We thought they'd exit the year, you know, closer to, you know, 25 billion a quarter and we don't think they're gonna get there. >>Look, we pretty much nailed Azure even though our prediction W was was correct about g Google Cloud platform surpassing Alibaba, Alibaba, we way overestimated the performance of both of those companies. So we're gonna give ourselves a C plus here and we think, yeah, you might think it's a little bit harsh, we could argue for a B minus to the professor, but the misses on GCP and Alibaba we think warrant a a self penalty on this one. All right, let's move on to our prediction about Supercloud. We said it becomes a thing in 2022 and we think by many accounts it has, despite the naysayers, we're seeing clear evidence that the concept of a layer of value add that sits above and across clouds is taking shape. And on this slide we showed just some of the pickup in the industry. I mean one of the most interesting is CloudFlare, the biggest supercloud antagonist. >>Charles Fitzgerald even predicted that no vendor would ever use the term in their marketing. And that would be proof if that happened that Supercloud was a thing and he said it would never happen. Well CloudFlare has, and they launched their version of Supercloud at their developer week. Chris Miller of the register put out a Supercloud block diagram, something else that Charles Fitzgerald was, it was was pushing us for, which is rightly so, it was a good call on his part. And Chris Miller actually came up with one that's pretty good at David Linthicum also has produced a a a A block diagram, kind of similar, David uses the term metacloud and he uses the term supercloud kind of interchangeably to describe that trend. And so we we're aligned on that front. Brian Gracely has covered the concept on the popular cloud podcast. Berkeley launched the Sky computing initiative. >>You read through that white paper and many of the concepts highlighted in the Supercloud 3.0 community developed definition align with that. Walmart launched a platform with many of the supercloud salient attributes. So did Goldman Sachs, so did Capital One, so did nasdaq. So you know, sorry you can hate the term, but very clearly the evidence is gathering for the super cloud storm. We're gonna take an a plus on this one. Sorry, haters. Alright, let's talk about data mesh in our 21 predictions posts. We said that in the 2020s, 75% of large organizations are gonna re-architect their big data platforms. So kind of a decade long prediction. We don't like to do that always, but sometimes it's warranted. And because it was a longer term prediction, we, at the time in, in coming into 22 when we were evaluating our 21 predictions, we took a grade of incomplete because the sort of decade long or majority of the decade better part of the decade prediction. >>So last year, earlier this year, we said our number seven prediction was data mesh gains momentum in 22. But it's largely confined and narrow data problems with limited scope as you can see here with some of the key bullets. So there's a lot of discussion in the data community about data mesh and while there are an increasing number of examples, JP Morgan Chase, Intuit, H S P C, HelloFresh, and others that are completely rearchitecting parts of their data platform completely rearchitecting entire data platforms is non-trivial. There are organizational challenges, there're data, data ownership, debates, technical considerations, and in particular two of the four fundamental data mesh principles that the, the need for a self-service infrastructure and federated computational governance are challenging. Look, democratizing data and facilitating data sharing creates conflicts with regulatory requirements around data privacy. As such many organizations are being really selective with their data mesh implementations and hence our prediction of narrowing the scope of data mesh initiatives. >>I think that was right on J P M C is a good example of this, where you got a single group within a, within a division narrowly implementing the data mesh architecture. They're using a w s, they're using data lakes, they're using Amazon Glue, creating a catalog and a variety of other techniques to meet their objectives. They kind of automating data quality and it was pretty well thought out and interesting approach and I think it's gonna be made easier by some of the announcements that Amazon made at the recent, you know, reinvent, particularly trying to eliminate ET t l, better connections between Aurora and Redshift and, and, and better data sharing the data clean room. So a lot of that is gonna help. Of course, snowflake has been on this for a while now. Many other companies are facing, you know, limitations as we said here and this slide with their Hadoop data platforms. They need to do new, some new thinking around that to scale. HelloFresh is a really good example of this. Look, the bottom line is that organizations want to get more value from data and having a centralized, highly specialized teams that own the data problem, it's been a barrier and a blocker to success. The data mesh starts with organizational considerations as described in great detail by Ash Nair of Warner Brothers. So take a listen to this clip. >>Yeah, so when people think of Warner Brothers, you always think of like the movie studio, but we're more than that, right? I mean, you think of H B O, you think of t n t, you think of C N N. We have 30 plus brands in our portfolio and each have their own needs. So the, the idea of a data mesh really helps us because what we can do is we can federate access across the company so that, you know, CNN can work at their own pace. You know, when there's election season, they can ingest their own data and they don't have to, you know, bump up against, as an example, HBO if Game of Thrones is going on. >>So it's often the case that data mesh is in the eyes of the implementer. And while a company's implementation may not strictly adhere to Jamma Dani's vision of data mesh, and that's okay, the goal is to use data more effectively. And despite Gartner's attempts to deposition data mesh in favor of the somewhat confusing or frankly far more confusing data fabric concept that they stole from NetApp data mesh is taking hold in organizations globally today. So we're gonna take a B on this one. The prediction is shaping up the way we envision, but as we previously reported, it's gonna take some time. The better part of a decade in our view, new standards have to emerge to make this vision become reality and they'll come in the form of both open and de facto approaches. Okay, our eighth prediction last year focused on the face off between Snowflake and Databricks. >>And we realized this popular topic, and maybe one that's getting a little overplayed, but these are two companies that initially, you know, looked like they were shaping up as partners and they, by the way, they are still partnering in the field. But you go back a couple years ago, the idea of using an AW w s infrastructure, Databricks machine intelligence and applying that on top of Snowflake as a facile data warehouse, still very viable. But both of these companies, they have much larger ambitions. They got big total available markets to chase and large valuations that they have to justify. So what's happening is, as we've previously reported, each of these companies is moving toward the other firm's core domain and they're building out an ecosystem that'll be critical for their future. So as part of that effort, we said each is gonna become aggressive investors and maybe start doing some m and a and they have in various companies. >>And on this chart that we produced last year, we studied some of the companies that were targets and we've added some recent investments of both Snowflake and Databricks. As you can see, they've both, for example, invested in elation snowflake's, put money into Lacework, the Secur security firm, ThoughtSpot, which is trying to democratize data with ai. Collibra is a governance platform and you can see Databricks investments in data transformation with D B T labs, Matillion doing simplified business intelligence hunters. So that's, you know, they're security investment and so forth. So other than our thought that we'd see Databricks I p o last year, this prediction been pretty spot on. So we'll give ourselves an A on that one. Now observability has been a hot topic and we've been covering it for a while with our friends at E T R, particularly Eric Bradley. Our number nine prediction last year was basically that if you're not cloud native and observability, you are gonna be in big trouble. >>So everything guys gotta go cloud native. And that's clearly been the case. Splunk, the big player in the space has been transitioning to the cloud, hasn't always been pretty, as we reported, Datadog real momentum, the elk stack, that's open source model. You got new entrants that we've cited before, like observe, honeycomb, chaos search and others that we've, we've reported on, they're all born in the cloud. So we're gonna take another a on this one, admittedly, yeah, it's a re reasonably easy call, but you gotta have a few of those in the mix. Okay, our last prediction, our number 10 was around events. Something the cube knows a little bit about. We said that a new category of events would emerge as hybrid and that for the most part is happened. So that's gonna be the mainstay is what we said. That pure play virtual events are gonna give way to hi hybrid. >>And the narrative is that virtual only events are, you know, they're good for quick hits, but lousy replacements for in-person events. And you know that said, organizations of all shapes and sizes, they learn how to create better virtual content and support remote audiences during the pandemic. So when we set at pure play is gonna give way to hybrid, we said we, we i we implied or specific or specified that the physical event that v i p experience is going defined. That overall experience and those v i p events would create a little fomo, fear of, of missing out in a virtual component would overlay that serves an audience 10 x the size of the physical. We saw that really two really good examples. Red Hat Summit in Boston, small event, couple thousand people served tens of thousands, you know, online. Second was Google Cloud next v i p event in, in New York City. >>Everything else was, was, was, was virtual. You know, even examples of our prediction of metaverse like immersion have popped up and, and and, and you know, other companies are doing roadshow as we predicted like a lot of companies are doing it. You're seeing that as a major trend where organizations are going with their sales teams out into the regions and doing a little belly to belly action as opposed to the big giant event. That's a definitely a, a trend that we're seeing. So in reviewing this prediction, the grade we gave ourselves is, you know, maybe a bit unfair, it should be, you could argue for a higher grade, but the, but the organization still haven't figured it out. They have hybrid experiences but they generally do a really poor job of leveraging the afterglow and of event of an event. It still tends to be one and done, let's move on to the next event or the next city. >>Let the sales team pick up the pieces if they were paying attention. So because of that, we're only taking a B plus on this one. Okay, so that's the review of last year's predictions. You know, overall if you average out our grade on the 10 predictions that come out to a b plus, I dunno why we can't seem to get that elusive a, but we're gonna keep trying our friends at E T R and we are starting to look at the data for 2023 from the surveys and all the work that we've done on the cube and our, our analysis and we're gonna put together our predictions. We've had literally hundreds of inbounds from PR pros pitching us. We've got this huge thick folder that we've started to review with our yellow highlighter. And our plan is to review it this month, take a look at all the data, get some ideas from the inbounds and then the e t R of January surveys in the field. >>It's probably got a little over a thousand responses right now. You know, they'll get up to, you know, 1400 or so. And once we've digested all that, we're gonna go back and publish our predictions for 2023 sometime in January. So stay tuned for that. All right, we're gonna leave it there for today. You wanna thank Alex Myerson who's on production and he manages the podcast, Ken Schiffman as well out of our, our Boston studio. I gotta really heartfelt thank you to Kristen Martin and Cheryl Knight and their team. They helped get the word out on social and in our newsletters. Rob Ho is our editor in chief over at Silicon Angle who does some great editing for us. Thank you all. Remember all these podcasts are available or all these episodes are available is podcasts. Wherever you listen, just all you do Search Breaking analysis podcast, really getting some great traction there. Appreciate you guys subscribing. I published each week on wikibon.com, silicon angle.com or you can email me directly at david dot valante silicon angle.com or dm me Dante, or you can comment on my LinkedIn post. And please check out ETR AI for the very best survey data in the enterprise tech business. Some awesome stuff in there. This is Dante for the Cube Insights powered by etr. Thanks for watching and we'll see you next time on breaking analysis.

(upbeat music) >> Welcome back to theCUBE's continuing coverage of Snowflake Summit '22 live from Caesar's Forum in Las Vegas. I'm Lisa Martin, my cohost Dave Vellante, we've been here the last day and a half unpacking a lot of news, a lot of announcements, talking with customers and partners, and we have another great session coming for you next. We've got a customer and a partner talking tech and data mash. Please welcome Mitesh Shah, VP in market strategy at Elation. >> Great to be here. >> and Ash Naseer great, to have you, senior director of data engineering at Warner Brothers Discovery. Welcome guys. >> Thank you for having me. >> It's great to be back in person and to be able to really get to see and feel and touch this technology, isn't it? >> Yeah, it is. I mean two years or so. Yeah. Great to feel the energy in the conference center. >> Yeah. >> Snowflake was virtual, I think for two years and now it's great to kind of see the excitement firsthand. So it's wonderful. >> Th excitement, but also the boom and the number of customers and partners and people attending. They were saying the first, or the summit in 2019 had about 1900 attendees. And this is around 10,000. So a huge jump in a short time period. Talk a little bit about the Elation-Snowflake partnership and probably some of the acceleration that you guys have been experiencing as a Snowflake partner. >> Yeah. As a snowflake partner. I mean, Snowflake is an investor of us in Elation early last year, and we've been a partner for, for longer than that. And good news. We have been awarded Snowflake partner of the year for data governance, just earlier this week. And that's in fact, our second year in a row for winning that award. So, great news on that front as well. >> Repeat, congratulations. >> Repeat. Absolutely. And we're going to hope to make it a three-peat as well. And we've also been awarded industry competency badges in five different industries, those being financial services, healthcare, retail technology, and Median Telcom. >> Excellent. Okay. Going to right get into it. Data mesh. You guys actually have a data mesh and you've presented at the conference. So, take us back to the beginning. Why did you decide that you needed to implement something like data mesh? What was the impetus? >> Yeah. So when people think of Warner brothers, you always think of like the movie studio, but we're more than that, right? I mean, you think of HBO, you think of TNT, you think of CNN, we have 30 plus brands in our portfolio and each have their own needs. So the idea of a data mesh really helps us because what we can do is we can federate access across the company so that, you know, CNN can work at their own pace. You know, when there's election season, they can ingest their own data and they don't have to, you know, bump up against as an example, HBO, if Game of Thrones is going on. >> So, okay. So the, the impetus was to serve those lines of business better. Actually, given that you've got these different brands, it was probably easier than most companies. Cause if you're, let's say you're a big financial services company, and now you have to decide who owns what. CNN owns its own data products, HBO. Now, do they decide within those different brands, how to distribute even further? Or is it really, how deep have you gone in that decentralization? >> That's a great question. It's a very close partnership, because there are a number of data sets, which are used by all the brands, right? You think about people browsing websites, right? You know, CNN has a website, Warner brothers has a website. So for us to ingest that data for each of the brands to ingest that data separately, that means five different ways of doing things and you know, a big environment, right? So that is where our team comes into play. We ingest a lot of the common data sets, but like I said, any unique data sets, data sets regarding theatrical as an example, you know, Warner brothers does it themselves, you know, for streaming, HBO Max, does it themselves. So we kind of operate in partnership. >> So do you have a centralized data team and also decentralized data teams, right? >> That's right. >> So I love this conversation because that was heresy 10 years ago, five years ago, even, cause that's inefficient. But you've, I presume you've found that it's actually more productive in terms of the business output, explain that dynamic. >> You know, you bring up such a good point. So I, you know, I consider myself as one of the dinosaurs who started like 20 plus years ago in this industry. And back then, we were all taught to think of the data warehouse as like a monolithic thing. And the reason for that is the technology wasn't there. The technology didn't catch up. Now, 20 years later, the technology is way ahead, right? But like, our mindset's still the same because we think of data warehouses and data platforms still as a monolithic thing. But if you really sort of remove that sort of mental barrier, if you will, and if you start thinking about, well, how do I sort of, you know, federate everything and make sure that you let folks who are building, or are closest to the customer or are building their products, let them own that data and have a partnership. The results have been amazing. And if we were only sort of doing it as a centralized team, we would not be able to do a 10th of what we do today. So it's that massive scale in, in our company as well. >> And I should have clarified, when we talk about data mesh are we talking about the implementing in practice, the octagon sort of framework, or is this sort of your own sort of terminology? >> Well, so the interesting part is four years ago, we didn't have- >> It didn't exist. >> Yeah. It didn't exist. And, and so we, our principle was very simple, right? When we started out, we said, we want to make sure that our brands are able to operate independently with some oversight and guidance from our technology teams, right? That's what we set out to do. We did that with Snowflake by design because Snowflake allows us to, you know, separate those, those brands into different accounts. So that was done by design. And then the, the magic, I think, is the Snowflake data sharing where, which allows us to sort of bring data in here once, and then share it with whoever needs it. So think about HBO Max. On HBO Max, You not only have HBO Max content, but content from CNN, from Cartoon Network, from Warner Brothers, right? All the movies, right? So to see how The Batman movie did in theaters and then on streaming, you don't need, you know, Warner brothers doesn't need to ingest the same streaming data. HBO Max does it. HBO Max shares it with Warner brothers, you know, store once, share many times, and everyone works at their own pace. >> So they're building data products. Those data products are discoverable APIs, I presume, or I guess maybe just, I guess the Snowflake cloud, but very importantly, they're governed. And that's correct, where Elation comes in? >> That's precisely where Elation comes in, is where sort of this central flexible foundation for data governance. You know, you mentioned data mesh. I think what's interesting is that it's really an answer to the bottlenecks created by centralized IT, right? There's this notion of decentralizing that the data engineers and making the data domain owners, the people that know the data the best, have them be in control of publishing the data to the data consumers. There are other popular concepts actually happening right now, as we speak, around modern data stack. Around data fabric that are also in many ways underpinned by this notion of decentralization, right? These are concepts that are underpinned by decentralization and as the pendulum swings, sort of between decentralization and centralization, as we go back and forth in the world of IT and data, there are certain constants that need to be centralized over time. And one of those I believe is very much a centralized platform for data governance. And that's certainly, I think where we come in. Would love to hear more about how you use Elation. >> Yeah. So, I mean, elation helps us sort of, as you guys say, sort of, map, the treasure map of the data, right? So for consumers to find where their data is, that's where Elation helps us. It helps us with the data cataloging, you know, storing all the metadata and, you know, users can go in, they can sort of find, you know, the data that they need and they can also find how others are using data. So it's, there's a little bit of a crowdsourcing aspect that Elation helps us to do whereby you know, you can see, okay, my peer in the other group, well, that's how they use this piece of data. So I'm not going to spend hours trying to figure this out. You're going to use the query that they use. So yeah. >> So you have a master catalog, I presume. And then each of the brands has their own sub catalogs, is that correct? >> Well, for the most part, we have that master catalog and then the brands sort of use it, you know, separately themselves. The key here is all that catalog, that catalog isn't maintained by a centralized group as well, right? It's again, maintained by the individual teams and not only in the individual teams, but the folks that are responsible for the data, right? So I talked about the concept of crowdsourcing, whoever sort of puts the data in, has to make sure that they update the catalog and make sure that the definitions are there and everything sort of in line. >> So HBO, CNN, and each have their own, sort of access to their catalog, but they feed into the master catalog. Is that the right way to think about it? >> Yeah. >> Okay. And they have their own virtual data warehouses, right? They have ownership over that? They can spin 'em up, spin 'em down as they see fit? Right? And they're governed. >> They're governed. And what's interesting is it's not just governed, right? Governance is a, is a big word. It's a bit nebulous, but what's really being enabled here is this notion of self-service as well, right? There's two big sort of rockets that need to happen at the same time in any given organization. There's this notion that you want to put trustworthy data in the hands of data consumers, while at the same time mitigating risk. And that's precisely what Elation does. >> So I want to clarify this for the audience. So there's four principles of database. This came after you guys did it. And I wonder how it aligns. Domain ownership, give data, as you were saying to the, to the domain owners who have context, data as product, you guys are building data products, and that creates two problems. How do you give people self-service infrastructure and how do you automate governance? So the first two, great. But then it creates these other problems. Does that align with your philosophy? Where's alignment? What's different? >> Yeah. Data products is exactly where we're going. And that sort of, that domain based design, that's really key as well. In our business, you think about who the customer is, as an example, right? Depending on who you ask, it's going to be, the answer might be different, you know, to the movie business, it's probably going to be the person who watches a movie in a theater. To the streaming business, to HBO Max, it's the streamer, right? To others, someone watching live CNN on their TV, right? There's yet another group. Think about all the franchising we do. So you see Batman action figures and T-shirts, and Warner brothers branded stuff in stores, that's yet another business unit. But at the end of the day, it's not a different person, it's you and me, right? We do all these things. So the domain concept, make sure that you ingest data and you bring data relevant to the context, however, not sort of making it so stringent where it cannot integrate, and then you integrate it at a higher level to create that 360. >> And it's discoverable. So the point is, I don't have to go tap Ash on the shoulder, say, how do I get this data? Is it governed? Do I have access to it? Give me the rules of it. Just, I go grab it, right? And the system computationally automates whether or not I have access to it. And it's, as you say, self-service. >> In this case, exactly right. It enables people to just search for data and know that when they find the data, whether it's trustworthy or not, through trust flags, and the like, it's doing both of those things at the same time. >> How is it an enabler of solving some of the big challenges that the media and entertainment industry is going through? We've seen so much change the last couple of years. The rising consumer expectations aren't going to go back down. They're only going to come up. We want you to serve us up content that's relevant, that's personalized, that makes sense. I'd love to understand from your perspective, Mitesh, from an industry challenges perspective, how does this technology help customers like Warner Brothers Discovery, meet business customers, where they are and reduce the volume on those challenges? >> It's a great question. And as I mentioned earlier, we had five industry competency badges that were awarded to us by Snowflake. And one of those four, Median Telcom. And the reason for that is we're helping media companies understand their audiences better, and ultimately serve up better experiences for their audiences. But we've got Ash right here that can tell us how that's happening in practice. >> Yeah, tell us. >> So I'll share a story. I always like to tell stories, right? Once once upon a time before we had Elation in place, it was like, who you knew was how you got access to the data. So if I knew you and I knew you had access to a certain kind of data and your access to the right kind of data was based on the network you had at the company- >> I had to trust you. >> Yeah. >> I might not want to give up my data. >> That's it. And so that's where Elation sort of helps us democratize it, but, you know, puts the governance and controls, right? There are certain sensitive things as well, such as viewership, such as subscriber accounts, which are very important. So making sure that the right people have access to it, that's the other problem that Elation helps us solve. >> That's precisely part of our integration with Snowflake in particular, being able to define and manage policies within Elation. Saying, you know, certain people should have access to certain rows, doing column level masking. And having those policies actually enforced at the Snowflake data layer is precisely part of our value product. >> And that's automated. >> And all that's automated. Exactly. >> Right. So I don't have to think about it. I don't have to go through the tap on their shoulder. What has been the impact, Ash, on data quality as you've pushed it down into the domains? >> That's a great question. So it has definitely improved, but data quality is a very interesting subject, because back to my example of, you know, when we started doing things, we, you know, the centralized IT team always said, well, it has to be like this, Right? And if it doesn't fit in this, then it's bad quality. Well, sometimes context changes. Businesses change, right? You have to be able to react to it quickly. So making sure that a lot of that quality is managed at the decentralized level, at the place where you have that business context, that ensures you have the most up to date quality. We're talking about media industry changing so quickly. I mean, would we have thought three years ago that people would watch a lot of these major movies on streaming services? But here's the reality, right? You have to react and, you know, having it at that level just helps you react faster. >> So data, if I play that back, data quality is not a static framework. It's flexible based on the business context and the business owners can make those adjustments, cause they own the data. >> That's it. That's exactly it. >> That's awesome. Wow. That's amazing progress that you guys have made. >> In quality, if I could just add, it also just changes depending on where you are in your data pipeline stage, right? Data, quality data observability, this is a very fast evolving space at the moment, and if I look to my left right now, I bet you I can probably see a half-dozen quality observability vendors right now. And so given that and given the fact that Elation still is sort of a central hub to find trustworthy data, we've actually announced an open data quality initiative, allowing for best-of-breed data quality vendors to integrate with the platform. So whoever they are, whatever tool folks want to use, they can use that particular tool of choice. >> And this all runs in the cloud, or is it a hybrid sort of? >> Everything is in the cloud. We're all in the cloud. And you know, again, helps us go faster. >> Let me ask you a question. I could go on forever in this topic. One of the concepts that was put forth is whether it's a Snowflake data warehouse or a data bricks, data lake, or an Oracle data warehouse, they should all be inclusive. They should just be a node on the mesh. Like, wow, that sounds good. But I haven't seen it yet. Right? I'm guessing that Snowflake and Elation enable all the self-serve, all this automated governance, and that including those other items, it's got to be a one-off at this point in time. Do you ever see you expanding that scope or is it better off to just kind of leave it into the, the Snowflake data cloud? >> It's a good question. You know, I feel like where we're at today, especially in terms of sort of technology giving us so many options, I don't think there's a one size fits all. Right? Even though we are very heavily invested in Snowflake and we use Snowflake consistently across the organization, but you could, theoretically, could have an architecture that blends those two, right? Have different types of data platforms like a teradata or an Oracle and sort of bring it all together today. We have the technology, you know, that and all sorts of things that can make sure that you query on different databases. So I don't think the technology is the problem, I think it's the organizational mindset. I think that that's what gets in the way. >> Oh, interesting. So I was going to ask you, will hybrid tables help you solve that problem? And, maybe not, what you're saying, it's the organization that owns the Oracle database saying, Hey, we have our system. It processes, it works, you know, go away. >> Yeah. Well, you know, hybrid tables I think, is a great sort of next step in Snowflake's evolution. I think it's, in my opinion, I, think it's a game changer, but yeah. I mean, they can still exist. You could do hybrid tables right on Snowflake, or you could, you know, you could kind of coexist as well. >> Yeah. But, do you have a thought on this? >> Yeah, I do. I mean, we're always going to live in a time where you've got data distributed in throughout the organization and around the globe. And that could be even if you're all in on Snowflake, you could have data in Snowflake here, you could have data in Snowflake in EMEA and Europe somewhere. It could be anywhere. By the same token you might be using. Every organization is using on-premises systems. They have data, they naturally have data everywhere. And so, you know, this one solution to this is really centralizing, as I mentioned, not just governance, but also metadata about all of the data in your organization so that you can enable people to search and find and discover trustworthy data no matter where it is in your organization. >> Yeah. That's a great point. I mean, if you have the data about the data, then you can, you can treat these independent nodes. That's just that. Right? And maybe there's some advantages of putting it all in the Snowflake cloud, but to your point, organizationally, that's just not feasible. The whole, unfortunately, sorry, Snowflake, all the world's data is not going to go into Snowflake, but they play a key role in accelerating, what I'm hearing, your vision of data mesh. >> Yeah, absolutely. I think going forward in the future, we have to start thinking about data platforms as just one place where you sort of dump all the data. That's where the mesh concept comes in. It is going to be a mesh. It's going to be distributed and organizations have to be okay with that. And they have to embrace the tools. I mean, you know, Facebook developed a tool called Presto many years ago that that helps them solve exactly the same problem. So I think the technology is there. I think the organizational mindset needs to evolve. >> Yeah. Definitely. >> Culture. Culture is one of the hardest things to change. >> Exactly. >> Guys, this was a masterclass in data mesh, I think. Thank you so much for coming on talking. >> We appreciate it. Thank you so much. >> Of course. What Elation is doing with Snowflake and with Warner Brothers Discovery, Keep that content coming. I got a lot of stuff I got to catch up on watching. >> Sounds good. Thank you for having us. >> Thanks guys. >> Thanks, you guys. >> For Dave Vellante, I'm Lisa Martin. You're watching theCUBE live from Snowflake Summit '22. We'll be back after a short break. (upbeat music)

(upbeat music) >> Welcome back to theCUBE's continuing coverage of Snowflake Summit '22 live from Caesar's Forum in Las Vegas. I'm Lisa Martin, my cohost Dave Vellante, we've been here the last day and a half unpacking a lot of news, a lot of announcements, talking with customers and partners, and we have another great session coming for you next. We've got a customer and a partner talking tech and data mash. Please welcome Mitesh Shah, VP in market strategy at Elation. >> Great to be here. >> and Ash Naseer great, to have you, senior director of data engineering at Warner Brothers Discovery. Welcome guys. >> Thank you for having me. >> It's great to be back in person and to be able to really get to see and feel and touch this technology, isn't it? >> Yeah, it is. I mean two years or so. Yeah. Great to feel the energy in the conference center. >> Yeah. >> Snowflake was virtual, I think for two years and now it's great to kind of see the excitement firsthand. So it's wonderful. >> Th excitement, but also the boom and the number of customers and partners and people attending. They were saying the first, or the summit in 2019 had about 1900 attendees. And this is around 10,000. So a huge jump in a short time period. Talk a little bit about the Elation-Snowflake partnership and probably some of the acceleration that you guys have been experiencing as a Snowflake partner. >> Yeah. As a snowflake partner. I mean, Snowflake is an investor of us in Elation early last year, and we've been a partner for, for longer than that. And good news. We have been awarded Snowflake partner of the year for data governance, just earlier this week. And that's in fact, our second year in a row for winning that award. So, great news on that front as well. >> Repeat, congratulations. >> Repeat. Absolutely. And we're going to hope to make it a three-peat as well. And we've also been awarded industry competency badges in five different industries, those being financial services, healthcare, retail technology, and Median Telcom. >> Excellent. Okay. Going to right get into it. Data mesh. You guys actually have a data mesh and you've presented at the conference. So, take us back to the beginning. Why did you decide that you needed to implement something like data mesh? What was the impetus? >> Yeah. So when people think of Warner brothers, you always think of like the movie studio, but we're more than that, right? I mean, you think of HBO, you think of TNT, you think of CNN, we have 30 plus brands in our portfolio and each have their own needs. So the idea of a data mesh really helps us because what we can do is we can federate access across the company so that, you know, CNN can work at their own pace. You know, when there's election season, they can ingest their own data and they don't have to, you know, bump up against as an example, HBO, if Game of Thrones is going on. >> So, okay. So the, the impetus was to serve those lines of business better. Actually, given that you've got these different brands, it was probably easier than most companies. Cause if you're, let's say you're a big financial services company, and now you have to decide who owns what. CNN owns its own data products, HBO. Now, do they decide within those different brands, how to distribute even further? Or is it really, how deep have you gone in that decentralization? >> That's a great question. It's a very close partnership, because there are a number of data sets, which are used by all the brands, right? You think about people browsing websites, right? You know, CNN has a website, Warner brothers has a website. So for us to ingest that data for each of the brands to ingest that data separately, that means five different ways of doing things and you know, a big environment, right? So that is where our team comes into play. We ingest a lot of the common data sets, but like I said, any unique data sets, data sets regarding theatrical as an example, you know, Warner brothers does it themselves, you know, for streaming, HBO Max, does it themselves. So we kind of operate in partnership. >> So do you have a centralized data team and also decentralized data teams, right? >> That's right. >> So I love this conversation because that was heresy 10 years ago, five years ago, even, cause that's inefficient. But you've, I presume you've found that it's actually more productive in terms of the business output, explain that dynamic. >> You know, you bring up such a good point. So I, you know, I consider myself as one of the dinosaurs who started like 20 plus years ago in this industry. And back then, we were all taught to think of the data warehouse as like a monolithic thing. And the reason for that is the technology wasn't there. The technology didn't catch up. Now, 20 years later, the technology is way ahead, right? But like, our mindset's still the same because we think of data warehouses and data platforms still as a monolithic thing. But if you really sort of remove that sort of mental barrier, if you will, and if you start thinking about, well, how do I sort of, you know, federate everything and make sure that you let folks who are building, or are closest to the customer or are building their products, let them own that data and have a partnership. The results have been amazing. And if we were only sort of doing it as a centralized team, we would not be able to do a 10th of what we do today. So it's that massive scale in, in our company as well. >> And I should have clarified, when we talk about data mesh are we talking about the implementing in practice, the octagon sort of framework, or is this sort of your own sort of terminology? >> Well, so the interesting part is four years ago, we didn't have- >> It didn't exist. >> Yeah. It didn't exist. And, and so we, our principle was very simple, right? When we started out, we said, we want to make sure that our brands are able to operate independently with some oversight and guidance from our technology teams, right? That's what we set out to do. We did that with Snowflake by design because Snowflake allows us to, you know, separate those, those brands into different accounts. So that was done by design. And then the, the magic, I think, is the Snowflake data sharing where, which allows us to sort of bring data in here once, and then share it with whoever needs it. So think about HBO Max. On HBO Max, You not only have HBO Max content, but content from CNN, from Cartoon Network, from Warner Brothers, right? All the movies, right? So to see how The Batman movie did in theaters and then on streaming, you don't need, you know, Warner brothers doesn't need to ingest the same streaming data. HBO Max does it. HBO Max shares it with Warner brothers, you know, store once, share many times, and everyone works at their own pace. >> So they're building data products. Those data products are discoverable APIs, I presume, or I guess maybe just, I guess the Snowflake cloud, but very importantly, they're governed. And that's correct, where Elation comes in? >> That's precisely where Elation comes in, is where sort of this central flexible foundation for data governance. You know, you mentioned data mesh. I think what's interesting is that it's really an answer to the bottlenecks created by centralized IT, right? There's this notion of decentralizing that the data engineers and making the data domain owners, the people that know the data the best, have them be in control of publishing the data to the data consumers. There are other popular concepts actually happening right now, as we speak, around modern data stack. Around data fabric that are also in many ways underpinned by this notion of decentralization, right? These are concepts that are underpinned by decentralization and as the pendulum swings, sort of between decentralization and centralization, as we go back and forth in the world of IT and data, there are certain constants that need to be centralized over time. And one of those I believe is very much a centralized platform for data governance. And that's certainly, I think where we come in. Would love to hear more about how you use Elation. >> Yeah. So, I mean, elation helps us sort of, as you guys say, sort of, map, the treasure map of the data, right? So for consumers to find where their data is, that's where Elation helps us. It helps us with the data cataloging, you know, storing all the metadata and, you know, users can go in, they can sort of find, you know, the data that they need and they can also find how others are using data. So it's, there's a little bit of a crowdsourcing aspect that Elation helps us to do whereby you know, you can see, okay, my peer in the other group, well, that's how they use this piece of data. So I'm not going to spend hours trying to figure this out. You're going to use the query that they use. So yeah. >> So you have a master catalog, I presume. And then each of the brands has their own sub catalogs, is that correct? >> Well, for the most part, we have that master catalog and then the brands sort of use it, you know, separately themselves. The key here is all that catalog, that catalog isn't maintained by a centralized group as well, right? It's again, maintained by the individual teams and not only in the individual teams, but the folks that are responsible for the data, right? So I talked about the concept of crowdsourcing, whoever sort of puts the data in, has to make sure that they update the catalog and make sure that the definitions are there and everything sort of in line. >> So HBO, CNN, and each have their own, sort of access to their catalog, but they feed into the master catalog. Is that the right way to think about it? >> Yeah. >> Okay. And they have their own virtual data warehouses, right? They have ownership over that? They can spin 'em up, spin 'em down as they see fit? Right? And they're governed. >> They're governed. And what's interesting is it's not just governed, right? Governance is a, is a big word. It's a bit nebulous, but what's really being enabled here is this notion of self-service as well, right? There's two big sort of rockets that need to happen at the same time in any given organization. There's this notion that you want to put trustworthy data in the hands of data consumers, while at the same time mitigating risk. And that's precisely what Elation does. >> So I want to clarify this for the audience. So there's four principles of database. This came after you guys did it. And I wonder how it aligns. Domain ownership, give data, as you were saying to the, to the domain owners who have context, data as product, you guys are building data products, and that creates two problems. How do you give people self-service infrastructure and how do you automate governance? So the first two, great. But then it creates these other problems. Does that align with your philosophy? Where's alignment? What's different? >> Yeah. Data products is exactly where we're going. And that sort of, that domain based design, that's really key as well. In our business, you think about who the customer is, as an example, right? Depending on who you ask, it's going to be, the answer might be different, you know, to the movie business, it's probably going to be the person who watches a movie in a theater. To the streaming business, to HBO Max, it's the streamer, right? To others, someone watching live CNN on their TV, right? There's yet another group. Think about all the franchising we do. So you see Batman action figures and T-shirts, and Warner brothers branded stuff in stores, that's yet another business unit. But at the end of the day, it's not a different person, it's you and me, right? We do all these things. So the domain concept, make sure that you ingest data and you bring data relevant to the context, however, not sort of making it so stringent where it cannot integrate, and then you integrate it at a higher level to create that 360. >> And it's discoverable. So the point is, I don't have to go tap Ash on the shoulder, say, how do I get this data? Is it governed? Do I have access to it? Give me the rules of it. Just, I go grab it, right? And the system computationally automates whether or not I have access to it. And it's, as you say, self-service. >> In this case, exactly right. It enables people to just search for data and know that when they find the data, whether it's trustworthy or not, through trust flags, and the like, it's doing both of those things at the same time. >> How is it an enabler of solving some of the big challenges that the media and entertainment industry is going through? We've seen so much change the last couple of years. The rising consumer expectations aren't going to go back down. They're only going to come up. We want you to serve us up content that's relevant, that's personalized, that makes sense. I'd love to understand from your perspective, Mitesh, from an industry challenges perspective, how does this technology help customers like Warner Brothers Discovery, meet business customers, where they are and reduce the volume on those challenges? >> It's a great question. And as I mentioned earlier, we had five industry competency badges that were awarded to us by Snowflake. And one of those four, Median Telcom. And the reason for that is we're helping media companies understand their audiences better, and ultimately serve up better experiences for their audiences. But we've got Ash right here that can tell us how that's happening in practice. >> Yeah, tell us. >> So I'll share a story. I always like to tell stories, right? Once once upon a time before we had Elation in place, it was like, who you knew was how you got access to the data. So if I knew you and I knew you had access to a certain kind of data and your access to the right kind of data was based on the network you had at the company- >> I had to trust you. >> Yeah. >> I might not want to give up my data. >> That's it. And so that's where Elation sort of helps us democratize it, but, you know, puts the governance and controls, right? There are certain sensitive things as well, such as viewership, such as subscriber accounts, which are very important. So making sure that the right people have access to it, that's the other problem that Elation helps us solve. >> That's precisely part of our integration with Snowflake in particular, being able to define and manage policies within Elation. Saying, you know, certain people should have access to certain rows, doing column level masking. And having those policies actually enforced at the Snowflake data layer is precisely part of our value product. >> And that's automated. >> And all that's automated. Exactly. >> Right. So I don't have to think about it. I don't have to go through the tap on their shoulder. What has been the impact, Ash, on data quality as you've pushed it down into the domains? >> That's a great question. So it has definitely improved, but data quality is a very interesting subject, because back to my example of, you know, when we started doing things, we, you know, the centralized IT team always said, well, it has to be like this, Right? And if it doesn't fit in this, then it's bad quality. Well, sometimes context changes. Businesses change, right? You have to be able to react to it quickly. So making sure that a lot of that quality is managed at the decentralized level, at the place where you have that business context, that ensures you have the most up to date quality. We're talking about media industry changing so quickly. I mean, would we have thought three years ago that people would watch a lot of these major movies on streaming services? But here's the reality, right? You have to react and, you know, having it at that level just helps you react faster. >> So data, if I play that back, data quality is not a static framework. It's flexible based on the business context and the business owners can make those adjustments, cause they own the data. >> That's it. That's exactly it. >> That's awesome. Wow. That's amazing progress that you guys have made. >> In quality, if I could just add, it also just changes depending on where you are in your data pipeline stage, right? Data, quality data observability, this is a very fast evolving space at the moment, and if I look to my left right now, I bet you I can probably see a half-dozen quality observability vendors right now. And so given that and given the fact that Elation still is sort of a central hub to find trustworthy data, we've actually announced an open data quality initiative, allowing for best-of-breed data quality vendors to integrate with the platform. So whoever they are, whatever tool folks want to use, they can use that particular tool of choice. >> And this all runs in the cloud, or is it a hybrid sort of? >> Everything is in the cloud. We're all in the cloud. And you know, again, helps us go faster. >> Let me ask you a question. I could go on forever in this topic. One of the concepts that was put forth is whether it's a Snowflake data warehouse or a data bricks, data lake, or an Oracle data warehouse, they should all be inclusive. They should just be a node on the mesh. Like, wow, that sounds good. But I haven't seen it yet. Right? I'm guessing that Snowflake and Elation enable all the self-serve, all this automated governance, and that including those other items, it's got to be a one-off at this point in time. Do you ever see you expanding that scope or is it better off to just kind of leave it into the, the Snowflake data cloud? >> It's a good question. You know, I feel like where we're at today, especially in terms of sort of technology giving us so many options, I don't think there's a one size fits all. Right? Even though we are very heavily invested in Snowflake and we use Snowflake consistently across the organization, but you could, theoretically, could have an architecture that blends those two, right? Have different types of data platforms like a teradata or an Oracle and sort of bring it all together today. We have the technology, you know, that and all sorts of things that can make sure that you query on different databases. So I don't think the technology is the problem, I think it's the organizational mindset. I think that that's what gets in the way. >> Oh, interesting. So I was going to ask you, will hybrid tables help you solve that problem? And, maybe not, what you're saying, it's the organization that owns the Oracle database saying, Hey, we have our system. It processes, it works, you know, go away. >> Yeah. Well, you know, hybrid tables I think, is a great sort of next step in Snowflake's evolution. I think it's, in my opinion, I, think it's a game changer, but yeah. I mean, they can still exist. You could do hybrid tables right on Snowflake, or you could, you know, you could kind of coexist as well. >> Yeah. But, do you have a thought on this? >> Yeah, I do. I mean, we're always going to live in a time where you've got data distributed in throughout the organization and around the globe. And that could be even if you're all in on Snowflake, you could have data in Snowflake here, you could have data in Snowflake in EMEA and Europe somewhere. It could be anywhere. By the same token you might be using. Every organization is using on-premises systems. They have data, they naturally have data everywhere. And so, you know, this one solution to this is really centralizing, as I mentioned, not just governance, but also metadata about all of the data in your organization so that you can enable people to search and find and discover trustworthy data no matter where it is in your organization. >> Yeah. That's a great point. I mean, if you have the data about the data, then you can, you can treat these independent nodes. That's just that. Right? And maybe there's some advantages of putting it all in the Snowflake cloud, but to your point, organizationally, that's just not feasible. The whole, unfortunately, sorry, Snowflake, all the world's data is not going to go into Snowflake, but they play a key role in accelerating, what I'm hearing, your vision of data mesh. >> Yeah, absolutely. I think going forward in the future, we have to start thinking about data platforms as just one place where you sort of dump all the data. That's where the mesh concept comes in. It is going to be a mesh. It's going to be distributed and organizations have to be okay with that. And they have to embrace the tools. I mean, you know, Facebook developed a tool called Presto many years ago that that helps them solve exactly the same problem. So I think the technology is there. I think the organizational mindset needs to evolve. >> Yeah. Definitely. >> Culture. Culture is one of the hardest things to change. >> Exactly. >> Guys, this was a masterclass in data mesh, I think. Thank you so much for coming on talking. >> We appreciate it. Thank you so much. >> Of course. What Elation is doing with Snowflake and with Warner Brothers Discovery, Keep that content coming. I got a lot of stuff I got to catch up on watching. >> Sounds good. Thank you for having us. >> Thanks guys. >> Thanks, you guys. >> For Dave Vellante, I'm Lisa Martin. You're watching theCUBE live from Snowflake Summit '22. We'll be back after a short break. (upbeat music)

>>The cube presents, Coon and cloud native con Europe, 2022, brought to you by red hat, the cloud native computing foundation and its ecosystem partners. >>Welcome to Licia Spain in Coon and cloud native con Europe, 2022. I'm Keith Townsend, along with my cohot Paul Gillon, who's been putting in some pretty good work talking to incredible people. And we have, I don't wanna call, heard the face of CNCF, but you kind of introduced me to, you don't know this, but you know, charmer executive director of CNCF. You introduced me to Kuan at Cuan San Diego's my one of my first CU coupons. And I was trying to get my bearings about me and you're on stage and I'm like, okay. Uh, she looks like a reasonable person. This might be a reasonable place to learn about cloud native. Welcome to the show. >>Thank you so much for having me. And that's so nice to hear >><laugh> it is an amazing show, roughly 7,500 people. >>Yes, that's right. Sold out >>Sold. That's a big show. And with that comes, you know, uh, so someone told me, uh, CNCF is an outstanding organization, which it, which it is you're the executive director. And I told them, you know what, that's like being the president of the United States without having air force one. <laugh> like you get home. I dunno >>About that. You >>Get, no, you get all of the, I mean, 7,500 people from across, literally across the world. That's true at Europe. We're in Europe, we're in, we're coming out of times that have been, you know, it can't be overstated. It, this, this is unlike any other times. >>Yes, absolutely >>Difficult decisions. There was a whole co uh, uh, I don't know the term, uh, uh, cuffa uh, or blow up about mask versus no mask. How do you manage just, just the diversity of the community. >>That is such a great question, because I, as I mentioned in my keynote a little bit, right? At this point, we're a community of what, 7.1 million developers. That's a really big group. And so when we think about how should we manage the diversity, the way I see it, it's essential to treat each other with kindness, professionalism, and respect. Now that's easy to say, right. Because it sounds great. Right. Old paper is awesome. Yeah. Yeah. Great >>Concept. 0.1 million people later. >><laugh> exactly. And so, uh, this is why like, uh, I phoned a friend on stage and, um, van Jones came and spoke with us. Who's the renowned CNN contributor, uh, commentator, sorry. And his advice was very much that in such a diverse community, there's always gonna be lots of perspectives, lots opinions. And we need to a always bring the version of ourselves, which we think will empower this ecosystem, BEC what are, what we are doing. If everybody did that, is that gonna be a good thing or a bad thing? And the other is we need to give each other space and grace, um, space to do what we need to do. Grace. If there are mistakes, if there are challenges. And so those are, those are some good principles for us to live by. And I think that in terms of how CNCF tries to enable the diversity, it's by really trying to hear from everybody possible, the vocal loud voices, as well as the folks who you need to reach out a little bit, pull in a little bit. So it's an ongoing, it's an ongoing challenge that we do our best with. >>How do you balance? And I've been to a lot of trade shows and conferences over the years, their trade organizers are very coin operated. You know, they're there, they're there for the money. Yeah. <laugh> and you have traditional trade shows and you have a situation here where an open source community that is motivated by very different, um, principles, but you need to make money. You need the show to be profitable. Uh, you need to sell some sponsorships, but you also need to keep it available and open to the people who, who don't have the big budgets. How are you balancing that? >>So I would actually like to, uh, share something that may not be obvious, which is that we don't actually do the shows to make money. We, um, as you said, like, uh, a lot of trade shows are coin up and the goal there is like, um, well actually they're different kinds of, I think if it's an independent event organization, it can be like, Hey, let's make as much revenue as possible. If it's part of a large, um, large company, like, like cloud provider, et cetera, the events tend to be lost leaders because they're like lead gen, I think, >>But they're, they're lost leaders, but they're profit makers ultimately >>Long term. Yeah. Yeah. It's like top of the funnel. I, I guess for us, we are only doing the events to enable the community and bring people from different companies together. So our goal is to try and break even <laugh> >>Well, that's, that's laudable. Um, the, how big does it get though? I mean, you're at the point with 7,500 attendees here where you're on the cusp of being a really big event, uh, would you limit it size eventually? Or are you just gonna let this thing run? Its course. >>So our inherent belief is that we want to be accessible and open to more and more and more people because the mission is to make cloud native ubiquitous. Right. Uh, and so that means we are excited about growth. We are excited about opening the doors for as everyone, but I think actually the one, one good thing that came out of this pandemic is that we've become a lot more comfortable with hybrid. So we have a virtual component and an in-person component. So combining that, I think makes it well, it's very challenging cause like running to events, but it's also like, it can scale a little bit better. And then if the numbers increase from like, if they double, for example, we're still, I think we're still not in the realm of south by Southwest, which, which feels like, oh, that's the step function difference. So linear increases in number of attendees, I think is a good thing. If, and when we get to the point where it's, um, you know, exponential growth at that point, we have to think about, um, a completely different event really. Right, >>Right. So 7 billion people in the world approaching 8 billion, 7.1 members in the community. Technology is obviously an enabler where I it's enabled me to, to be here and Licia Spain experiencing this beautiful city. There's so much work to be done. What mm-hmm <affirmative> what is the role of CNCF in providing access to education and technology for the rest of the world? >>Absolutely. So, you know, one of the key, uh, areas we focus on is learning and development in supporting the ecosystem in learners beginners to start their cloud native journey or expand their cloud native journey with training certifications, and actually shared this in the keynote every year. Uh, the increase in number of people taking certifications grows by 216% year over year growth. It's a lot, right? And every week about a thousand people are taking a certification exam. So, and we set that up primarily to bring people in and that's one of our more successful initiatives, but we do so many, we do mentorship programs, internship programs. We, uh, a lot of diversity scholarships, these events, it all kind of comes together to support the ecosystem, to grow >>The turning away from the events, uh, toward just toward the CNCF Brit large, you have a growing number of projects. The, the number of projects within CNCF is becoming kind of overwhelming. Is there an upper threshold at which you would, do you tighten the, the limits on, on what projects you will incubate or how big does that tent become? >>Right. I think, you know, when we had 50 projects, we were feeling overwhelmed then too, but we seem to have cop just fine. And there's a reason for that. The reason is that cloud native has been growing so fast with the world. It's a representative of what's going on in our world over the course of the pandemic. As you know, every company became a technology company. People had to like double their engineering staffs over without anybody ever having met in person mm-hmm <affirmative> right. And when that kind of change is going around the world cloud needing be being the scaffolding of how people build and deploy modern software just grew really with it. And the use cases we needed to support grew. That's why the types of projects and kinds of projects is growing. So there's a method. There's a reason to the madness I should say. And I think, um, as the world and, uh, the landscape of technology evolves cloud native will, will evolve and keep developing in either into new projects or consolidation of projects and everything is on the table. >>So I think one of these perceptions Riley Arone is that CNCF is kind of where the big people go to play. If you're a small project and you're looking at CNCF, you're thinking one day I'll get big enough. Like how should small project leaders or leaders of small projects, how should they engage CNCF? >>Totally. And, you know, I want to really change this narrative because, um, in CNCF we have three tiers of projects. There's the graduated ones, which are at the top. These are the most mature ones we really believe and put our sand behind them. They, uh, then there's the incubating projects, which are pretty solid technologies with good usage that are getting there. And then there's the sandbox, which is literally a sandbox and op open ground for innovation. And the bar to entry is low in that it's, uh, easy to apply. There's a mass boat to get you in. And once you're in, you have a neutral IP zone created by being a CNCF project that you can attract more maintainers, more companies can start collaborating. So we, we become an enabler for the small projects, so everybody should know that >>FYI. Yeah. So I won't be interested to know how that, so I have an idea. So let's say I don't have an idea, but let's say that idea have, >>I'm sure you have an idea. <laugh>, I'm >>Sure I have idea. And, and I just don't have the infrastructure to run a project. I need help, but I think it it's going to solve a pro problem. Yeah. What's that application process like, >>So, okay. So you apply after you already have let's a GitHub repo. Okay. Yeah. >>So you, I have a GI help repo. >>Yeah. As in like your pro you've started the project, you started the coding, you've like, put it out there on GitHub, you have something going. And so it's not at just ideal level. Mm-hmm, <affirmative>, it's at like early stage of execution level. Um, and so, and then your question was, how do you apply? >>Yeah. So how do I, so I have, let's say that, uh, let, let's talk about something I'm thinking about doing, and I actually do, is that we're thinking about doing a open store, a cloud native framework for people migrating to the public cloud, to, or to cloud native. There's just not enough public information about that. And I'm like, you know what? I wanna contribute what I know to it. So that's a project in itself, not necessarily a software project, but a IP project, or let's say I have a tool to do that migration. And I put that up on my GitHub report. I want people to iterate on that tool. >>Right. So it would be a simple process of literally there is when you go to, um, our, uh, online, uh, materials, there's a simple process for sandbox where you fill a Google form, where you put in your URL, explain what you're doing, or some basic information hit submit. And we batch process these, um, about every once a month, I think. And, uh, the TC looks at the, what you've filled in, takes a group vote and goes from there. >>When about your operating model, I mean, do, do you, you mentioned you don't look to make a profit in this show. Do you look, and I wanna be sure CNCF is a non-profit, is that correct? Correct. Do you look, what models do you look at in determining your own governance? Do you look at a commercial business? Do you look at a nonprofit? Um, like of ourselves? Yeah. What's your model for how you run CNCF. >>Oh, okay. So it's a nonprofit, as I said, and our model is very simple. We want to raise the funds that we are able to raise in order to then invest them into community initiatives that play the supporter enabler role to all these projects we just talked about. We're not, we are never the project. We are the top cheerleader of the project. Think of us like that. And in terms of, um, but interestingly, unlike, I, I mean, I don't know much about other found, uh, nonprofit session compare, but interestingly, the donating companies are relevant, not just because of their cash that they have put in, but because those companies are part of this ecosystem and they need to, um, them being in this ecosystem, they help create content around cloud native. They, they do more than give us money. And that's why we really like our members, uh, they'll provide contributing engineers to projects. They will help us with marketing with case studies and interviews and all of that. And so it, it becomes this like healthy cycle of it starts with someone donating to become a member, but they end up doing so many different things. Mm-hmm <affirmative> and ultimately the goal is make cloud native ubiquitous and all this goes towards >>That. So talk to me about conflict resolution, because there's some really big projects in CNC, but only some stuff that is changed, literally changing the world, but there's competing interest between some of the projects. I mean, you, you, there there's, if you look at service mesh, there's a lot of service mesh solutions Uhhuh. Yes. And there's just different visions. Where's the CNCF and, and kind of just making sure the community aspect is thought across all of the different or considered across all the different projects as they have the let's say inevitably bump heads. >>Yeah. So by design CNCF was never meant to be a king maker where you picked one project. Right. And I think that's been working out really well because, um, one is when you accept a project, you're not a hundred percent sure that specific one is gonna take over that technology space. Right. So we're leaving it open to see who works it out. The second is that as every company is becoming a technology company, use cases are different. So a service mesh service mesh a might work really well for my company, but it really may not be a fit for your code base. And so the diversity of options is actually a really good thing. >>So talk to me about, uh, saw an interesting note coming out of the keynote yesterday, 65% of the participants here at CU con are new to Kuan. I'm like, oh, I'm a, I'm a vet. You are, I went to two or three before this. So O GE yeah, OG actually, that's what I tweeted OG of Kuan, but, uh, who, who are they like, what's making up? Are they developers? Are they traditional enterprises? Are they contributing companies? Who's the 65%, >>Um, who's the 65%, >>Right? The new, new, >>Well, it's all kinds of C companies sending their developers, right? It's sometimes there's a lot of them are end users. I think at least half or a third, at least of attendees are end user companies. And, uh, then there is also like the new startups around town. And then there is like the, every big company or small has been hiring developers as fast as possible. And even if they've always been a player in cloud native, they need to send all these people to this ecosystem to start building the relationships start like learning the technology. So it's all kinds of folks are collecting to that here. >>As I, as I think about people starting to learn the technologies, learn the communities, the one thing the market change for this coupon for me over others is the number of customers, sharing stories, end user organizations. Mm-hmm, <affirmative>, mm-hmm, <affirmative> much of the cuon that I've been through many of the open source conferences. It's always been like vendors pushing their message, et cetera. What talk, tell me about that. C change. >>One thing that's like just immediate, um, and the case right now is that all the co-chairs for the event who are in charge of designing the agenda are end users. So we have Emily Fox from apple. We have Jasmine James from Twitter, and we have Ricardo Roka from se. So they're all end users. So naturally they're like, you know, picking talks that they're like, well, this is very relevant. Imma go for that and I'm here for it. Right? So that's one thing that's just happening. The other though is a greater trend, which is, as I was saying in the pandemic, so many companies has to get going and quickly that they have built expertise and users are no longer the passive recipients of information. They're equal contributors. They know what they need, what they want, they have experiences to share. And you're seeing that reflected in the conference. >>One thing I've seen at other conferences in the past that started out really for practitioners, uh, is that invariably, they want to go upscale and they wanna draw the CIOs and the, oh yeah. The, uh, you know, the executive, the top executives. Is that an objective, uh, for you or, or do you really want to keep this kind of a, a t-shirt crowd for the long term? >>Hey, everyone's welcome. That's really important, you know? Right. And, um, so we, and that's why we are trying to expand. It's like, you know, middle out as they had in the Silicon valley show the idea being, sorry, I just meant this a little. Okay. So the idea being that we've had the core developer crews, developer, DevOps, SRE crowd, right op over the course of the last virtual events, we actually expanded in the other direction. We put in a business value track, which was more for like people in the business, but not in as a developer or DevOps engineer. We also had a student thing where it's like, you're trying to get all the university crowd people, and it's been working phenomen phenomenally. And then actually this, this event, we went, uh, in the other direction as well. We hosted our inaugural CTO summit, which is for senior leadership and end user companies. And the idea is they're discussing topics of technology that are business relevant. So our topic this time was resiliency in multi-cloud and we're producing a research paper about it. That's gonna come out in some weeks. So BA so with, for us, it's about getting everybody under this tent. Right. And, but it will never mean that we deprioritize what we started with, which is the engineering crowd. It's just an expansion >>Stay true to your roots. >>Yes. Well, Prianca, we're going to talk to a lot of those startup communities tomorrow. Ah, tomorrow's coverage. It's all about startups. Why should CTOs, uh, new startups talk to these upstarts of as opposed to some of the bigger players here on the show floor, over 170 sponsoring companies, the show floor has been vibrant engaging. Yes. And we're going to get into that community tomorrow's coverage on the cube from Valencia Spain. I'm Keith Townson, along with Paul Gillon and you're watching the cube, the leader and high tech coverage.

>>Hi, everyone. Welcome to the cubes presentation of the AWS startup showcase the theme. This episode is data as code, and this is season two, episode two of the ongoing series covering exciting startups from the ecosystem in cloud and the future of data analytics. I'm your host, John furry. You're getting great featured panel here with AWS heroes, Lynn blankets, the CEO of Lindbergh Lega consulting, Peter Hanson's, founder of cloud Cedar and Alex debris, principal of debris advisory. Great to see all of you here and, uh, remotely and look forward to see you in person at the next re-invent or other event. >>Thanks for having us. >>So Lynn, you're doing a lot of work in healthcare, Peter you're in the middle of all the action as data as code Alex. You're in deep on the databases. We've got a good round up of, of topics here ranging from healthcare to getting under the hood on databases. So as we'll start with you, what are you working on right now? What trends do you see in the database space? >>Yeah, sure. So I do, uh, I do a lot of consulting work working with different people and, you know, often with, with dynamo DB or, or just general serverless technology type stuff. Um, if you want to talk about trends that I'm seeing right now, I would say trends you're seeing as a lot, just more serverless native databases or cloud native databases where you're seeing these cool databases come out that really take advantage of, uh, this new cloud environment, right? Where you have scalability, you have plasticity of the clouds. So you're not having, you know, instant space environments anymore. You're paying for capacity, you're paying for throughput. You're able to scale up and down. You're not managing individual instances. So a lot of cool stuff that we're seeing, you know, um, with this new generation of, of infrastructure and in particular database is taking advantage of this, this new cloud world >>And really lot deep into the database side in terms of like cloud native impact, diversity of database types, when to use certain databases that also a big deal. >>Yeah, absolutely. I like, I totally agree. I love seeing the different types of databases and, you know, AWS has this whole, uh, purpose-built database strategy. And I think that, that makes a lot of sense. Um, you know, I want to go too far with it. I would, I would more think about purpose-built categories and things like that, you know, specialize in an OLTB database within your, within your organization, whether that's dynamo DB or document DB or relational database Aurora or something like that. But then also choose some sort of analytics database, you know, if it's drew it or Redshift or Athena, and then, you know, if you have some specialized needs, you want to show some real time stuff to your users, check out rock site. If you want to, uh, you know, do some graph analytics, fraud detection, checkout tiger graph, a lot of cool stuff that we're seeing from the startup showcase here. >>Looking forward to unpacking that Lynn you've been in love now, a healthcare action with cloud ops, the pandemic pushes hard core on everybody. What are you working on? >>Yeah, it's all COVID data all the time. Uh, before the pandemic, I was supporting research groups for cancer genomics, which I still do, but, um, what's, uh, impactful is the explosive data volumes. You know, when you there's big data and there's genomic data, you know, I've worked with clients that have broken data centers, broken public cloud provider data centers because of the daily volume they're putting in. So there's this volume aspect. And then there's a collaboration, particularly around COVID research because of pandemic. And so you have this explosive volume, you have this, um, need for, uh, computational complexity. And that means cloud the challenge is it, you know, put the pedal to the metal. So you've got all these bioinformatics researchers that are used to single machine. Suddenly they have to deal with distributed compute. So it's a wild time to be in this space. >>What was the big change that you've seen with the, uh, the pandemic and in genomic cloud genomic specifically what's the big change has happened. >>The amount of data that is being put into the public cloud, um, previously people would have their data on their local, uh, capacity, and then they would publish their paper and the data may or may not become available for, uh, reproducing the research, uh, to accelerate for drug discovery and even variant identification. The data sets are being pushed to public cloud repositories, which is a whole new set of concerns. You have not only dealing with the volume and cost, but security, you know, there's federated security is non-trivial and not well understood by this domain. So there's so much work available here. >>Awesome. Peter, you're doing a lot with the data as a platform kind of view and platform engineering data as code is, is something that's being kicked around. What are you working on and how does platform engineering change as data becomes so much more prevalent in its value proposition? >>Yeah. So I'm the founder of cloud Cedar and, um, we sort of built this company out, this consultancy all around the challenges that a lot of companies have got with getting their data sorted, getting it organized, getting it ready for other use cases, such as analytics and machine learning, um, AI workloads and the like. So typically a platform engineering team will look after the organization of a company infrastructure, making sure that it's coherent across the company and a data platform, engineering teams doing something similar in that sense where they're, they're looking at making sure that, uh, data teams have a solid foundation to build upon, uh, that everything's quite predictable and what that enables is a faster velocity and the ability to use data as code as a way of specifying and onboarding data, building that, translating it, transforming it out into its specific domains and then on to data products. >>I have to ask you while you're here. Um, there's a big trend around data meshes right now. You're hearing, we've had a lot of stuff on the cube. Um, what are practical that people are using data mesh, first of all, is it relevant and how are people looking at this data mesh conversation? >>I think it becomes more and more relevant, uh, the bigger the organization that you're dealing with. So, you know, often times in the enterprise, you've got, uh, projects with timelines of five to 10 years often outlasting technology life cycles. The technology that you're building on is probably irrelevant by the time that you complete it. And what we're seeing is that data engineering teams and data teams more broadly, this organizational bottleneck and data mesh is all about, uh, breaking down that, um, bottleneck and decentralizing the work, shifting that work back onto, uh, development teams who oftentimes have got more of the context and a centralized data engineering team. And we're seeing a lot of, uh, Philocity increases as a result of that. >>It's interesting. There's so many different aspects of how data is changing the world. Lynn talks about the volume with the cloud and genomics. We're hearing data engineering at a platform level. You're talking about slicing and dicing and real-time information. You mentioned rock set, Alex. So I'd like to ask each of you to answer this next question, which is how has the team dynamics changed with data engineering because every single company's impacted. So if you're researchers, Lynn, you're pumping more data into the cloud, that's got a little bit of data engineering to it. Do they even understand that is that impacting them? So how has data changed the responsibilities or roles in this new emerging area of data engineering or whatever you want to call it? Lynn, we'll start with you. What do you, what do you see this impact? >>Well, you know, I mean, dev ops becomes data ops and ML ops and, uh, you know, this is a whole emergent area of work and it starts with an understanding of container technologies, which, you know, in different verticals like FinTech, that's a given, right, but in bioinformatics building an appropriately optimized Docker container is something I'm still working with customers now on because they have the concept of a Docker container is just a virtual machine, which obviously it isn't, or shouldn't be. So, um, you have, again, as I mentioned previously, this humongous skill gap, um, concepts like D, which are prevalent in ad tech FinTech, that's not available yet for most of my customers. So those are the things that I'm building. So the whole ops space is, um, this a wide open area. And really it's a question of practicality. Um, you know, I have, uh, a lot of experience with data lakes and, you know, containerizing and using the data lake platform. But a lot of my customers are going to move to like an interim pass based solutions. If they're using spark, for example, they might use to use a managed spark solution as an interim, um, step up to the cloud before they build their own containers. Because the amount of knowledge to do that effectively is non-trivial >>Peter, you mentioned data, you mentioned data lakes, onboarding data into lake house architectures, for instance, something that you're familiar with. Um, this is not obvious to some verticals obvious to others. What do you see this data engineering impact from a personnel standpoint? And then ultimately how things get built, >>You know, are you directing that to me, >>Peter? >>Yeah. So I think, um, first and foremost, you know, the workload that data engineering teams are dealing with is ever increasing. Usually there's a 10 X ratio of, um, software engineers to data engineers within a business and usually double the amount of analysts to data engineers again. And so they're, they're fighting it ever increasing backload. And, uh, so they're fighting an ever increasing backlog of, of, uh, tasks to do and tickets to, to, to churn through. And so what we're seeing is that data engineering teams are becoming data platform engineering teams where they're building capability instead of constantly hamster wheels spinning if you will. And so with that in mind, with onboarding data into, uh, a Lakehouse architecture or a data lake where data engineering teams, uh, uh, getting wins is developing a very good baseline of structure where they're getting the categorization, the data tagging, whether this data is of a particular domain, does it contain some, um, PII data, for instance, uh, and, and, and, and then the security aspects, and also, you know, the mechanisms on which to do the data transformations, >>Alex, on the database side, those are known personas in an enterprise, a them, the database team, but now the scale is so big. Um, and there's so much going on in databases. How does the data engineering impact organizations from your standpoint? >>Yeah, absolutely. I think definitely, you know, gone are the days where you have a single relational database that is serving operational queries for your users, and you can also serve analytics queries, you know, for your internal teams. It's, it's now split up into those purpose-built databases, like we've said. Uh, but now you've got two different teams managing it and they're, they're designing their data model for different things. You know? So L LLTP might have a more de-normalized model, something that works for very fast operations and it's optimized for that, but now you need to suck that data out and get it elsewhere so that your, your PM or your business analyst, or whoever can crunch through some of that. And, you know, now it needs to be in a more normalized format. How do you sort of bridge that gap? That's a tough one. I think you need to, you know, build empathy on each side of, of what each side is doing and, and build the tools to say, Hey, this is going to help you, uh, you know, LLTP team, if we know what, what users are actually doing, and, and if you can get us into the right format there, so that then I can, you know, we can analyze it, um, on the backend. >>So I think, I think building empathy across those teams is helpful. >>When I left to come back to, you mentioned a health and informatics is coming back. Um, but it's interesting, you know, I look at a database world and you look at the solutions that are out there. A lot of companies that build data solutions don't have a data problem. They've never, they're not swimming in a lot of data, but then you look at like the field that you're working in right now with the genomics and health and, and quantum, they're always, they're dealing with data all the time. So you have people who deal with a lot of data all the time are breaking through New Zealand. People who are don't have that experience are now becoming data full, right? So people are now either it's a first time problem, or they've always been swimming in a ton of data. So it's more of what's the new playbook. And then, wow, I've never had to deal with a lot of data before. What's your take? >>It's interesting. Cause they know, uh, bioinformatics hires, um, uh, grad students. So grad students, you know, use their, our scripts with their file on their laptop. And so, um, to get those folks to understand distributed container-based computing is like I said, a not non-trivial problem. What's been really interesting with the money pouring in to COVID research is when I first started, some of the workflows would take, you know, literally 500 hours and that was just okay. And coming out of FinTech, I was, uh, I could, I was blown away like FinTech is like, could that please take a millisecond rather than a second? Right. And so what has now happened, which makes it, you know, like I said, even more fun to work in this domain is, uh, the research dollars have really gone up because of the pandemic. And so there are, there are, there's this blending of people like me with more of a big data background coming into bioinformatics and working side by side. >>So it's this interesting sort of translation because you have the whole taxonomy of bioinformatics with genomics and sequencers and all the weird file types that you get. And then you have the whole taxonomy of dev ops data ops, you know, containers and Kubernetes and all that. And trying to get that into pipelines that can actually, you know, be efficient, given the constraints. Of course, we, on the tech side, we always want to make it super optimized. I had a customer that we got it down from 500 hours to minutes, but they wanted to stay with the past solution because it was easier for them to go from 500 hours to five hours was good enough, but you know, the techies want to get it down to five minutes. >>This is, this is, we've seen this movie before dev ops, um, edge and op operations, you know, IOT, world scenes, the convergence of cultures. Now you have data and then old, old school operations kind of coming up. So this kind of supports the thesis. That data as code is the next infrastructure as code. What do you guys, what's the reaction there for you guys? What do you think about that? What does data's code mean? If infrastructure's code was cloud and dev ops, what is data as code? What does that mean? >>I could take it if you like. I think, um, data teams, organizations, um, have been long been this bottleneck within the organization and there's like this dark matter of untapped energy and potential waiting to be unleashed a data with the advent of open source projects like DBT, um, have been slowly sort of embracing software development, lifecycle practices. And this is really sort of seeing a, a big steep increase in, um, in their velocity. And, and this is only going to increase and improve as we're seeing data teams, um, embrace starter as code. I think it's, uh, the future is bright for data. So I'm very excited. >>Lynn Peter reaction. I mean, agility data is code is developer concept CICB pipeline. You mentioned it new operational workflows coming into traditional operations reaction. >>Yeah. I mean, I think Peter's right on there. I'd say, you know, some of those tools we're seeing come in from, from software, like, like DBT, basically giving you that infrastructure as code, but applied to that data realm. Also there have been a few, like get for data type things, pack a derm, I believe is one and a few other ones where you bring that in and you also see a lot of immutability concepts flowing into the data realm. So I think just seeing some of those software engineering concepts come over to the data world has, has been pretty interesting >>What we'll literally just versioning datasets and the identification of what's in a data set. What's not in a data set. Some of this is around ethical AI as well, um, is a whole, uh, area that has come out of research groups. Um, mostly AI research groups, but is being applied to medical data and needs to be obviously, um, so this, this, this, um, metadata and versioning around data sets is really, I think, a very of the moment area. >>Yeah, I think we, we, you guys are bringing up a really good kind of direction that's happening in data. And that is something that you're seeing on the software side, open source and now dev ops. And now going to data is that the supply chain challenges of we've been talking about it here on the cube and this, this, um, this episode is, you know, we've seen Ukraine war, but some open source, you know, malware hitting datasets is data secure. What is that going to look like? So you starting to get into this what's the supply chain, is it verified data sets if data sets have to be managed a whole nother level of data supply chain comes up, what do you guys think about that? >>I'll jump in. Oh, sorry. I'll jump in again. I think that, you know, there's, there's, um, some, some of the compliance requirements, um, around financial data are going to be applied to other types of data, probably health data. So immutability reproducibility, um, that is, uh, legally required. Um, also some of the privacy requirements that originated in Europe with GDPR are going to be replicated as more and more, um, types of data. And again, I'm always going to speak for health, but there's other types as well coming out of personal devices and that kind of stuff. So I think, you know, this idea of data as code is it's, it goes down to versioning and controlling and, um, that's, uh, that's sort of a real succinct way to say it that we didn't used to think about that. We just put it in our, you know, relational database and we were good to go, but, um, versioning and controlling in the global ecosystem is kind of, uh, where I'm focusing my efforts. >>It brings up a good question. If databases, if data is going to be part of the development process has to be addressable, which means horizontally scalable. That means it has to be accessible and open. How do you make that work and not foreclose it with a lot of restrictions? >>I think the use of data catalogs and appropriate tagging and categorization, you know, I think, you know, everyone's heard of the term data swamp, and I think that just came about because that everyone saw like, oh, wow, S3, you know, infinite storage. We just, you know, throw whatever in there for as long as we want. And I think at times, you know, the proliferation of S3 buckets, um, and the like, you know, we've just seen, uh, perhaps security, not maintained as well as it could have been. And I think that's kind of where data platform engineering teams have really sort of, uh, come into the, for, you know, creating a governance set of buckets like formation on top. But I think that's kind of where we need to see a lot more work with appropriate tags and also the automatic publishing of metadata into data catalogs so that, um, folks can easily search and address particular data sets and also control the access. You know, for instance, you've got some PII data, perhaps really only your marketing folks should be looking at email addresses and the like not perhaps your finance folks. So I think, you know, there's, there's a lot to be leveraged there in formation and other solutions, >>Alex, let's back up and talk about what's in it for the customer, right. Let's zoom back and saying reality is I just got to get my data to make sure it's secure always on and not going to be hackable. And I just got to get my data available on river performance. So then, then I got to start thinking about, okay, how do I intersect it? So what should teams be thinking about right now as I look up all their data options or databases across their enterprise? >>Yeah, it's, it's a, it's a good question. I just, you know, I think Peter made some good points there and you can think of history as sort of ebbing and flowing between centralization and decentralization a lot of times. And you know, when storage was expensive, data was going to be sort of centralized and Maine maintained, sort of a, you know, by the, uh, the people that are in charge of it. But then when, when S3 comes along, it really decreases storage. Now we can do a lot more experiments on it. We can store a lot more of our data, keep it around and do different things on it. You know, now we've got regulations again, we were, we gotta, we gotta be more realistic about, about keeping that data secure and make sure we're, we're doing the right things with it. So it's, we're gonna probably go through a period of, of centralization as we work out some of this tooling around, you know, tagging and, and ethical AI that, that both Peter. And when we're talking about here and maybe get us into that, that next wearable world of de-centralization again. But I, I think that ebb and flow is going to be natural in response to, you know, the problems of the, the other extreme, >>Where are we in the market right now from progress standpoint, because data lakes don't want to be data swamps. You seeing lake formation as a data architecture, as an example, where are we with customers? What are they doing right now? Where would you put them in the progress bar of, of evolution towards the Nirvana of having this data sovereignty? And this data is code environment. Are they just now in the data lake store, everything real-time and historical? >>Well, I can jump in there. Um, SQL on files is the, is the driver. And so we know when Amazon got Athena, um, that really drove a lot of the customers to really realistically look at data lake technologies, but data warehouses are not going away. And the integration between the two is not seamless. No, we, we are partners with AWS, but we don't work for them. So we can tell you the truth here. Um, there's, there's work to it, but it really, for my customers, it really upped the ante around data lake, uh, because Athena and technologies like that, the serverless, um, SQL queries or the familiar quarry, um, uh, libraries really drove a movement away from either OLTB or OLAP, more expensive, more cumbersome structures, >>But they still need that. Oh, LTP, like if they have high latency issues, they want to be low latency. Can they have the best of both worlds? That's the question. >>I mean, I w I would say we're getting, you know, we're getting closer. We're always going to be, uh, you know, that technology is going to be moving forward, and then we'll just move the goalpost again, in terms of, of what we're asking from it. But I think, you know, the technology that's getting out there, you can get, get really well. And then, you know, just what I work in the dynamo DB world. So you can get really great low latency. So, you know, single digit millisecond LLTP response times on that. I think some of the analytics stuff has been a problem with that. And there, there are different solutions out there to where you can export dynamo to S3, and then you can be doing SQL on your FA your files with Athena Lakeland's talking about, or now you see, you know, rock set of partner here that that'll just ingest your dynamo, DB data, you know, make all those changes. So if you're doing a lot of, uh, changes to your data and dynamo is going to reflect in Roxanna, and then you can do analytics queries, you can do complex filters, different things like that. So, you know, I, I think we continue to push the envelope and then we moved the goalpost again. But, um, you know, I think we're in a, a lot better place than we were a few years ago, for sure. >>Where do you guys see this going relative to the next level? If data as code becomes that next agile, um, software defined environment with open source? Well, all of these new tools with serverless things happening with data lakes are built in with nice architectures with data warehouses, where does it go next? What happens next? If this becomes an agile environment, what's the impact? >>Well, I don't want to be so dominant, but I have, I feel strongly, so I'm going to jump in here. So, so I, um, I feel like, you know, now for my, my, my most computationally intensive workloads, I'm using GPS, I'm bursting to GPU for TensorFlow neural networks. So I've been doing quite a bit of exploration around Amazon bracket for QPS and it's early. Um, and it's specialty. It's not, you know, for everybody. And the learning curve again is pretty daunting, but, um, there are some use cases out there. I mean, I got ahold of a paper where some people did some, um, it was a Q CNN, um, quantum convolutional neural network for lung cancer images, um, from COVID patients and the, the, uh, the QP Hugh, um, algorithm pipeline performed more accurately and faster. So I think, um, bursting to quantum is something to pay attention to. >>Awesome. Peter, what's your take on what's next? >>Well, I think there's still, um, that, that was absolutely fascinating from Lynn, but I think also there's, there's, uh, you know, some more sort of low-level, uh, low-hanging fruit available in, in the data stack. I think there's a lot of, there's still a lot of challenges around the transformation there, getting our data from sort of raw landed data into business domains, and that sort of talks to a lot of what data mesh is all about. I think if we can somehow make that a little more frictionless, because that that's really where the like labor intensive work is. That's, that's kinda dominating, uh, data engineering teams and where we're sort of trying to push that, that workload back onto, um, you know, software engineering teams. >>Alice will give you the final word. What's the impact. What's the next step? What's it look like in the future? >>Yeah, for sure. I mean, I've never had the, uh, breaking a data center problem that wind's had, or the bursting the quantum problem, for sure. But, you know, if you're in that, you know, the pool I swim and of terabytes of data and below and things like that, I think it's a good time. It just like we saw, you know, like we were talking about dev ops and, and pushing, uh, you know, allowing software engineers to handle more of, of the operation stuff. I think the same thing with data can happen where, you know, software engineering teams can handle not just their code, not just, you know, deploying and operating it, but also thinking about their data around the code. And that doesn't mean you won't have people assist you within your organization. You won't have some specialists in there, but I think pushing more stuff, even onto the individual development teams where they have ownership of that. And they're thinking about it through all this different life cycle. I mean, I'm pretty bullish on that. And I think that's an exciting development >>Was that shift, what left with left is security. What does that mean to >>Shipped so much stuff left, but now, you know, the things that were at the end are back at the end again, but, uh, you know, at least we think we can think about that stuff early in the process, which is good, >>Great conversation, very provocative, very realistic and great impact on the future data as code is real, the developers I do believe will have a great operational role and the data stack concept and impacting things like quantum, it's all kind of lining up nicely. Um, and it's a great opportunity to be in this field from a science and policy standpoint. Um, data engineering is legit. It's going to continue to grow and thanks for unpacking that here on the queue. Appreciate it. Okay. Great panel D AWS heroes. They work with AWS and the ecosystem independently out there. They're in the trenches doing the front lines, cracking the code here with data as code season two, episode two of the ongoing series of the 80, but startups I'm John for your host. Thanks for watching.

>>Mm. >>Okay. Hello, and welcome to the Cubes Coverage of the International Women in Tech Showcase featuring National Women's Day. I'm John for a host of the Cube. We have a great guest here of any theory a PJ head of aerospace and satellite for A W S A P J s Asia Pacific in Japan. Great to have you on many thanks for joining us. Talk about Space and International Women's Day. Thanks for coming on. >>Thanks, John. It's such a pleasure to be here with you. >>So obviously, aerospace space satellite is an area that's growing. It's changing. AWS has made a lot of strides closure, and I had a conversation last year about this. Remember when Andy Jassy told me about this initiative to 2.5 years or so ago? It was like, Wow, that makes a lot of sense Ground station, etcetera. So it just makes a lot of sense, a lot of heavy lifting, as they say in the satellite aerospace business. So you're leading the charge over there in a p J. And you're leading women in space and beyond. Tell us what's the Storey? How did you get there? What's going on. >>Thanks, John. Uh, yes. So I need the Asia Pacific business for Clint, um, as part of Amazon Web services, you know, that we have in industry business vertical that's dedicated to looking after our space and space customers. Uh, my journey began really? Three or four years ago when I started with a W s. I was based out of Australia. Uh, and Australia had a space agency that was being literally being born. Um, and I had the great privilege of meeting the country's chief scientist. At that point. That was Dr Alan Finkel. Uh, and we're having a conversation. It was really actually an education conference. And it was focused on youth and inspiring the next generation of students. Uh, and we hit upon space. Um, and we had this conversation, and at that stage, we didn't have a dedicated industry business vertical at A W s well supported space customers as much as we did many other customers in the sector, innovative customers. And after the conversation with Dr Finkel, um, he offered to introduce me, uh, to Megan Clark, who was back back then the first CEO of the Australian Space Agency. So that's literally how my journey into space started. We had a conversation. We worked out how we could possibly support the Australian Space Agency's remit and roadmap as they started growing the industry. Uh, and then a whole industry whole vertical was set up, clinic came on board. I have now a global team of experts around me. Um, you know, they've pretty much got experience from everything creating building a satellite, launching a satellite, working out how to down link process all those amazing imagery that we see because, you know, um, contrary to what a lot of people think, Uh, space is not just technology for a galaxy far, far away. It is very much tackling complex issues on earth. Um, and transforming lives with information. Um, you know, arranges for everything from wildfire detection to saving lives. Um, smart, smart agriculture for for farmers. So the time of different things that we're doing, Um, and as part of the Asia Pacific sector, uh, my task here is really just to grow the ecosystem. Women are an important part of that. We've got some stellar women out here in region, both within the AWS team, but also in our customer and partner sectors. So it's a really interesting space to be. There's a lot of challenges. There's a lot of opportunities and there's an incredible amount of growth so specific, exciting space to be >>Well, I gotta say I'm super inspired by that. One of the things that we've been talking about the Cuban I was talking to my co host for many, many years has been the democratisation of digital transformation. Cloud computing and cloud scale has democratised and change and level the playing field for many. And now space, which was it's a very complex area is being I want kind of democratised. It's easier to get access. You can launch a satellite for very low cost compared to what it was before getting access to some of the technology and with open source and with software, you now have more space computing things going on that's not out of reach. So for the people watching, share your thoughts on on that dynamic and also how people can get involved because there are real world problems to solve that can be solved now. That might have been out of reach, but now it's cloud. Can you share your thoughts. >>That's right. So you're right, John. Satellites orbiting There's more and more satellites being launched every day. The sensors are becoming more sophisticated. So we're collecting huge amounts of data. Um, one of our customers to cut lab tell us that we're collecting today three million square kilometres a day. That's gonna increase to about three billion over the next five years. So we're already reaching a point where it's impossible to store, analyse and make sense of such massive amounts of data without cloud computing. So we have services which play a very critical role. You know, technologies like artificial intelligence machine learning. Help us help these customers build up products and solutions, which then allows us to generate intelligence that's serving a lot of other sectors. So it could be agriculture. It could be disaster response and recovery. Um, it could be military intelligence. I'll give you an example of something that's very relevant, and that's happening in the last couple of weeks. So we have some amazing customers. We have Max our technologies. They use a W S to store their 100 petabytes imagery library, and they have daily collection, so they're using our ground station to gather insight about a lot of changing conditions on Earth. Usually Earth observation. That's, you know, tracking water pollution, water levels of air pollution. But they're also just tracking, um, intelligence of things like military build up in certain areas. Capella space is another one of our customers who do that. So over the last couple of weeks, maybe a couple of months, uh, we've been watching, uh, images that have been collected by these commercial satellites, and they've been chronicling the build up, for instance, of Russian forces on Ukraine's borders and the ongoing invasion. They're providing intelligence that was previously only available from government sources. So when you talk about the democratisation of space, high resolution satellite images are becoming more and more ridiculous. Um, I saw the other day there was, uh, Anderson Cooper, CNN and then behind him, a screenshot from Capella, which is satellite imagery, which is very visible, high resolution transparency, which gives, um, respected journalists and media organisations regular contact with intelligence, direct intelligence which can help support media storytelling and help with the general public understanding of the crisis like what's happening in Ukraine. And >>I think on that point is, people can relate to it. And if you think about other things with computer vision, technology is getting so much stronger. Also, there's also metadata involved. So one of the things that's coming out of this Ukraine situation not only is tracking movements with the satellites in real time, but also misinformation and disinformation. Um, that's another big area because you can, uh, it's not just the pictures, it's what they mean. So it's well beyond just satellite >>well, beyond just satellite. Yeah, and you know, not to focus on just a crisis that's happening at the moment. There's 100 other use cases which were helping with customers around the globe. I want to give you a couple of other examples because I really want people to be inspired by what we're doing with space technology. So right here in Singapore, I have a company called Hero Factory. Um, now they use AI based on Earth observation. They have an analytics platform that basically help authorities around the region make key decisions to drive sustainable practises. So change detection for shipping Singapore is, you know, it's lots of traffic. And so if there's oil spills, that can be detected and remedy from space. Um, crop productivity, fruit picking, um, even just crop cover around urban areas. You know, climate change is an increasing and another increasing, uh, challenges global challenge that we need to tackle and space space technology actually makes it possible 15 50% of what they call e CVS. Essential climate variables can only be measured from space. So we have companies like satellite through, uh, one of our UK customers who are measuring, um, uh, carbon emissions. And so the you know, the range of opportunities that are out there, like you said previously untouched. We've just opened up doors for all sorts of innovations to become possible. >>It totally is intoxicating. Some of the fun things you can discuss with not only the future but solving today's problems. So it's definitely next level kind of things happening with space and space talent. So this is where you start to get into the conversation like I know some people in these major technical instance here in the US as sophomore second year is getting job offers. So there's a There's a there's a space race for talent if you will, um and women talent in particular is there on the table to So how How can you share that discussion? Because inspiration is one thing. But then people want to know what to do to get in. So how do you, um how do you handle the recruiting and motivating and or working with organisations to just pipeline interest? Because space is one of the things you get addicted to. >>Yeah. So I'm a huge advocate for science, technology, engineering, math. We you know, we highlights them as a pathway into space into technology. And I truly believe the next generation of talent will contribute to the grand challenges of our time. Whether that climate change or sustainability, Um, it's gonna come from them. I think I think that now we at Amazon Web services. We have several programmes that we're working on to engage kids and especially girls to be equipped with the latest cloud skills. So one of the programmes that we're delivering this year across Singapore Australia uh, we're partnering with an organisation called the Institute for Space Science, Exploration and Technology and we're launching a programme called Mission Discovery. It's basically students get together with an astronaut, NASA researcher, technology experts and they get an opportunity to work with these amazing characters, too. Create and design their own project and then the winning project will be launched will be taken up to the International space station. So it's a combination of technology skills, problem solving, confidence building. It's a it's a whole range and that's you know, we that's for kids from 14 to about 18. But actually it, in fact, because the pipeline build is so important not just for Amazon Web services but for industry sector for the growth of the overall industry sector. Uh, there's several programmes that were involved in and they range from sophomore is like you said all the way to to high school college a number of different programmes. So in Singapore, specifically, we have something called cloud Ready with Amazon Web services. It's a very holistic clouds killing programme that's curated for students from primary school, high school fresh graduates and then even earlier careers. So we're really determined to work together closely and it the lines really well with the Singapore government's economic national agenda, um so that that's one way and and then we have a tonne of other programmes specifically designed for women. So last year we launched a programme called She Does It's a Free online training learning programme, and the idea is really to inspire professional women to consider a career in the technology industry and show them pathways, support them through that learning process, bring them on board, help drive a community spirit. And, you know, we have a lot of affinity groups within Amazon, whether that's women in tech or a lot of affinity groups catering for a very specific niches. And all of those we find, uh, really working well to encourage that pipeline development that you talk about and bring me people that I can work with to develop and build these amazing solutions. >>Well, you've got so much passion. And by the way, if you have, if you're interested in a track on women in space, would be happy to to support that on our site, send us storeys, we'll we'll get We'll get them documented so super important to get the voices out there. Um and we really believe in it. So we love that. I have to ask you as the head of a PJ for a W S uh aerospace and satellite. You've you've seen You've been on a bunch of missions in the space programmes of the technologies. Are you seeing how that's trajectory coming to today and now you mentioned new generation. What problems do you see that need to be solved for this next generation? What opportunities are out there that are new? Because you've got the lens of the past? You're managing a big part of this new growing emerging business for us. But you clearly see the future. And you know, the younger generation is going to solve these problems and take the opportunities. What? What are they? >>Yes, Sometimes I think we're leaving a lot, uh, to solve. And then other times, I think, Well, we started some of those conversations. We started those discussions and it's a combination of policy technology. We do a lot of business coaching, so it's not just it's not just about the technology. We do think about the broader picture. Um, technology is transferring. We know that technology is transforming economies. We know that the future is digital and that diverse backgrounds, perspective, skills and experiences, particularly those of women minority, the youth must be part of the design creation and the management of the future roadmaps. Um, in terms of how do I see this going? Well, it's been sort of we've had under representation of women and perhaps youth. We we just haven't taken that into consideration for for a long time now. Now that gap is slowly becoming. It's getting closer and closer to being closed. Overall, we're still underrepresented. But I take heart from the fact that if we look at an agency like the US Mohammed bin Rashid Space Centre, that's a relatively young space agency in your A. I think they've got about three or 400 people working for them at this point in time, and the average age of that cohort John, is 28. Some 40% of its engineers and scientists are women. Um, this year, NASA is looking to recruit more female astronauts. Um, they're looking to recruit more people with disabilities. So in terms of changing in terms of solving those problems, whatever those problems are, we started the I guess we started the right representation mix, so it doesn't matter. Bring it on, you know, whether it is climate change or this ongoing crisis, productive. Um, global crisis around the world is going to require a lot more than just a single shot answer. And I think having diversity and having that representation, we know that it makes a difference to innovation outputs. We know that it makes a difference to productivity, growth, profit. But it's also just the right thing to do for so long. We haven't got it right, and I think if we can get this right, we will be able to solve the majority of some of the biggest things that we're looking at today. >>And the diversity of problems in the diversity of talent are two different things. But they come together because you're right. It's not about technology. It's about all fields of study sociology. It could be political science. Obviously you mentioned from the situation we have now. It could be cybersecurity. Space is highly contested. We dated long chat about that on the Last Cube interview with AWS. There's all these new new problems and so problem solving skills. You don't need to have a pedigree from Ivy League school to get into space. This is a great opportunity for anyone who can solve problems because their new No one's seen them before. >>That's exactly right. And you know, every time we go out, we have sessions with students or we're at universities. We tell them, Raise your voices. Don't be afraid to use your voice. It doesn't matter what you're studying. If you think you have something of value to say, say it. You know, by pushing your own limits, you push other people's limits, and you may just introduce something that simply hasn't been part of before. So your voice is important, and we do a lot of lot of coaching encouraging, getting people just to >>talk. >>And that in itself is a great start. I think >>you're in a very complex sector, your senior leader at AWS Amazon Web services in a really fun, exciting area, aerospace and satellite. And for the young people watching out there or who may see this video, what advice would you have for the young people who are trying to navigate through the complexities of now? Third year covid. You know, seeing all the global changes, um, seeing that massive technology acceleration with digital transformation, digitisation it's here, digital world we're in. >>It could >>be confusing. It could be weird. And so how would you talk to that person and say, Hey, it's gonna be okay? And what advice would you give? >>It is absolutely going to be okay. Look, from what I know, the next general are far more fluent in digital than I am. I mean, they speak nerd. They were born speaking nerd, so I don't have any. I can't possibly tell them what to do as far as technology is concerned because they're so gung ho about it. But I would advise them to spend time with people, explore new perspectives, understand what the other is trying to do or achieve, and investing times in a time in new relationships, people with different backgrounds and experience, they almost always have something to teach you. I mean, I am constantly learning Space tech is, um it's so complicated. Um, I can't possibly learn everything I have to buy myself just by researching and studying. I am totally reliant on my community of experts to help me learn. So my advice to the next generation kids is always always in this time in relationships. And the second thing is, don't be disheartened, You know, Um this has happened for millennia. Yes, we go up, then we come down. But there's always hope. You know, there there is always that we shape the future that we want. So there's no failure. We just have to learn to be resilient. Um, yeah, it's all a learning experience. So stay positive and chin up, because we can. We can do it. >>That's awesome. You know, when you mentioned the Ukraine in the Russian situation, you know, one of the things they did they cut the Internet off and all telecommunications and Elon Musk launched a star linked and gives them access, sending them terminals again. Just another illustration. That space can help. Um, and these in any situation, whether it's conflict or peace and so Well, I have you here, I have to ask you, what is the most important? Uh uh, storeys that are being talked about or not being talked about are both that people should pay attention to. And they look at the future of what aerospace satellite these emerging technologies can do for the world. What's your How would you kind of what are the most important things to pay attention to that either known or maybe not being talked about. >>They have been talked about John, but I'd love to see more prominent. I'd love to see more conversations about stirring the amazing work that's being done in our research communities. The research communities, you know, they work in a vast area of areas and using satellite imagery, for instance, to look at climate change across the world is efforts that are going into understanding how we tackle such a global issue. But the commercialisation that comes from the research community that's pretty slow. And and the reason it's loads because one is academics, academics churning out research papers. The linkage back into industry and industry is very, um, I guess we're always looking for how fast can it be done? And what sort of marginal profit am I gonna make for it? So there's not a lot of patients there for research that has to mature, generate outputs that you get that have a meaningful value for both sides. So, um, supporting our research communities to output some of these essential pieces of research that can Dr Impact for society as a whole, Um, maybe for industry to partner even more, I mean, and we and we do that all the time. But even more focus even more. Focus on. And I'll give you a small example last last year and it culminated this earlier this month, we signed an agreement with the ministry of With the Space Office in Singapore. Uh, so it's an MOU between AWS and the Singapore government, and we are determined to help them aligned to their national agenda around space around building an ecosystem. How do we support their space builders? What can we do to create more training pathways? What credits can we give? How do we use open datasets to support Singaporeans issues? And that could be claimed? That could be kind of change. It could be, um, productivity. Farming could be a whole range of things, but there's a lot that's happening that is not highlighted because it's not sexy specific, right? It's not the Mars mission, and it's not the next lunar mission, But these things are just as important. They're just focused more on earth rather than out there. >>Yeah, and I just said everyone speaking nerd these days are born with it, the next generations here, A lot of use cases. A lot of exciting areas. You get the big headlines, you know, the space launches, but also a lot of great research. As you mentioned, that's, uh, that people are doing amazing work, and it's now available open source. Cloud computing. All this is bringing to bear great conversation. Great inspiration. Great chatting with you. Love your enthusiasm for for the opportunity. And thanks for sharing your storey. Appreciate it. >>It's a pleasure to be with you, John. Thank you for the opportunity. Okay. >>Thanks, Manny. The women in tech showcase here, the Cube is presenting International Women's Day celebration. I'm John Ferrier, host of the Cube. Thanks for watching. Mm mm.

(upbeat music) >> Hi, everyone. Welcome to today's session of theCUBE presentation of AWS Startup Showcase, the new breakthrough in DevOps, data analytics, cloud management tools, featuring OpsRamp for the cloud management migration track. I'm John Furrier, your hosts of theCUBE Today, we're joined by Jordan Sheer, vice president of corporate marketing and Michael Fisher, director of product management in OpsRamp. Gentlemen, thank you for joining us today for this topic of challenges of delivering availability for the modern enterprise. >> Thanks, John. >> Yeah, thanks for having us. >> Hey, so first of all, I have to congratulate you guys on the successful launch and growth of your company. You've been in the middle of the action of all this DevOps, microservices, cloud scale, and availability is the hottest topic right now. IT Ops, AI Ops, whoever you want to look at it, IT is automating a way in a lot of value. You guys are in the middle of it. Congratulations on that, and congratulations on being featured. Take a minute to explain what you guys do. What's the strategy? What's the vision? What's the platform. >> Yeah, I'll take that one. So I would just kind of take a step back and we look at the broader landscape of the ecosystem of tools that all sits in. There's a lot of promises and a lot of whats and features and functionality that are being announced. Three pillars of durability and all these tools are really trying to solve a fundamental problem we see in the market and this problem transcends the classic IT ops and it's really front and center, even in this modern DevOps market, this is the problem of availability. And so when we talk about availability, we don't just mean the four nines for an uptime metric, availability to the modern enterprise, is really about an application doing what it needs to do to serve the users in a way that works for the business. And I always like to have a classic example of an e-commerce site, right? So maybe you can get to an e-commerce sites online, but you can't add an item to a cart, right? Well, you can't do something that is a meaningful transaction for the business. And because of that, that experience is not available to you as a user and it's not available to the business because it didn't result in a positive outcome. So the promise of OpsRamp is really around this availability concept and the way we rationalize this as a three pillar formats. And so we think the three pillars of availability are the ability to observe data, this is the first piece of it all. And from a problem perspective, what we're really trying to say is do we have the right data at any given point in time to accurately diagnose, assess, and troubleshoot application behavior? And we see it as a huge problem with a lot of enterprises, because data that can be often siloed, too many tools, many teams, and each one has a slightly different understanding of application health. For example, the DevOps team may have a instance of Prometheus or they may have some other monitoring tool, or the IT team may have their own set, right? But when you have that kind of segmented view of the world, you're not really having the data in a central place to understand availability at the most holistic level, which is really from an end-user to that middleware, to the databases, to underlying microservices, which are really providing the end-user experience. So that observed problem is that first thing OpsRamp tries to solve. Secondly, this is the analyze phase, right? So analyze to us means are we giving the proper intelligence on top of the data to drive meaningful insights to this operator and user? And the promise here is that can we understand that baseline performance and potentially even mitigate future instance from happening? How often do we hear a cloud provider going down or some SaaS provider going down because of some microservice migration issue or some third party application or networking they're relying on? I can think of dozens on my head. So that's kind of the second piece. And then lastly is around this act. This is an area of a lot of investment for ops because we think this is the final pillar for nailing this availability problem. Because again, IT teams are not getting larger, they're getting smaller, right? Everyone's trying to do more with less. And so from a platform perspective, how do we enable teams to focus on the most business critical tasks, which are your cloud migrations, adopting microservices to run your modern applications, innovative projects. These are the things that IT and DevOps teams are tasked with. And maintaining availability is not something people want to do, that should be automated. And so when you think of automation, this is a big piece for us. So again, the key problem is how can we enable these IT or DevOps teams to focus on those business critical things, and automate it with the rest. And so this is the OpsRamp's three pillars of availability. >> John: Talk about the platform, if you don't mind. I know you've got a slide on this. I want to jump into it because this comes up a lot, availability's not just throughout uptime, because you know, uptime, five nine reliability is an old school concept. Now you have different kinds of services that might be up but slow, would cause some problems, as applications and this modern era have all these new sets of services. Can you go through and talked about the platform? >> Yeah, absolutely. So OpsRamp has a very... We address this availability problem pretty holistically, like I mentioned. From a platform perspective, there that two core lines that are comprising a product. One is this hybrid monitoring piece. This is that data layer. And the next one is event management, it's more of the we'll talk about that analysis. And so we treat the monitor as a direct feed into this event management. We're layering that on top, or layering machine learning and AI to augment the insights derived from that first pillar. And so this is where we see a really interesting intersection of data science and monitoring tools. We invest a lot in this area because there's a lot of meaningful problems to solve. In particular alert fatigue, or potentially root cause analysis, things that can take an operator or a developer a long time to do on their own, OpsRamp tries to augment that knowledge of your systems and applications so that you can get to the bottom of things faster and get on with your day. And so it's not just for the major outages, it's not just for the things that are on Twitter or CNN that's for daily things that can just distract you from the ability to do your job, which is to be a core innovator for a business. >> I will really say John, that we are already seeing some couple things here. Number one, we're already actually seeing fundamental transformations in the marketplace. Customers who have seen reduction in alert volumes of up to 95% in some cases, which is as you can imagine, that's completely transformational for these businesses. And number two, I think one of the promises of hybrid of observability working in tandem with event and incident management is the idea of finding unknown unknowns within your organization and being able to act upon them. All too many times nowadays, monitoring tools are there to just surface issues that you may know that you're looking for and then help you find it and then take action on them. But I think the idea of OpsRamp is that we really using that big data platform that Michael talks about is to really surface all the issues that you might not be able to see, identify the root cause, and then take action on those root causes. So in our world, application availability is a much more proactive activity where the IT operations team can actually be proactive about these incidents and then take action on them. >> Yes. Jordan, if you don't mind, I'm following up on that real quick. Talk about the difference uptime versus availability, because something could be up and reliable but not available and its services get flaky. Things may look like they're up and running. Can you just unpack that a little? >> So to me, I mean the really key aspect of availability that I think the old definition of uptime doesn't address is performance. That something can be up, but not performing, but still not really be available. And his e-commerce example, I think is a great one. Let's take, for example, you get on Amazon, right? The Amazon e-commerce experience is always available. And what that means is that at any given moment, when I want to click through the e-commerce experience, it performs. It's available. It's always there and I can buy it at any given time. If there's a latency issue, if the application has a lag, if it takes 30 seconds to really perform an activity on that application, in the alternative definition, that's not available anymore. Even though the application may be up, it's not performing, it's not providing a frictionless end customer experience, and it's not driving the business forward, and therefore it's not available. The definition of availability in OpsRamp is creating a meaningful customer experience that actually drives the business forward. So in that definition, if a service is up but it's latent, but it's not providing excellent customer experience that the business wants to promise to its end-user, it's not available. So that's really how we're redefining this whole notion of availability and we're urging our customers and people in the marketplace to do the same. Ask yourself the hard question, is your application available or is it just up? >> Yeah, and I think that the confluence of the business logic around what the outcome is, and I think this is the classic cliche, "Oh, it's all about outcomes." Here, you're saying that the outcome can be factored into the policy of the tech, meaning this is the experience we want for our users, our customers, and this is what we determined as acceptable and excellent. That's the new metric, so that's the new definition. You can almost flip the script. It feels like it's being flipped around. Is that the right way to think about it? >> Well, yeah, I think that's actually absolutely correct that an application needs to be business aware, especially in the modern day because all of the businesses that we work with, their applications are really the stock and trade of the business. And so if you create an application that is not business aware, that is just there for its own sake or is not performing according to the revenue goals or the targets of the business, then it's no longer available. >> I mean, it could be little things. It could be like an interface on the UI, it could be something really small or a microservice that's not getting to the database in time or some backup or some sort of high availability. Really interesting things could happen with microservices and DevOps, can you guys share some examples of what people might fall into from a trap standpoint or just from a bad architecture? What are some of the things that they might see in their environment that would say that they need help? >> Yeah, I can probably take that one. So there's a lot of, I call them symptoms of a bad availability experience. And I wouldn't even say it's a pure microservice specific thing. I would say it's really any application that's end-user phasing. I see similar pitfalls. One is a networking issue. I see the number one thing usually with these kinds of issues that networking or config changes that can cause environments to go down. And so when we talk to organizations get to the bottom of this is usually a config wasn't thought through thoroughly, or it was a QAed, they didn't have the proper controls in place. I would say that's probably the number one reasons I see applications go unavailable. I think that's some majority of DevOps teams that can empathize with that is someone did something and I didn't know, and it caused some applications servers go down and it causes cascading event of issues. That's like modern paradigm of issues. On old school days, it's a layer zero issue, someone unplugged something. Well, modern times it's someone pushed something I don't have an idea of what we're doing opposing a downstream effect it would have been and therefore my application went unavailable. So that's again, probably the number one pitfall. And again, I think the hardest problem in microservices still around networking, right? Enterprise level networking and connecting that with many data center applications. For example, Kubernetes, which is the provider or the opera orchestrator of any microservice is still getting to the level, many organizations are still getting a level of comfort with trusting production applications to run on it because one is a skill gap. There's not many large organizations have a huge Kubernetes application team, usually they're fairly small agile units. And so with that, there's a skill gaps, right? How do you network in Kubernetes? How do you persist in storage? How to make sure that your application has the proper security built into it, right? Because that these are all legacy problems kind of catching up with the modern environments, because just because you're modernizing, it doesn't mean these old problems go away. It just take a different form. >> Yeah. That's a great point. Modernization. You guys, can you guys talk about this modern application movement in context to how DevOps has risen really into providing value there? Certainly with cloud scale and how companies are dealing with the old legacy model of centralized IT or security teams who slow things down? Because one of the things that we're seeing in this market is speed, faster developer time to market, time to value. Especially if you're an e-commerce site, you're seeing potentially real-time impact. So you have the speed game on the application side that's actually good, being slowed down by lack of automation or just slow response to a policy or a change or an incident. I mean, this seems to be a big discussion. Can you guys share your thoughts on this and your reaction to that? >> I can tell you that one of the places that we are displacing, one of the markets that we are displacing is the legacy ITOM market, because it can't provide the speed that you're talking about, John. I think about a couple of specific examples. I won't necessarily name the providers, but there are several legacy item providers that for example, require an appliance. They require an appliance for you to administer IT operations management services. And that in and of itself is a much slower way of deploying item. Number two, they require this customized proof of value, proof of concept operation, where companies, enterprise organizations need to orchestrate the customization of the item platform for their use. You buy separate management packs that would integrate with different existing applications on your stack. To us, that's too slow. It means you have to make a bunch of decisions upfront about your item practice and then live with those decisions for years to come, especially with software licenses. So by even moving that entire operation to SaaS, which is what the OpsRamp platform has done, has accelerated the ability to drive availability for applications. Number two, and I'd like to pitch this over to Michael, because I think this is really fundamental to how OpsRamp is driving availability, is the use of artificial intelligence. So when we think about being proactive and we think about moving more quickly, it takes machine learning to do a lot of that work to be able to monitor alert streams and alert floods, especially with the smaller scale down IT teams that Michael has mentioned before. You need to harness the power of artificial intelligence to do some of that work. So those are two key ways that I see the platform driving additional speed, especially in a DevOps environment. And I'd love to hear as well from Michael, additional enhancements. >> Michael, if you don't mind, I'll add one thing. First of all, great call out there, Jordan. Yeah. So the legacy slow down, it's like say appliance or whatever that also impacts potentially the headroom on automation. So if you could also talk about the AI machine learning, AI piece, as well as how that impacts automation, because the end of the day automation is going to have to be lock step in with the AI. >> Yeah. And this kind of goes back to that OpsRamp three pillars of availability, right? So that's the what we do, but again, it's all goes back to the availability problem. But we see that observe, analyze, and act as a seamless flow, right? To have it under the same group or the same tent provides tremendous opportunity and value for our DevOps or IT Ops teams that trust the OpsRamp platform because I'm a big believer that garbage in, garbage out. Having the monitoring data in native or having this data native to your tool provides a lot of meaningful value for customers because they have their monitoring data, which is coming from the OpsRamp tool. They have the intelligence, which is being provided by their ops cube machine learning. And they have our process automation and workflow to feed off that directly. And so when I think of this modernization problem, I really think about modern DevOps teams and the problems they face, which is around doing more with less, that's kind of the paradigm of many teams, each one is trying to learn, how do I do security for Kubernetes? How do I observe my security in the Kubernetes' cluster? How do I make sure my CI/CD pipeline is set up in such a way that I don't need to monitor it, or I don't need to give it attention? And so having a really seamless flow from that observe, analyze, act enables those problems to be solved in a much more seamless way that I don't see many legacy providers be able to keep up with. >> Awesome. Jordan, if you don't mind, I'd love to get your definition of what modern availability means. >> Yeah. So, you know, as I've gone through a little bit previously, so modern availability to me is availability uptime. It's also performance, right? Is the app location marks set down by both the application team, but also by the business. And number three is it business aware. So a truly modern available application is being able, is driving an excellent customer experience according to the product roadmap, but it's also doing it in a way that moves the business forward. Right? And if your applications today are not meeting those benchmarks, if they're performing but they're not driving the business forward, if they're not performing, if they're not up, if they don't meet any one of those three core tenants, they're not truly available. And I think that what's most impactful to me about what the platform, what OpsRamp in particular does in today's environment is operating under that modern definition of available is more difficult than ever. It is more difficult because we are living in a hybrid, distributed, multi-cloud world with tons of software vendors that are being sold into these organizations today that are promising similar results. So when you're an IT operator, how do you drive availability in light of that kind of environment? You have reduced budget. You have greater complexity, you have more tools than ever, and yet your software is more impactful to the bottom line than ever before. It's in this environment that we took a hard look at what's going on in the world, and we say these operators need help driving availability. That's the germination of the OpsRamp platform. >> That's a great point. We're going to come into the culture. And the second Emily Freeman's keynote about the revolution in DevOps talks about this, multiple personas and multiple tools that drive specialism, specialties that actually don't help in the modern era. So I'm going to hold that for a second. We'll come to the cultural question in a minute. Michael, if you don't mind to pivot off that definition, what are the metrics? With all those tools out there, all these new things, what are the new metrics for modern availability? It's more than MTTR. >> Yeah. This whole metrics that I think people spend a lot of time on, I think it's actually people thinking in the wrong direction if you ask me. So I've seen a lot of work. People say that the red metrics, that rate error duration or its views, utilization, saturation errors, or it's these other more contrived application metrics. I think they're looking at a piece of the stack, they're not looking at the right things. Even things like mean time to resolve and critical and server response time, mean time to tech, those are all downstream indicators. I like to look at much more proactive signals. So things like app deck score, your application index, or application performance index, these are things that are much more end-user facing or even things like NPS score, right? This has never really been a classic metric for these operations teams, but what a NPS score shows you is are your users happy using your applications? Is your experience giving what they expect it to be? And usually when you ask these two questions, even if you ask the DevOps team do you know what your Atlas score is? And you use NPS score, but what are those, right? Because it's just never been in that conversation. Those have been more maybe on the business side or maybe on the product management side. But I think that as organizations modernize, we see a much more homogenous group forming among these DevOps and product units to answer these kinds of questions. That's something we focus a lot on OpsRamp it's not seeing the silo of DevOps product or Ops. We're each thinking of how do you have a better NPS and how do we drive a better app decks? Because those are our leading indicators of whether or not our applications available. >> So I want to ask you guys both before, again, back to the own cultural question I really want to get into, but from a customer standpoint, they're being bombarded with sales folks, "Hey, buy my tool. I got some monitoring over a year. I got AI ops. I got observability." I mean, there's a zillion venture back companies that just do observability, just monitoring, just AI Ops. As the modern error is here, what's going on in the psychology of the customer because they want to like clear the noise. We saw it in cybersecurity years ago. Right? They buy everything, and next thing you know, they're going to fog of tools. What's the current state of the customer? What do they need right now as to be positioned for the automation, for the edge, all these cool cloud-scale next gen opportunities? >> Yeah. So in my mind, it's basically three things, right? Customers, number one, they want a vision. They want a vision that understands their position in the enterprise organization and what the vision for application development is going to be moving forward. Number two, they don't want to be sold anymore. You're absolutely right. It's harder and harder to make a traditional enterprise sale nowadays. It's because there's a million vendors. They're just like us. They're trying to get people on the phone and it can be tough out there. And number three, they want to be able to validate on their own with their own time. So in light of that, we've introduced a free trial of our cloud monitoring. It's a lightweight version of the OpsRamp platform, but it is a hundred percent free right now. It is available for two weeks with an unlimited number of users and resource count. And you come in and you can get started on your own using preloaded infrastructure from us if you want, or you could bring your own infrastructure. And we can tell you that customers who onboard through the free trial can see insights on their infrastructure within 20 minutes of onboarding. And that experience in and of itself is a differentiator and it allows our customers to buy on their own terms and timelines. >> Sure. And that's a great point. We brought this up last quarter in the showcase, one of the VCs brought up and says he was an old school VC, kind of still in the game, but he was saying in the old days in shelf where you didn't know if it was going to be successful until like downstream, now it's SaaS. If a customer doesn't see the value immediately. It's there. I mean, there's no hiding. You cannot hide from the truth of value here in the modern era. That's a huge impact on how customers now are evaluating and making decisions. >> Absolutely. And you know, I don't think any customer out there wants to read it on the white paper on the state of enterprise IT anymore. We recognize that and so we are hyper-focused on driving value for our customers and prospects as fast as possible, and still providing them the control that they need to make decisions on their own terms. >> Michael, I've got to ask you, since you have the keys to the kingdom on the product management side, what's the priorities on your side for customers, obviously the pressure's there, you guys are doing great, customers try it out for free. They can get, see the value and then double down on it. That's the cloud way. That's what's DevOps all about. You have to prioritize the key things, what's going on with your world. >> Yeah. And I would say of course prod has their own perspective on this. Our number one goal right now is to accelerate that time to value. And so when we look at one who we're targeting, right? So there's DevOps user, this modern application of operator, what are their core concerns in the world? One is, again, that data problem. Are we bringing the right type of data to solve meaningful problems? And two, are we making insights out of that? So from my priority's perspective, we're really driving more focus on this time to value problem and reduced time to there's some key value metrics we have and I'll go to that, but it's all an effort to make sure that when they hit our platform and they use our platform, we're showing them their return on investment as fast as possible. And so, what a return on investment means (indistinct) can slightly vary, but we try to narrow focus on our key target persona and market and focused on them. So right now it definitely is on that modern DevOps team enterprise, looking to provide modern application availability. >> Awesome. Hey guys, for the last two minutes, I'd love to shift now to the culture. So Jordan, you mentioned that appliance, the item example, which is I think indicative of many scenarios in the legacy old world, old guard school, where there's a cultural shift where some people are pissed off, they're going to go and they slowing things down, right? So you see people that are unhappy, the sites having performance of an e-commerce sites, having five second delays or some impact to the business, and the developers are moving fast with DevOps. The DevOps has risen up now where it's driving the agenda. Kind of impacting the old school departments, whether it's security or IT, central groups that are responding in days and weeks to requests, not minutes. This is a huge cultural thing. What's your thoughts on this? >> I absolutely think it's true. I think the reason were options differ slightly on that is we do see the rise of DevOps culture and how it starts to take control and rest the customer experience back from the legacy providers within the organization, but we still see that there's value in having a foot in the old and a foot in the new, and it's why that term hybrid, we talked about hybrid observability is really important to us. It's true, DevOps culture has a lot of great reasons why it's taken over, right? Increases in speed, increases in quality, increases in innovation, all of that. And yet the enterprise is still heavily invested in the old way. And so what they are looking for is a platform to get them from the old way to the new way fast. And that's where we really shine. We say we can enable, we can work with the existing tool set that you have, and we can move you even more in the future of this new definition of availability. And we can get you that DevOps state of play even quicker. And so you don't have to make a heavy lift and you don't have to take a big gamble right now. You can still provide this kind of slow moving migration plan that you need to feel comfortable, and it doesn't force you to throw away a bunch of stuff. >> And if you guys can comment on whole day two operations, that's where the whole ops reliability thing comes in, right? This is kind of where we're at right now, Dev and Ops. Ops really driving the quality and reliability, availability and your definition. This is key, right? This is where we're starting to see the materialization of DevOps. >> It's why we have guys like Michael Fisher who are really driving our agenda forward, right? Because I think he represents the vision of the future that we all want to get to. And the platform that the product team in OpsRamp is building is there, right? But we also want to provide a path for day two, right? There are still some companies are living in day one and they want to get to day two. And so that's where we drive out here. >> And Michael, the platform with the things like containers really helps people get there. They don't have to kill the old to bring in the new, they can coexist. Can you quickly comment your reaction to that? >> Yeah, absolutely. And I talked to a lot of, I won't name any but large scale web companies, and they're actually balancing this today. They have some infrastructure or applications running on bare metal that somebody's got Kubernetes, and there's actually, it's not so much, everything has to go one direction. It actually is what makes the business, right? Even for migrating to the cloud, there has to be a compelling business reason to do so. And I think a lot of companies are realizing that for the application side as well. What runs where and how do we run it? Do we migrate a legacy monolith to a microservice? How fast do we do it? What's the business impact of doing it? These are all critical things that DevOps teams are engaged with on a daily basis as part of the core workflows, so that's my take on that. >> Guys. Great segment. Thanks for coming on and sharing that insight. Congratulates the OpsRamp, doing really extremely well, right in the right position on ramp for operations to be DevOps, whatever you want to call it, you guys are in the center of it with a platform. I think that's what people want, delivering on these availability, automation, AI. Congratulations and thanks for coming on theCUBE for the Showcase Summit. >> Thanks so much. >> Thank you so much, John. >> Okay, theCUBE's coverage of AWS showcase hottest startups in cloud. I'm John Furrier, your host. Thanks for watching. (relaxing music)

(upbeat music) >> Hello, welcome back to "theCUBE"'s coverage of DockerCon 2021. I'm John Furrier, host of "theCUBE". Great lineup in this event. Got some great guests. Matt Falk, VP of engineering at Orbital Insight. Matt, great to see you. Great keynote, thanks for coming on this cube. Appreciate it. >> Great, thanks for having me, John. >> So you at Orbital Insight, you guys doing some cutting edge work. Geospatial, big data, real-world problems. I mean, it's almost sci-fi for me. I mean, I just love how space, cybersecurity, data, all kind of rolling into like this whole very cool vibe. You know, drones, satellites, all this kind of you know, stuff going on in the cloud. But there's like real action happening, right? (chuckles) We all live on GPSes. Like, this is like very cool and relevant technology happening right now. Give us your take. What are you guys seeing, how's business? Give us a quick overview of what your journey is and how you guys are executing. >> Sure. And I think you're right there, it is a little bit like sci-fi actually, even to myself. Even having been in the industry for a few years at this point. You know, we all think about big data, it's become much more a thing especially over the past decade or two. Everything that we try and solve as big data, artificial intelligence, machine learning, they all thrive, they all need this big data. An untapped area about big data though, is geospatial data. And really data that comes from overhead sensors, coming from space. So to me, that feels a little bit like sci-fi like you're saying because that time is now. That time for us to be able to use and harness that data and provide actual or meaningful insights is here. You know, as a company for Orbital Insight, we got into it about seven, eight years ago and the title wave of this data was just forming. There weren't as many satellite provider companies, there weren't as many different types of disparate geospatial data. And by geospatial data you know, anything with a latitude and longitude associated with it, right? This data was, it was there but it wasn't as abundant there. It wasn't clear about how we could use that data. And over the past few years so many new use cases had really popped out and so many new disparate types of data and it's really about the fusion of all of them and getting more and more of that data. So right now the most exciting thing is really just how much of that data exists and how much is going to exist in the next few years. And honestly, we want to ride that tidal wave along with our customers. We can deal with many different types of data here. It's overhead satellite imagery, it's cell phone pings, it's identification system from ships, it's everything that you can get your hands on and incorporate into this platform. And then using this to feed the artificial intelligence and machine learning algorithms to derive new insights so it's sci-fi but it is here. >> Yeah and it's real computer science problems too. A lot of networking as well. You got this and it's transitioning too. You were out early doing these new use cases. But what's interesting about your journey and I want to get your thoughts on this, is that you guys really evolve from tackling these first kind of time problems, making solutions out of them to sequencing it to a fully on, fully-built scalable insight platform. Okay and this the pattern that we see in cloud native. Companies go from going in and doing things, that one-off, one-offs projects, POCs and then sequencing to a either cloud native or full blown platform. You guys have had that journey, take us through that effort and what's the result today? >> No, that's exactly right. The way we started, just like you mentioned, with many other companies was really around this proof of concept idea. It was going out, talking to customers, finding what their pain points were and figuring out what can we do to solve those pain points. So it was all about, "Okay, for this particular customer "take this data set, take satellite imagery "of this location, take cell phone things for this location, "take a digital elevation model from this area "of the planet, fuse them together in some "very specific custom way to try and solve that problem." And that's how we started. Over the first few years of that, this doesn't really scale as well. We had to keep building new solutions to solve new problems. What we started to identify was that there's so much commonality, there's so much overlap between a lot of these problems. So the solutions for them you know, if we're taking truck counts, if we're able to look at parking lots and detect cars from satellite imagery and use that to determine level of trends or sales at you know, a retail store. Well, we can also use that same overhead imagery to detect cars and look for their movement patterns, look for how they're going from a port to a particular warehouse or how they're driving on the road. Same thing with trucks. When we started identifying that a lot of the value from these different analytics and data sources can be used to solve many different problems in different ways but the underlying core technology is very similar. So for us about a few years ago, about two or three years ago at this point, we started really developing this into a platform that can solve generally any question about geospatial data analytics. So instead of, "I have a very specific problem, I wanted "to count the number of trees in this particular area." It's, "No, no, you want to do something similar to land use. "You want to try and classify different areas of the planet "that are covered in this particular type of land use "and then use that as an estimate for how many trees "there are." So we started to find these commonalities and that allowed us to really build this generic platform we call GO and use that to start solving many, many more questions. >> That's good. So real world problems are emerging. I'd love to get your thoughts on what those geospatial problems are because you can almost think about how the traditional distributed computing world looks at the edge for instance. Like you know, the intelligent edge or industrial edge, edge of the network as they call it in the hybrid world. You're bringing not just moving packets, you're talking about other data media. So pictures, images and (chuckles) different data sources. This is a huge aperture that changes the game on the analytics. Could you share some of the problems you saw that opened up with the new types of sources? Because it's not just packet to a device or an edge point in the traditional sense. >> Sure, sure. No, that's a great question for us. And I think a lot of people that don't really understand or haven't dealt with geospatial data before, don't understand necessarily a lot of the nuances that come with this data, there are quite a few of them. So I'll focus on satellite imagery in particular for the first part here. But you know, when people think about, even today what we can do with artificial intelligence. Computer vision on an image, you know, most people go to the images they can see from the cell phones. Most people start to think about self-driving cars and seeing the resolution of those images. Well, when you're dealing with overhead imagery, satellite imagery in particular, the game changes, the game is completely different. Yes, you're still working with an image but so many of the pieces of that image are just different and more complex. So for instance, when you're taking an image miles away from the planet, the first thing you have to realize is your resolution is not quite what people think. In particularly now, if we go outside and we stand in a satellite image, the best commercial sensors today, we're at most one pixel. And with one pixel you really can't identify people. You know, even a car or a truck at most, you're looking at 15 to 20 pixels and it becomes extremely difficult to classify objects at that resolution or at that scale compared to if you're using a cell phone picture, for instance. So that's really for us the first you know, major difficulty or change that comes into play. The second one would be the temporal aspect. So not only the spatial resolution but temporal resolution. You don't get an image every day. You don't even get an image, sometimes every week. So how do you, do you impute different data, impute different points to make your overall analysis worthwhile? Again, a slew of additional challenges that come with that. Other things like for instance georegistration or orthorectification. So the idea, when you take a satellite image of somewhere on the planet, you actually don't get very precisely where that image is. It could be off by five, 10, 15 or 20 meters even and you have to do work to actually relocate that image in the right particular position. Orthorectification refers to the angle at which the image was taken. If you're taking an image from a bird's eye view, yeah you're going to be able to see straight down and you're going to see buildings and they look like you know, squares of certain right edged polygons like that. But you can also take an angle 30 centimeters or excuse me, 30 degrees off, you can look at the side when you're taking this image and then the satellite image looks completely different. So that's another technique that we have to combat and another difficulty we have to work with in order to make this data usable. So for satellite imagery for other data sources too, there's a slew of additional challenges that we have to compete, that we have to fix and work with to make this data usable. And that's what our platform does. It takes this data, it fixes all those issues and allows you to compute the analytics on top of them. >> I love it. I mean, as a consumer, I can relate to like, say Google. I know there's trees there but they look like they just put trees there 'cause they can. Like there was a tree there and they fill it out, right? I mean kind of similar things going on there. All great stuff. I love the tech and I think it's going to be one of these eras that's going to be super valuable as more of these use cases can get this tap the data for not only insights but also maybe for features in software. Which brings me to the next question. How do people use you guys? I mean, as you have these use cases that are emerging, a lot of disparate use cases, different data sources now analytics as a platform, are you guys selling software is it a service, as a fee? Can you explain you know, how I might want to geek out and integrate you into my product or feature? Or do you do it that way? How does it work? >> Oh sure, thank you. So, there's two different aspects here really. And one of them is how people actually have access to this data. So the first is that we actually make this data available. And the second is the analytics that we put on top of this data. So for the first piece, a lot of these data sets are extremely expensive. So the average even consumer or a lot of businesses, it's much too expensive to go and actually buy all this satellite imagery or all this geolocation ping data or shipping information. It's just too expensive to buy these disparate data sources if you only have particular, single need for them. So the first thing our platform does, is it integrates with many different data providers. It integrates with, like I said, anything that has a latitude and longitude with it, we try and get that into our platform. And we become the broker almost, the provider of all that disparate data into a single unified source. So that's that first aspect, that's how they use us to get access to that data that otherwise they wouldn't be able to get access to. The second piece is the analytics. So for this, our platform, it does really you put three things into our platform. You ask where you want to look on the planet, what you want to look for and when do you want to look for it? And our platform takes care of going and getting all that information that it needs to compute that answer. Then using our custom analytics to derive what you're looking for, the question you're actually asking and produce similar to a data feed but it's much more custom than that, particular insights for the customer. So what comes out of the platform is effectively a time series that you're able to go explore and drill down into further. So a particular example of this is for supply chain right now and this is a problem that we're very passionate about right now. Especially with last year, how COVID impacted things. But from a supply chain perspective, we're actually able to identify locations on the planet that you're interested in, typically operating facilities and start looking at trends for where people are going to and from that particular facility. So we can see, "Oh, there were a hundred people that visited "this facility on a you know, the last seven days." And maybe produce a time series of how many people were there each day. But we can also then say, "Of those hundred people, "16 of them came from this location, 52 of them came "from this location, 53 of them visited this location "two days after visiting that location." And we can start to build this entire traceability map of that particular location and that our customers can use to identify patterns and then anomalies really, in their own supply chains. Or different things about their operating facilities. >> So pinging, like graph data, for instance. We got some insights into how to restructure their value chains or reconfigure their economics. Something like that would be like a use case. >> Exactly, exactly. Finding further efficiencies, ways they can optimize their supply chains or anticipating disruptions in it. If they know that part of their supply chain is dependent on you know, a particular facility, a particular location or a particular region. And they know from other news that something is about to happen to that region, they can know practically how to change their supply chain in order to you know, alleviate that pain before it even happens. >> Well, real time in the news just recently, just this month earlier in the month, we saw that gas shortage or stoppage or shortage/supply chain disruption, happen in the East coast, right? From the pipeline hack, the ransomware attack. That's a good example. I mean, some people don't even know the difference between a supply chain disruption and a shortage, there are two different things so I saw that big debate happening. This is kind of real world example where you can say, "Okay, we have a supply chain, "potential predictive disruption." Then maybe look at ways to do that. Am I thinking in the right way here? >> No, that's exactly the right way to think about it. You can start to see... So from that event, if you're operating a facility or a facility warehouse, you can look at that event, ask the question of, "How is this going to impact "my supply chain?" And the first thing you need to know is are you dependent on that, is that something that actually impacts or it plays a part in your supply chain? So you'd use our software, plug into your own operating facility, start to trace where people are coming from or going to. First thing you can do then, is identify, "Is that location "part of my supply chain?" If not, you know potentially you're in the clear. If so, then you can start to identify different locations that might be a suitable replacement for your supply chain. So can you practically avoid that going forward and make that move sooner than you would've been able to otherwise. >> I love the complexity challenge here. You guys doing the heavy lifting here in offering as a service makes total sense. You can almost democratizing the whole complexity of the data acquisition and then you know, providing value on top of it. The question I have for you is, what other learnings have you had? I mean, what was some of the difficulties? You mentioned you know, the artifacts, atmosphere, haze, noise, spatial temporal frequencies before. What are some of the other things that you're seeing and learnings that people might not know about that you guys have solved in this data capturing from the satellites? >> That's a great question. And there's plenty of them. A lot of things I think it would come down to is how to use this data or how best to combat some of the challenges like we talked about earlier, come with this data set. In particular, if we look at the foot traffic data that we look at. So pingings coming from different cell phones or what we call geolocation pings. Largely, you can think of that as any IOT device that's pinging their location you know, we can aggregate that data in and start using it within the platform. And what we've learned for that data is, it's very dependent on how you can actually get that to be normalized. And what I mean by that is none of this data is providing a complete picture of what's actually there. So again, if we look back at you know, even from image perspective when you're getting a satellite image when you're getting cell phone pings, you're only getting at best, 15 to 20% of the actual picture. And the challenges are really about going from that 20% view to the full contextual 100% view. So tactically, what that looks like for geolocation pings, where you're not getting geolocation pings from every person on the planet, we're not getting pings from every IOT device or every cell phone. We're getting a particular, almost randomized subset of those pings and they're all anonymized. So how do you go from that to an actual insight? How do you go from that to a full complete view of what's happening? And that's where our normalization algorithms come into play and other capabilities that we have that take that data and try and extrapolate what's truly happening. So if you're looking at you know, for instance if you look at a gas station. And it's a gas station in a you know, an area that's not very highly populated. And you're only getting two or three pings a day or some days you're getting none. Is that truly a signal of no one's going to that gas station or are you just missing the data? And you don't always know so part of what we've learned is how to take that data and actually translate it into the complete picture. We have very complex algorithms and they're constantly being improved on to account for differences like people turning their cell phone off or more than one person being in a car or things like that. So that's what we've really learned in it. It's all about taking that incomplete picture and trying to produce the most complete picture with as most context as possible to solve problems. >> So what's the secret sauce on all of this? Is it algorithms, is it data usage, all the above? I mean, take me through some of the secret sauce that's going on that you guys are building to make all this work. >> Sure, sure. And I'll go into it as much as I can. (John and Matt laugh) >> (speaking faintly) >> But there's a few different- Exactly. But a few different pieces really. The first one is the data itself, right? At the end of the day, no matter how good your machine learning capabilities are, if you don't have the data, you can't do anything. And this is true for all types of artificial intelligence or machine learning algorithm. If you don't have something to allow the system to learn from, you're at a loss. So the first piece of it really is getting the right data and making sure we have enough different or disparate data sources to really complete that overall picture. The second piece of it is allowing our platform to do this at a high scale. So it'd be one thing if you can produce a particular algorithm and get it to run in a single location one time. But it's all about for us, asking that aggregate question. So we're not you know, if somebody is asking about a particular gas station or a retail store, more often than not, they're not caring about just one location. They're caring about the aggregate, they're trying to look at this country as a whole and seeing what the trends or patterns are. So the second piece of secret sauce really is our platform and the ability to scale that up dynamically and allow you to ask any size question. So not just one AOI at a particular location but thousands of different locations and how that answer really compiles together. Third one is definitely the artificial intelligence and machine learning. For us, that is a extremely core competency. Something that allows us to really take that data and produce the insights. And that's a key factor of it. Like I mentioned, with the different challenges, part of our secret sauce there is not just the algorithms themselves but additional techniques or different R&D that we can do to solve or combat some of the additional issues that we have with overhead sensors. In particular, I'll point out two here but one is the rare object issue. So a lot of times if you're doing with satellite imagery and you're trying to find an object, it's very difficult to find a satellite image of that object. If you're looking for a particular type of ship you might only find one or two of them in thousands of images. So how do you build a machine learning algorithm that really uses that really small amount of data to produce an algorithm? And this is where our R&D capabilities come into play. And one to highlight is synthetic data. So the ability to produce almost fake or generated satellite images that actually produce these objects you're looking for so that we can train or learn off of that. So things like that really build our, I'll say, our secret sauce, are our R&D core competencies. The ability to produce newer novel techniques to generate data where satellite images or other geospatial data have deficiencies that we can combat. >> Yeah, I like that feature. Because you're almost saying, "The ship might look like this "depending upon where they're looking and muting that in there, good call. I guess the question I have for you is first of all, great tech loved the story. You guys are onto some really cool stuff and very relevant. The question, is in minds of peoples right now who are watching is why now for the critical time, why is now a critical time for geospatial analytics? What's your answer to that? >> Sure. That's a great, actually the answer is great for us too as a company. As I was kind of alluding to in the beginning, there's this tidal wave of geospatial data. And you know, if we were to look at five, 10, 15 years ago, the data itself and the technology was not really there to allow us to do what we're trying to do now. If you look back, I think it was in 2013, there was a particular computer vision paper that came out that really was the birth of the CNN world. And for that, that is a core compute capability that allows us to do the computer vision we need to be able to do. So that was a extreme catalyst for companies like ours, are being able to do this type of data fusion analytics. And the second one is the birth of the new sensors coming up right now. If you look back five years ago and where we're going five years from now, it's almost like Moore's Law. Where every other year, things are just starting to double. There's more and more satellites being thrown up, there's more and more data being thrown out and frankly, it's almost too much data at this point. There's just more and more data coming up. We already have petabytes of satellite imagery in our system, hundreds of terabytes of IOT device data and it's everyday just more and more of this data is coming up and being produced. So now is that perfect time because the data is finally there and it's only getting better over time. >> Yeah, I know we have a little bit of time left. I do want to ask just kind of, I'm curious. I'm sure people are too. As leader in that company, as engineering leader, you got a team that's working on some pretty cool stuff. A lot of computer science, a lot of new technology opportunities, kind of new problems that are emerging that are exciting. So everyone likes to solve hard problems, right? You got one, right? You got synthetic data, massive ingestion pipelines, normalizing algorithms, spatial imputation, et cetera, all this good stuff. How do you organize, how do you attract people? How are you looking at this? Because you have to lean into this. It's not like you're waiting for the market to come to you. You guys are going out there, making the market technically as well. So how do you organize, how do you recruit? (chuckles) Take us through some of the inside the ropes there. >> Sure, sure. So I'll start with kind of just how our engineering team is organized right now and where are we try and do find people and pull additional folks into our team. Right now we are split into four or five different areas. So like most cloud based platforms, we do have an infrastructure team. So you know, DevOps, site reliability, IT, everything that goes into that core cloud layer. So we do have an infrastructure team that builds that. On top of that, we do have our platform engineering team. So that team largely builds our microservices that play together to produce our external API. On top of that, we have a product engineering team that builds really with developing our UI, adding in our UX, making sure everything on top of the API plays nicely together. And also building a few additional dockerized computer vision and machine learning models that can plug into the platform. Separately, we have our R&D team. This is like as you know, where we talked about our synthetic data and all the other research areas that we get into, they focus there. Then we have our senior data engineering team. This team is largely focused on pulling disparate data sources, massaging them, cleaning them up into the right format so that it can be plugged into our platform. So from this, this is kind of how our team is structured. You're right, it's a ton of technical challenge. A lot of fun challenges. We're about 50 engineers right now. We're actually, we're looking to grow almost doubling in size by the end of the year. We're going to be bringing on an additional 30 people over the next few months. And what we're looking for is people from a wide area of expertise. So people that have you know, microservice core platform backgrounds, able to build on the backend system, deal with tons of tons of you know, transactions per second and really allow us to scale our platform. That's one set of expertise we like. Another one is people that really just have geospatial data backgrounds. And which to be honest at this point, it's somewhat of a rare niche finding people that have worked on a platform but also worked in geospatial data. But that's something that we love to bring into our system so we can add additional expertise and eventually get new data sources in. And then lastly, it's really around that core competency of machine learning and artificial intelligence. So we will look for anybody that has machine learning you know deep math, deep computer science background to come in and be a part of that team. If they're capable from a research perspective, we are actually you know, it's possible to teach them some of the computer vision aspects as well. So, if they have a computer vision background, great. If they have a data science and machine learning background, great. We want that diverse set of interests and diverse set of thinking to come in and really build our R&D team as well. >> Yeah. And I obviously DockerCon is here. You're talking about containers and that leads into Kubernetes, microservices, all kinds of cloud native technologies. Because what you guys are doing is you're taking an old construct. I mean, fairly old, I mean it's you know, it's data. But you're leveraging it in new ways. In a way that's kind of what cloud native's about. How are you seeing that world evolve? Obviously we're here at DockerCon, containers helps big time thoughts on the containerization wave that continues. And you got Kubernetes and more and more cloud natives, more SRE's are going to be hired. Again, people are scaling up. What's your take on what's going on around DockerCon? >> No, this is actually for us. It's really powerful and it's a really powerful tool. Whether it's Dock or (speaking faintly), the idea of containerization as a whole, it really allows our platform to get to that next level of scale. One piece I you know, originally we were not a microservice platform. Like I said, we were starting to do some more POCs. As we got into this platform play, one of the things we knew we needed to be able to do was scale different parts of our system. So whether it was scaling to ingest more data, scaling to involve new algorithms or scaling really to involve or be able to compute massive computation requests that come from our customer. This requires different pieces of our system to scale. If we were a monolithic application, if we were running on premise, that type of scale just wouldn't really be possible at the level that we need it to. So for us, the solution is all around being able to dockerize different parts of our system, keep them isolated, keep them talking to each other via different interfaces. And then as need be horizontally scale different pieces of our system to compat with that. So really the you know, Kubernetes Docker together, the ability for us it's allowing our developers to focus on the code that they need to be writing and not focusing on the SRE or the DevOps perspective of it. And then letting our DevOps team use these additional tools to make themselves more efficient. You know, we can do that with a smaller team now we don't need a team of 50 people on DevOps or infrastructure. You can do it with five or six solid engineers that can really you know, manage your entire environment. >> Yeah, I think having that horizontal scalability is critical and the containerizing it, so many benefits there allowing things to be completely portable and integrating really well. Great stuff. Unbelievable gems dropped here. My final question for you while I got you here is you know, as you look at other peers and people in the marketplace, the people who were on the right side of history are experiencing, certainly entrepreneurs and people who are in businesses and enterprises are waking up and going, "Hey, that can really change the game and flip "the script with cloud native." So people are experiencing similar journeys where they got product engineering saying, "We are more of a platform. "I could sequence and build out that platform and then build "my infrastructure on the cloud." So you're starting to see these point applications turn into platforms. What's your advice to people out there that are going to move from product engineering departments or groups to bring on that platform construction or that work and then build that infrastructure like you guys are? What's your advice to folks that are going to make that journey? >> No, that's a great question, John. I think the quick advice I would give to anybody you know, that has a product engineering team considering moving to the platform right now is do it now. There's no time better than right now and what I mean by that is, the longer you delay the harder it gets. You're going to be missing out on a lot of the new technologies that are really being solidified as part of the cloud computing world. Yes you know, there are trade-offs. Especially you know, you might have to go to your exec team or your product team and make these trade-offs and you won't be able to develop teachers as quickly as you're spending time porting to a platform play. But the benefits are amazing. And once you actually get there, you'll really be thankful that you took the time to do it. Yes it's, you know again, it's going to be challenging because it's one of those things where it's an engineering benefit at first. It's not going to be something you're going to say, "Yes product, "in two months, you're going to get this benefit from it." Or, "In you know, three months, you're going to get "this benefit for it." It's, "One year from now, this is how our platforms "are written, our new product is really going to be able "to expand and grow." And the best way to get there is to just do it now. Really starting encapsulating your system, break it out into different pieces, put it in a cloud, allow it to scale. And so yeah, my advice is to just bite the bullet and do it now. >> So people who buy into that notion of moving from monolithic to microservice based applications want that horizontal scalability, as you mentioned. What are some of the first principles in that platform? What's on the mind of the architect or the leader as they start thinking about those first principles for the modern platform? >> Sure. I'd say the first one is don't over design. So some people have a tendency when they start thinking about microservices is they really go to microservice almost nanoservices. They really start breaking off you know, as many different pieces of the code, making them as small as possible. And to some extent, that's what you want to do with microservices but you don't want to go too far. I mean, it's easy to go down that rabbit hole. So in particular, there are certain services or microservices that you find out, they're tightly coupled. They're constantly passing data back and forth and that's when you realize, passing data back and forth between two different logical separation of code, it takes time. So it might make sense for them to be one unified microservice as opposed to two. So the most important thing to think about is you know, what pieces really make sense to logically separate and how does that actually impact the flow of data or flow of information through your system? If you're adding too many hops between you know, a certain end point and the call to the backend system, it might be time to rethink the way you're breaking system the system down. But you really want to start out with what can be broken down into the logically encapsulated pieces? And that's where we want to pull our microservices. >> Highly cohesive decoupling, that's a concept. An operating system as we say, it's the platform, that's the cloud. >> It's not new, that's right. >> It's been around. Matt, great interview. Thanks for dropping the gems and sharing your knowledge. And congratulations for the work you're doing at Orbital Insight. Great focus, love the company, love the excitement. Thanks for coming on. >> Perfect. Pleasure chatting with you too, John. And thanks for having me. And thanks for having me be a part of DockerCon. >> All right. DockerCon 2021, CUBE coverage. I'm John Furrier, host of "theCUBE". Thanks for watching. (lighthearted music)

>>Yeah, yeah. >>Welcome back to our final session in cultivating a data fluent culture track earlier today, we heard from experts like Valerie from the Data Lodge who shared best practices that you can apply to build that data flew into culture in your organization and tips on how to become the next analyst of the future from Yasmin at Comcast and Steve at all Terex. Then we heard from a captivating session with Cindy Hausen and Ruhollah Benjamin, professor at Princeton, on how now is our chance to change the patterns of injustice that we see have been woven into the fabric of society. If you do not have a chance to see today's content, I highly recommend that you check it out on demand. There's a lot of great information that you could start applying today. Now I'm excited to introduce our next session, which will take a look at how the democratization of data is powering digital transformation in the insurance industry. We have two prestigious guests joining us today. First Jim Bramblett, managing director of North America insurance practice, lead at its center. Throughout Jim's career, he's been focused on large scale transformation from large to midsize insurance carriers. His direct experience with clients has traditionally been in the intersection of technology, platform transformation and operating remodel redesign. We also have Michael cast Onus, executive VP and chief operating officer at DNA. He's responsible for all information technology, analytics and operating functions across the organization. Michael has led major initiatives to launch digital programs and incorporating modern AP I architectures ER, which was primarily deployed in the cloud. Jim, please take it away. >>Great. Thanks, Paula E thought we'd cover a few things today around around data. This is some of the trends we see in data within the insurance sector. And then I'll hand it over to Michael Teoh, take you through his story. You know, I think at the macro level, as we think about data and we think about data in the context of the insurance sector, it's interesting because the entire history of the insurance sector has been built on data and yet, at the same time, the entire future of it relies on that same data or similar similar themes for data. But but different. Right? So we think about the history, what has existed in an insurance companies. Four walls was often very enough, very enough to compete, right? So if you think about your customer data, claims, data, CRM, data, digital data, all all the data that was yeah, contained within the four walls of your company was enough to compete on. And you're able to do that for hundreds of years. But as we we think about now as we think about the future and the ability to kind of compete on data, this data comes from many more places just than inside your four walls. It comes from every device, every human, every vehicle, every property, every every digital interaction. Um in upon this data is what we believe insurers need to pivot to. To compete right. They need to be able to consume this data at scale. They need to be able to turn through this data to drive analytics, and they serve up insights based on those analytics really at the desktop of insurance professionals. And by the way, that has to be in the natural transition of national transaction. Of that employees work day. So an underwriter at a desktop claim him on the desktop, the sales associate of desktop. Those insights need to be served up at that point in time when most relevant. And you know. So if we think about how insurance companies are leveraging data, we see this really on kind of three horizons and starting from the left hand side of the page here, this is really brilliant basics. So how my leveraging core core data and core applied intelligence to monetize your existing strategy? And I think this brilliant based, brilliant basics concept is where most of most of my clients, at least within insurance are are today. You know, how are we leveraging data in the most effective way and putting it in the hands of business decision makers to make decisions largely through reporting and some applied intelligence? Um, Horizon two. We see, you know, definitely other industries blazing a trail here, and this is really about How do we integrate ecosystems and partners Now? I think within insurance, you know, we've had data providers forever, right? Whether it's NPR data, credit data risk data, you know, data aggregators and data providers have been a critical part of the insurance sector for for decades. I think what's different about this this ecosystem and partnership model is that it's much more Oneto one and it's much more, you know, kind of. How do we integrate more tightly and how do we become more embedded in each other's transactions? I think that we see some emergence of this, um, in insurance with automotive manufacturers with building management systems. But I think in the grand scheme of things, this is really very, very nascent for us as a sector. And I think the third horizon is is, you know, how do we fundamentally think about data differently to drive new business models? And I, you know, I don't know that we haven't ensure here in North America that's really doing this at any sort of scale. We certainly see pilots and proofs of concepts. We see some carriers in Europe farther down this path, but it's really it's really very new for us. A Z Think about these three horizons for insurance. So you know what's what's behind all this and what's behind. You know, the next powering of digital transformation and and we think at the end of the exercise, its data data will be the next engine that powers digital transformation. So in this exhibit, you know we see the three horizons across the top. You know, data is activated and activating digital transformation. And this, you know, this purple 3rd, 3rd road here is we think some of the foundational building blocks required to kind of get this right. But I think what's most important about about this this purple third bar here is the far right box, which is business adoption. Because you can build this infrastructure, you can have. You know, this great scalable cloud capability. Um, you can create a bunch of applications and intelligence, but unless it's adopted by the business, unless it's democratized, unless those insights and decisions air served up in the natural course of business, you're gonna have trouble really driving value. So that way, I think this is a really interesting time for data. We think this is kind of the next horizon to power the next age of digital transformation for insurance companies. With that brief prelude, I am, I'm honored. Thio, turn it over to Michael Stone Is the Cielo at CNN Insurance? >>Thanks, Jim, for that intro and very exciting Thio be here is part of part of beyond when I think a digital transformation within the context of insurance, actually look at it through the lens of competing in an era of near perfect information. So in order to be able to deliver all of the potential value that we talked about with regard to data and changing ecosystem and changing demands, the question becomes, How do you actually harness the information that's available to everybody to fundamentally change the business? So if you'll indulge me a bit here, let me tell you just a little bit more for those that don't know about insurance, what it really is. And I use a very long run on sentence to do that. It's a business model where capital is placed against risk in the form of products and associated services sold the customers through channels two companies to generate a return. Now, this sounds like a lot of other businesses in across multiple industries that were there watching today. But the difference within insurance is that every major word in that long run on sentence is changing sources of capital that we could draw on to be able to underwrite risk of going away. The nature of risk itself is changing from the perspective of policies that live six months to a year, the policies that could last six minutes. The products that we're creating are changing every day for our ability to actually put a satellite up in the air or ensure against the next pandemic. Our customers are not just companies or individuals, but they could be governments completely different entities than we would have been in sharing in the past and channels were changing. We sell direct, we sell through brokers and products are actually being embedded in other products. So you may buy something and not even know that insurance is a part of it. And what's most interesting here is the last word which is around return In the old world. Insurance was a cash flow business in which we could bring the premium in and get a level of interest income and being able to use that money to be able thio buffer the underwriting results that we would have. But those returns or dramatically reduced because of the interest income scenario, So we have to generate a higher rate of return. So what do we need to do? Is an insurance company in through this digital transformation to be able to get there? Well, fundamentally, we need to rethink how we're using information, and this is where thought spot and the cloud coming for us. We have two basic problems that we're looking to solve with information. The first one is information veracity. Do we believe it? When we get it? Can we actually trust it? Do we know what it means when we say that this is a policy in force or this is a new customer where this is the amount of attention or rate that we're going to get? Do we actually believe in that piece of data? The second is information velocity. Can we get it fast enough to be able to capitalize upon it? So in other words, we're We're working in a situation where the feedback loop is closing quickly and it's operating at a speed that we've never worked in before. So if we can't solve veracity and velocity, then we're never going to be able to get to where we need to go. So when we think of something like hot spot, what do we use it for? We use it to be able to put it in the hands of our business years so that they could ask the key questions about how the business is running. How much profit of my generating this month? What brokers do I need to talk? Thio. What is my rate retention? Look like what? The trends that I'm seeing. And we're using that mechanism not just to present nice visualizations, but to enable that really quick, dynamic question and answer and social, socially enabled search, which completely puts us in a different position of being able to respond to the market conditions. In addition, we're using it for pattern recognition. Were using it for artificial intelligence. We're gonna be capitalizing on the social aspect of of search that's that's enabled through thought spot and also connecting it into our advanced machine learning models and other capabilities that we currently have. But without it solving the two fundamental problems of veracity and velocity, we would be handicapped. So let me give you some advice about if I were in your position and you don't need to be in sleepy old industry like insurance to be able to do this, I'll leave you with three things. The first one is picking water holes so What are the things that you really want to be good at? What are the pieces of information that you really need to know more about? I mean, in insurance, its customers, it's businesses, locations, it's behavior. There are only a few water also really understand and pick those water holes that you're going to be really good at. The second is stand on the shoulders of giants. You know, in the world of technology, there's often a philosophy that says, Well, I can build it something better than somebody else create if I have it in house. But I'm happy to stand on the shoulders of giants like Thought Spot and Google and others to be able to create this capability because guess what? They're gonna out innovate any of the internal shops all day and every day. So don't be afraid. Thio. Stand side by side on the shoulders of giants as part of your journey. Unless you've got to build these organizations not just the technology for rapid experimentation and learning, because guess what? The moment you deliver insight, it begs another question, which also could change the business process, which could change the business model and If your organization the broader organization of business technology, analytics, customer service operations, etcetera is not built in a way that could be dynamic and flexible based on where the market is or is going, then you're gonna miss out on the opportunity. So again, I'm proud to be part of the fast black community. Really love the technology. And if if you look too, have the same kind of issues with your given industry about how you can actually speed up decision making, deliver insights and deliver this kind of search and recommended to use it. And with that, let's go to some questions. >>Awesome. Thank you so much, Michael and Jim for that in depth perspective and those tangible takeaways for our audience. We have a few minutes left and would love to ask a few questions. So here's the first one for Michael Michael. What are some of the most important things that you know now that you didn't know before you started this process? I think one of >>the things that's a great question. I think one of the things that really struck me is that, you know, traditional thinking would be very use case centric or pain point centric Show me, uh, this particular model or a particular question you want me to answer that can build your own analytics to do that or show me a deficiency in the system and I can go and develop a quick head that will do well, then you know, wallpaper over that particular issue. But what we've really learned is the foundation matters. So when we think about building things is building the things that are below the waterline, the pipes and plumbing about how you move data around how the engines work and how it all connects together gives you the above the waterline features that you could deliver to. You know, your employees into your customers much faster chasing use cases across the top above the waterline and ignoring what's below the water line to me. Is it really, uh, easy recipe too quick? Get your way to nothing. So again, focus on the foundation bill below the water line and then iterated above the water line that z what the lessons we've learned. It has been very effective for us. >>I think that's a very great advice for all those watching today on. But Here's one for Jim. Jim. What skills would you say are required for teams to truly adopt this digital transformation process? >>Yeah, well, I think that's a really good question, and I think I'd start with it's It's never one. Well, our experience has shown us number a one person show, right? So So we think to kind of drive some of the value that that that Michael spoke about. We really looked across disciplinary teams, which is a an amalgamation of skills and and team members, right? So if you think about the data science skills required, just kinda under under understand how toe toe work with data and drive insights, Sometimes that's high end analytic skills. Um, where you gonna find value? So some value architectural skills Thio really articulate, you know, Is this gonna move the needle for my business? I think there's a couple of critical critical components of this team. One is, you know, the operation. Whatever. That operation maybe has to be embedded, right, because they designed this is gonna look at a piece of data that seems interesting in the business Leader is going to say that that actually means nothing to me in my operation. So and then I think the last the last type of skill would be would be a data translator. Um, sitting between sometimes the technology in the business so that this amalgamation of skills is important. You know, something that Michael talked about briefly that I think is critical is You know, once you deliver insight, it leads to 10 more questions. So just in a intellectual curiosity and an understanding of, you know, if I find something here, here, the implications downstream from my business are really important. So in an environment of experimenting and learning thes thes cross discipline teams, we have found to be most effective. And I think we thought spot, you know, the platform is wired to support that type of analysis and wired to support that type of teaming. >>Definitely. I think that's though there's some really great skills. That's for people to keep in mind while they are going through this process. Okay, Michael, we have another question for you. What are some of the key changes you've had to make in your environment to make this digital transformation happen? >>That's a great question. I think if you look at our environment. We've got a mixture of, you know, space agent Stone age. We've got old legacy systems. We have all sorts of different storage. We have, you know, smatterings of things that were in cloud. The first thing that we needed to do was make a strong commitment to the cloud. So Google is our partner for for the cloud platform on unabashedly. The second thing that we needed to dio was really rethink the interplay between analytics systems in operational systems. So traditionally, you've got a large data warehouses that sit out over here that, you know, we've got some kind of extract and low that occurs, and we've got transactional operational systems that run the business, and we're thinking about them very differently from the perspective of bringing them together. How Doe I actually take advantage of data emotion that's in the cloud. So then I can actually serve up analytics, and I can also change business process as it's happening for the people that are transacting business. And in the meantime, I can also serve the multiple masters of total cost and consumption. So again, I didn't applications are two ships that pass in the night and never be in the world of Sienna. When you look at them is very much interrelated, especially as we want to get our analytics right. We want to get our A i m all right, and we want to get operational systems right By capturing that dated motion force across that architecture er that was an important point. Commit to the cloud, rethink the way we think analytics systems, work and operational systems work and then move them in tandem, as opposed to doing one without the other one in the vacuum. >>That's that's great advice, Michael. I think it's very important those key elements you just hit one question that we have final question we have for Jim. Jim, how do you see your client sustain the benefits that they've gained through this process? >>Yeah, it's a really good question. Um, you know, I think about some of the major themes around around beyond right, data fluency is one of them, right? And as I think about fluency, you only attain fluency through using the language every single day. They were day, week, over week, month over month. So you know, I think that applies to this. This problem too. You know, we see a lot of clients have to change probably two things at the same time. Number one is mindset, and number two is is structure. So if you want to turn these data projects from projects into processes, right, so so move away from spinning up teams, getting getting results and winding down. You wanna move away from that Teoh process, which is this is just the way working for these teams. Um, you have to change the mindset and often times you have to marry that with orb structure change. So So I'm gonna spin up these teams, but this team is going to deliver a set of insights on day. Then we're gonna be continuous improvement teams that that persist over time. So I think this shifting from project teams to persistent teams coupled with mindset coupled with with or structure changed, you know, a lot of times has to be in place for a period of time to get to get the fluency and achieve the fluency that that most organizations need. >>Thanks, Jim, for that well thought out answer. It really goes to show that the transformation process really varies when it comes to organizations, but I think this is a great way to close out today's track. I like to think Jim, Michael, as well as all the experts that you heard earlier today for sharing. There's best practice as to how you all can start transforming your organization's by building a data fluent culture, Um, and really empowering your employees to understand what data means and how to take actions with it. As we wrap up and get ready for the next session, I'd like to leave you all with just a couple of things. Number one if you miss anything or would like to watch any of the other tracks. Don't worry. We have everything available after this event on demand number two. If you want to ask more questions from the experts that you heard earlier today, you have a chance to do so. At the Meet The Experts Roundtable, make sure to attend the one for track four in cultivating a data fluent culture. Now, as we get ready for the product roadmap, go take a sip of water. This is something you do not want to miss. If you love what you heard yesterday, you're gonna like what you hear today. I hear there's some type of Indiana Jones theme to it all, so I won't say anything else, but I'll see you there.

>>welcome back. And thanks for joining us for our second session. External data, your new leading indicators. We'll be hearing from industry leaders as they share best practices and challenges in leveraging external data. This panel will be a true conversation on the part of the possible. All right, let's get to >>it >>today. We're excited to be joined by thought spots. Chief Data Strategy Officer Cindy Housing Deloitte's chief data officer Manteo, the founder and CEO of Eagle Alfa. And it Kilduff and Snowflakes, VP of data marketplace and customer product strategy. Matt Glickman. Cindy. Without further ado, the floor is yours. >>Thank you, Mallory. And I am thrilled to have this brilliant team joining us from around the world. And they really bring each a very unique perspective. So I'm going to start from further away. Emmett, Welcome. Where you joining us from? >>Thanks for having us, Cindy. I'm joining from Dublin, Ireland, >>great. And and tell us a little bit about Eagle Alfa. What do you dio >>from a company's perspective? Think of Eagle Alfa as an aggregator off all the external data sets on a word I'll use a few times. Today is a big advantage we could bring companies is we have a data concierge service. There's so much data we can help identify the right data sets depending on the specific needs of the company. >>Yeah. And so, Emma, you know, people think I was a little I kind of shocked the industry. Going from gardener to a tech startup. Um, you have had a brave journey as well, Going from financial services to starting this company, really pioneering it with I think the most data sets of any of thes is that right? >>Yes, it was. It was a big jump to go from Morgan Stanley. Uh, leave the comforts of that environment Thio, PowerPoint deck and myself raising funding eight years ago s So it was a big jump on. We were very early in our market. It's in the last few years where there's been real momentum and adoption by various types of verticals. The hedge funds were first, maybe then private equity, but corporate sar are following quite quickly from behind. That will be the biggest users, in our view, by by a significant distance. >>Yeah, great. Thank um, it So we're going to go a little farther a field now, but back to the U. S. So, Juan, where you joining us from? >>Hey, Cindy. Thanks for having me. I'm joining you from Houston, Texas. >>Great. Used to be my home. Yeah, probably see Rice University back there. And you have a distinct perspective serving both Deloitte customers externally, but also internally. Can you tell us about that? >>Yeah, absolutely. So I serve as the Lord consultants, chief data officer, and as a professional service firm, I have the responsibility for overseeing our overall data agenda, which includes both the way we use data and insights to run and operate our own business, but also in how we develop data and insights services that we then take to market and how we serve our dealers and clients. >>Great. Thank you, Juan. And last but not least, Matt Glickman. Kind of in my own backyard in New York. Right, Matt? >>Correct. Joining I haven't been into the city and many months, but yes, um, based in New York. >>Okay. Great. And so, Matt, you and Emmett also, you know, brave pioneers in this space, and I'm remembering a conversation you and I shared when you were still a J. P. Morgan, I believe. And you're Goldman Sachs. Sorry. Sorry. Goldman. Can you Can you share that with us? >>Sure. I made the move back in 2015. Um, when everyone thought, you know, my wife, my wife included that I was crazy. I don't know if I would call it Comfortable was emitted, but particularly had been there for a long time on git suffered in some ways. A lot of the pains we're talking about today, given the number of data, says that the amount of of new data sets that are always demand for having run analytics teams at Goldman, seeing the pain and realizing that this pain was not unique to Goldman Sachs, it was being replicated everywhere across the industry, um, in a mind boggling way and and the fortuitous, um, luck to have one of snowflakes. Founders come to pitch snowflake to Goldman a little bit early. Um, they became a customer later, but a little bit early in 2014. And, you know, I realized that this was clearly, you know, the answer from first principles on bond. If I ever was going to leave, this was a problem. I was acutely aware of. And I also was aware of how much the man that was in financial services for a better solution and how the cloud could really solve this problem in particular the ability to not have to move data in and out of these organizations. And this was something that I saw the future of. Thank you, Andi, that this was, you know, sort of the pain that people just expected to pay. Um, this price if you need a data, there was method you had thio. You had to use you either ftp data in and out. You had data that was being, you know, dropped off and, you know, maybe in in in a new ways and cloud buckets or a P i s You have to suck all this data down and reconstruct it. And God forbid the formats change. It was, you know, a nightmare. And then having issues with data, you had a what you were seeing internally. You look nothing like what the data vendors were seeing because they want a completely different system, maybe model completely differently. Um, but this was just the way things were. Everyone had firewalls. Everyone had their own data centers. There was no other way on git was super costly. And you know this. I won't even share the the details of you know, the errors that would occur in the pain that would come from that, Um what I realized it was confirmed. What I saw it snowflake at the time was once everyone moves to run their actual workloads in this in the cloud right where you're now beyond your firewall, you'll have all this scale. But on top of that, you'll be able to point at data from these vendors were not there the traditional data vendors. Or, you know, this new wave of alternative data vendors, for example, like the ones that eagle out for brings together And bring these all these data sets together with your own internal data without moving it. Yeah, this was a fundamental shift of what you know, it's in some ways, it was a side effect of everyone moving to the cloud for costs and scale and elasticity. But as a side effect of that is what we talked about, You know it snowflake summit, you know, yesterday was this notion of a data cloud that would connect data between regions between cloud vendors between customers in a way where you could now reference data. Just like your reference websites today, I don't download CNN dot com. I point at it, and it points me to something else. I'm always seeing the latest version, obviously, and we can, you know, all collaborate on what I'm seeing on that website. That's the same thing that now can happen with data. So And I saw this as what was possible, and I distinctly asked the question, you know, the CEO of the time Is this possible? And not only was it possible it was a fundamental construct that was built into the way that snowflake was delivered. And then, lastly, this is what we learned. And I think this is what you know. M It also has been touting is that it's all great if data is out there and even if you lower that bar of access where data doesn't have to move, how do I know? Right? If I'm back to sitting at Goldman Sachs, how do I know what data is available to me now in this this you know, connected data network eso we released our data marketplace, which was a very different kind of marketplace than these of the past. Where for us, it was really like a global catalog that would elect a consumer data consumer. Noah data was available, but also level the playing field. Now we're now, you know, Eagle, Alfa, or even, you know, a new alternative data vendor build something in their in their basement can now publish that data set so that the world could see and consume and be aligned to, you know, snowflakes, core business, and not where we wouldn't have to be competing or having to take, um, any kind of custody of that data. So adding that catalog to this now ubiquitous access, um really changed the game and, you know, and then now I seem like a genius for making this move. But back then, like I said, we've seen I seem like instant. I was insane. >>Well, given, given that snowflake was the hottest aipo like ever, you were a genius. Uh, doing this, you know, six years in advance. E think we all agree on that, But, you know, a lot of this is still visionary. Um, you know, some of the most leading companies are already doing this. But one What? What is your take our Are you best in class customers still moving the data? Or is this like they're at least thinking about data monetization? What are you seeing from your perspective? >>Yeah, I mean, I did you know, the overall appreciation and understanding of you know, one. I got to get my house in order around my data, um, has something that has been, you know, understood and acted upon. Andi, I do agree that there is a shift now that says, you know, data silos alone aren't necessarily gonna bring me, you know, new and unique insights on dso enriching that with external third party data is absolutely, you know, sort of the the ship that we're seeing our customers undergo. Um, what I find extremely interesting in this space and what some of the most mature clients are doing is, you know, really taking advantage of these data marketplaces. But building data partnerships right there from what mutually exclusive, where there is a win win scenario for for you know, that organization and that could be, you know, retail customers or life science customers like with pandemic, right the way we saw companies that weren't naturally sharing information are now building these data partnership right that are going are going into mutually benefit, you know, all organizations that are sort of part of that value to Andi. I think that's the sort of really important criteria. And how we're seeing our clients that are extremely successful at this is that partnership has benefits on both sides of that equation, right? Both the data provider and then the consumer of that. And there has to be, you know, some way to ensure that both parties are are are learning right, gaining you insights to support, you know, whatever their business organization going on. >>Yeah, great one. So those data partnerships getting across the full value chain of sharing data and analytics Emmett, you work on both sides of the equation here, helping companies. Let's say let's say data providers maybe, like, you know, cast with human mobility monetize that. But then also people that are new to it. Where you seeing the top use cases? Well, >>interestingly, I agree with one of the supply side. One of the interesting trends is we're seeing a lot more data coming from large Corporates. Whether they're listed are private equity backed, as opposed to maybe data startups that are earning money just through data monetization. I think that's a great trend. I think that means a lot of the best. Data said it data is yet to come, um, in terms off the tough economy and how that's changed. I think the category that's had the most momentum and your references is Geo location data. It's that was the category at our conference in December 2000 and 12 that was pipped as the category to watch in 2019. On it didn't become that at all. Um, there were some regulatory concerns for certain types of geo data, but with with covert 19, it's Bean absolutely critical for governments, ministries of finance, central banks, municipalities, Thio crunch that data to understand what's happening in a real time basis. But from a company perspective, it's obviously critical as well. In terms of planning when customers might be back in the High Street on DSO, fourth traditionally consumer transaction data of all the 26 categories in our taxonomy has been the most popular. But Geo is definitely catching up your slide. Talked about being a tough economy. Just one point to contradict that for certain pockets of our clients, e commerce companies are having a field day, obviously, on they are very data driven and tech literate on day are they are really good client base for us because they're incredibly hungry, firm or data to help drive various, uh, decision making. >>Yeah, So fair enough. Some sectors of the economy e commerce, electron, ICS, healthcare are doing great. Others travel, hospitality, Um, super challenging. So I like your quote. The best is yet to come, >>but >>that's data sets is yet to come. And I do think the cloud is enabling that because we could get rid of some of the messy manual data flows that Matt you talked about, but nonetheless, Still, one of the hardest things is the data map. Things combining internal and external >>when >>you might not even have good master data. Common keys on your internal data. So any advice for this? Anyone who wants to take that? >>Sure I can. I can I can start. That's okay. I do think you know, one of the first problems is just a cataloging of the information that's out there. Um, you know, at least within our organization. When I took on this role, we were, you know, a large buyer of third party data. But our organization as a whole didn't necessarily have full visibility into what was being bought and for what purpose. And so having a catalog that helps us internally navigate what data we have and how we're gonna use it was sort of step number one. Um, so I think that's absolutely important. Um, I would say if we could go from having that catalog, you know, created manually to more automated to me, that's sort of the next step in our evolution, because everyone is saying right, the ongoing, uh, you know, creation of new external data sets. It's only going to get richer on DSO. We wanna be able to take advantage of that, you know, at the at the pacing speed, that data is being created. So going from Emanuel catalog to anonymous >>data >>catalog, I think, is a key capability for us. But then you know, to your second point, Cindy is how doe I then connect that to our own internal data to drive greater greater insights and how we run our business or how we serve our customers. Andi, that one you know really is a It's a tricky is a tricky, uh, question because I think it just depends on what data we're looking toe leverage. You know, we have this concept just around. Not not all data is created equal. And when you think about governance and you think about the management of your master data, your internal nomenclature on how you define and run your business, you know that that entire ecosystem begins to get extremely massive and it gets very broad and very deep on DSO for us. You know, government and master data management is absolutely important. But we took a very sort of prioritized approach on which domains do we really need to get right that drive the greatest results for our organization on dso mapping those domains like client data or employee data to these external third party data sources across this catalog was really the the unlocked for us versus trying to create this, you know, massive connection between all the external data that we're, uh, leveraging as well as all of our own internal data eso for us. I think it was very. It was a very tailored, prioritized approach to connecting internal data to external data based on the domains that matter most to our business. >>So if the domains so customer important domain and maybe that's looking at things, um, you know, whether it's social media data or customer transactions, you prioritized first by that, Is that right? >>That's correct. That's correct. >>And so, then, Matt, I'm going to throw it back to you because snowflake is in a unique position. You actually get to see what are the most popular data sets is is that playing out what one described are you seeing that play out? >>I I'd say Watch this space. Like like you said. I mean this. We've you know, I think we start with the data club. We solve that that movement problem, which I think was really the barrier that you tended to not even have a chance to focus on this mapping problem. Um, this notion of concordance, I think this is where I see the big next momentum in this space is going to be a flurry of traditional and new startups who deliver this concordance or knowledge graph as a service where this is no longer a problem that I have to solve internal to my organization. The notion of mastering which is again when everyone has to do in every organization like they used to have to do with moving data into the organization goes away. And this becomes like, I find the best of breed for the different scopes of data that I have. And it's delivered to me as a, you know, as a cloud service that just takes my data. My internal data maps it to these 2nd and 3rd party data sets. Um, all delivered to me, you know, a service. >>Yeah, well, that would be brilliant concordance as a service or or clean clean master data as a service. Um, using augmented data prep would be brilliant. So let's hope we get there. Um, you know, so 2020 has been a wild ride for everyone. If I could ask each of you imagine what is the art of the possible or looking ahead to the next to your and that you are you already mentioned the best is yet to come. Can you want to drill down on that. What what part of the best is yet to come or what is your already two possible? >>Just just a brief comment on mapping. Just this week we published a white paper on mapping, which is available for for anyone on eagle alfa dot com. It's It's a massive challenge. It's very difficult to solve. Just with technology Onda people have tried to solve it and get a certain level of accuracy, but can't get to 100% which which, which, which makes it difficult to solve it. If if if there is a new service coming out against 100% I'm all ears and that there will be a massive step forward for the entire data industry, even if it comes in a few years time, let alone next year, I think going back to the comment on data Cindy. Yes, I think boards of companies are Mawr and Mawr. Viewing data as an asset as opposed to an expense are a cost center on bond. They are looking therefore to get their internal house in order, as one was saying, but also monetize the data they are sitting on lots of companies. They're sitting on potentially valuable data. It's not all valuable on a lot of cases. They think it's worth a lot more than it is being frank. But in some cases there is valuable data on bond. If monetized, it can drop to the bottom line on. So I think that bodes well right across the world. A lot of the best date is yet to come on. I think a lot of firms like Deloitte are very well positioned to help drive that adoption because they are the trusted advisor to a lot of these Corporates. Um, so that's one thing. I think, from a company perspective. It's still we're still at the first base. It's quite frustrating how slow a lot of companies are to move and adopt, and some of them are haven't hired CDO. Some of them don't have their internal house in order. I think that has to change next year. I think if we have this conference at this time next year, I would expect that would hopefully be close to the tipping point for Corporates to use external data. And the Malcolm Gladwell tipping point on the final point I make is I think, that will hopefully start to see multi department use as opposed to silos again. Parliaments and silos, hopefully will be more coordinated on the company's side. Data could be used by marketing by sales by r and D by strategy by finance holds external data. So it really, hopefully will be coordinated by this time next year. >>Yeah, Thank you. So, to your point, there recently was an article to about one of the airlines that their data actually has more value than the company itself now. So I know, I know. We're counting on, you know, integrators trusted advisers like Deloitte to help us get there. Uh, one what? What do you think? And if I can also drill down, you know, financial services was early toe all of this because they needed the early signals. And and we talk about, you know, is is external data now more valuable than internal? Because we need those early signals in just such a different economy. >>Yeah, I think you know, for me, it's it's the seamless integration of all these external data sources and and the signals that organizations need and how to bring those into, you know, the day to day operations of your organization, right? So how do you bring those into, You know, you're planning process. How do you bring that into your sales process on DSO? I think for me success or or where I see the that the use and adoption of this is it's got to get down to that level off of operations for organizations. For this to continue to move at the pace and deliver the value that you know, we're all describing. I think we're going to get there. But I think until organizations truly get down to that level of operations and how they're using this data, it'll sort of seem like a Bolton, right? So for me, I think it's all about Mawr, the seamless integration. And I think to what Matt mentioned just around services that could help connect external data with internal data. I'll take that one step beyond and say, How can we have the data connect itself? Eso I had references Thio, you know, automation and machine learning. Um, there's significant advances in terms of how we're seeing, you know, mapping to occur in a auto generated fashion. I think this specific space and again the connection between external and internal data is a prime example of where we need to disrupt that, you know, sort of traditional data pipeline on. Try to automate that as much as possible. And let's have the data, you know, connect itself because it then sort of supports. You know, the first concept which waas How do we make it more seamless and integrated into, you know, the business processes of the organization's >>Yeah, great ones. So you two are thinking those automated, more intelligent data pipelines will get us there faster. Matt, you already gave us one. Great, Uh, look ahead, Any more to add to >>it, I'll give you I'll give you two more. One is a bit controversial, but I'll throw that you anyway, um, going back to the point that one made about data partnerships What you were saying Cindy about, you know, the value. These companies, you know, tends to be somehow sometimes more about the data they have than the actual service they provide. I predict you're going to see a wave of mergers and acquisitions. Um, that it's solely about locking down access to data as opposed to having data open up. Um to the broader, you know, economy, if I can, whether that be a retailer or, you know, insurance company was thes prime data assets. Um, you know, they could try to monetize that themselves, But if someone could acquire them and get exclusive access that data, I think that's going to be a wave of, um, in a that is gonna be like, Well, we bought this for this amount of money because of their data assets s. So I think that's gonna be a big wave. And it'll be maybe under the guise of data partnerships. But it really be about, you know, get locking down exclusive access to valuable data as opposed to trying toe monetize it itself number one. And then lastly, you know. Now, did you have this kind of ubiquity of data in this interconnected data network? Well, we're starting to see, and I think going to see a big wave of is hyper personalization of applications where instead of having the application have the data itself Have me Matt at Snowflake. Bring my data graph to applications. Right? This decoupling of we always talk about how you get data out of these applications. It's sort of the reverse was saying Now I want to bring all of my data access that I have 1st, 2nd and 3rd party into my application. Instead of having to think about getting all the data out of these applications, I think about it how when you you know, using a workout app in the consumer space, right? I can connect my Spotify or connect my apple music into that app to personalize the experience and bring my music list to that. Imagine if I could do that, you know, in a in a CRM. Imagine I could do that in a risk management. Imagine I could do that in a marketing app where I can bring my entire data graph with me and personalize that experience for, you know, for given what I have. And I think again, you know, partners like thoughts. But I think in a unique position to help enable that capability, you know, for this next wave of of applications that really take advantage of this decoupling of data. But having data flow into the app tied to me as opposed to having the APP have to know about my data ahead of time, >>Yeah, yeah, So that is very forward thinking. So I'll end with a prediction and a best practice. I am predicting that the organizations that really leverage external data, new data sources, not just whether or what have you and modernize those data flows will outperform the organizations that don't. And as a best practice to getting there, I the CDOs that own this have at least visibility into everything they're purchasing can save millions of dollars in duplicate spend. So, Thio, get their three key takeaways. Identify the leading indicators and market signals The data you need Thio. Better identify that. Consolidate those purchases and please explore the data sets the range of data sets data providers that we have on the thought spot. Atlas Marketplace Mallory over to you. >>Wow. Thank you. That was incredible. Thank you. To all of our Panelists for being here and sharing that wisdom. We really appreciate it. For those of you at home, stay close by. Our third session is coming right up and we'll be joined by our partner AWS and get to see how you can leverage the full power of your data cloud complete with the demo. Make sure to tune in to see you >>then

>> Announcer: From around the globe, it's theCUBE. With digital coverage of AWS re:Invent 2020. Special coverage sponsored by AWS global partner network. >> Okay, welcome back everyone to theCUBE virtual coverage of re:Invent 2020 virtual. Normally we're in person, this year because of the pandemic we're doing remote interviews and we've got a great coverage here of the APN, Amazon Partner Network experience. I'm your host John Furrier, we are theCUBE virtual. Got a great guest from Tel Aviv remotely calling in and videoing, Nimrod Vax, who is the chief product officer and co-founder of BigID. This is the beautiful thing about remote, you're in Tel Aviv, I'm in Palo Alto, great to see you. We're not in person but thanks for coming on. >> Thank you. Great to see you as well. >> So you guys have had a lot of success at BigID, I've noticed a lot of awards, startup to watch, company to watch, kind of a good market opportunity data, data at scale, identification, as the web evolves beyond web presence identification, authentication is super important. You guys are called BigID. What's the purpose of the company? Why do you exist? What's the value proposition? >> So first of all, best startup to work at based on Glassdoor worldwide, so that's a big achievement too. So look, four years ago we started BigID when we realized that there is a gap in the market between the new demands from organizations in terms of how to protect their personal and sensitive information that they collect about their customers, their employees. The regulations were becoming more strict but the tools that were out there, to the large extent still are there, were not providing to those requirements and organizations have to deal with some of those challenges in manual processes, right? For example, the right to be forgotten. Organizations need to be able to find and delete a person's data if they want to be deleted. That's based on GDPR and later on even CCPA. And organizations have no way of doing it because the tools that were available could not tell them whose data it is that they found. The tools were very siloed. They were looking at either unstructured data and file shares or windows and so forth, or they were looking at databases, there was nothing for Big Data, there was nothing for cloud business applications. And so we identified that there is a gap here and we addressed it by building BigID basically to address those challenges. >> That's great, great stuff. And I remember four years ago when I was banging on the table and saying, you know regulation can stunt innovation because you had the confluence of massive platform shifts combined with the business pressure from society. That's not stopping and it's continuing today. You seeing it globally, whether it's fake news in journalism, to privacy concerns where modern applications, this is not going away. You guys have a great market opportunity. What is the product? What is smallID? What do you guys got right now? How do customers maintain the success as the ground continues to shift under them as platforms become more prevalent, more tools, more platforms, more everything? >> So, I'll start with BigID. What is BigID? So BigID really helps organizations better manage and protect the data that they own. And it does that by connecting to everything you have around structured databases and unstructured file shares, big data, cloud storage, business applications and then providing very deep insight into that data. Cataloging all the data, so you know what data you have where and classifying it so you know what type of data you have. Plus you're analyzing the data to find similar and duplicate data and then correlating them to an identity. Very strong, very broad solution fit for IT organization. We have some of the largest organizations out there, the biggest retailers, the biggest financial services organizations, manufacturing and et cetera. What we are seeing is that there are, with the adoption of cloud and business success obviously of AWS, that there are a lot of organizations that are not as big, that don't have an IT organization, that have a very well functioning DevOps organization but still have a very big footprint in Amazon and in other kind of cloud services. And they want to get visibility and they want to do it quickly. And the SmallID is really built for that. SmallID is a lightweight version of BigID that is cloud-native built for your AWS environment. And what it means is that you can quickly install it using CloudFormation templates straight from the AWS marketplace. Quickly stand up an environment that can scan, discover your assets in your account automatically and give you immediate visibility into that, your S3 bucket, into your DynamoDB environments, into your EMR clusters, into your Athena databases and immediately building a full catalog of all the data, so you know what files you have where, you know where what tables, what technical metadata, operational metadata, business metadata and also classified data information. So you know where you have sensitive information and you can immediately address that and apply controls to that information. >> So this is data discovery. So the use case is, I'm an Amazon partner, I mean we use theCUBE virtuals on Amazon, but let's just say hypothetically, we're growing like crazy. Got S3 buckets over here secure, encrypted and the rest, all that stuff. Things are happening, we're growing like a weed. Do we just deploy smallIDs and how it works? Is that use cases, SmallID is for AWS and BigID for everything else or? >> You can start small with SmallID, you get the visibility you need, you can leverage the automation of AWS so that you automatically discover those data sources, connect to them and get visibility. And you could grow into BigID using the same deployment inside AWS. You don't have to switch migrate and you use the same container cluster that is running inside your account and automatically scale it up and then connect to other systems or benefit from the more advanced capabilities the BigID can offer such as correlation, by connecting to maybe your Salesforce, CRM system and getting the ability to correlate to your customer data and understand also whose data it is that you're storing. Connecting to your on-premise mainframe, with the same deployment connecting to your Google Drive or office 365. But the point is that with the smallID you can really start quickly, small with a very small team and get that visibility very quickly. >> Nimrod, I want to ask you a question. What is the definition of cloud native data discovery? What does that mean to you? >> So cloud native means that it leverages all the benefits of the cloud. Like it gets all of the automation and visibility that you get in a cloud environment versus any traditional on-prem environment. So one thing is that BigID is installed directly from your marketplace. So you could browse, find its solution on the AWS marketplace and purchase it. It gets deployed using CloudFormation templates very easily and very quickly. It runs on a elastic container service so that once it runs you can automatically scale it up and down to increase the scan and the scale capabilities of the solution. It connects automatically behind the scenes into the security hub of AWS. So you get those alerts, the policy alerts fed into your security hub. It has integration also directly into the native logging capabilities of AWS. So your existing Datadog or whatever you're using for monitoring can plug into it automatically. That's what we mean by cloud native. >> And if you're cloud native you got to be positioned to take advantage of the data and machine learning in particular. Can you expand on the role of machine learning in your solution? Customers are leaning in heavily this year, you're seeing more uptake on machine learning which is basically AI, AI is machine learning, but it's all tied together. ML is big on all the deployments. Can you share your thoughts? >> Yeah, absolutely. So data discovery is a very tough problem and it has been around for 20 years. And the traditional methods of classifying the data or understanding what type of data you have has been, you're looking at the pattern of the data. Typically regular expressions or types of kind of pattern-matching techniques that look at the data. But sometimes in order to know what is personal or what is sensitive it's not enough to look at the pattern of the data. How do you distinguish between a date of birth and any other date. Date of birth is much more sensitive. How do you find country of residency or how do you identify even a first name from the last name? So for that, you need more advanced, more sophisticated capabilities that go beyond just pattern matching. And BigID has a variety of those techniques, we call that discovery-in-depth. What it means is that very similar to security-in-depth where you can not rely on a single security control to protect your environment, you can not rely on a single discovery method to truly classify the data. So yes, we have regular expression, that's the table state basic capability of data classification but if you want to find data that is more contextual like a first name, last name, even a phone number and distinguish between a phone number and just a sequence of numbers, you need more contextual NLP based discovery, name entity recognition. We're using (indistinct) to extract and find data contextually. We also apply deep learning, CNN capable, it's called CNN, which is basically deep learning in order to identify and classify document types. Which is basically being able to distinguish between a resume and a application form. Finding financial records, finding medical records. So RA are advanced NLP classifiers can find that type of data. The more advanced capabilities that go beyond the smallID into BigID also include cluster analysis which is an unsupervised machine learning method of finding duplicate and similar data correlation and other techniques that are more contextual and need to use machine learning for that. >> Yeah, and unsupervised that's a lot harder than supervised. You need to have that ability to get that what you can't see. You got to get the blind spots identified and that's really the key observational data you need. This brings up the kind of operational you heard cluster, I hear governance security you mentioned earlier GDPR, this is an operational impact. Can you talk about how it impacts on specifically on the privacy protection and governance side because certainly I get the clustering side of it, operationally just great. Everyone needs to get that. But now on the business model side, this is where people are spending a lot of time scared and worried actually. What the hell to do? >> One of the things that we realized very early on when we started with BigID is that everybody needs a discovery. You need discovery and we actually started with privacy. You need discovery in route to map your data and apply the privacy controls. You need discovery for security, like we said, right? Find and identify sensitive data and apply controls. And you also need discovery for data enablement. You want to discover the data, you want to enable it, to govern it, to make it accessible to the other parts of your business. So discovery is really a foundation and starting point and that you get there with smallID. How do you operationalize that? So BigID has the concept of an application framework. Think about it like an Apple store for data discovery where you can run applications inside your kind of discovery iPhone in order to run specific (indistinct) use cases. So, how do you operationalize privacy use cases? We have applications for privacy use cases like subject access requests and data rights fulfillment, right? Under the CCPA, you have the right to request your data, what data is being stored about you. BigID can help you find all that data in the catalog that after we scan and find that information we can find any individual data. We have an application also in the privacy space for consent governance right under CCP. And you have the right to opt out. If you opt out, your data cannot be sold, cannot be used. How do you enforce that? How do you make sure that if someone opted out, that person's data is not being pumped into Glue, into some other system for analytics, into Redshift or Snowflake? BigID can identify a specific person's data and make sure that it's not being used for analytics and alert if there is a violation. So that's just an example of how you operationalize this knowledge for privacy. And we have more examples also for data enablement and data management. >> There's so much headroom opportunity to build out new functionality, make it programmable. I really appreciate what you guys are doing, totally needed in the industry. I could just see endless opportunities to make this operationally scalable, more programmable, once you kind of get the foundation out there. So congratulations, Nimrod and the whole team. The question I want to ask you, we're here at re:Invent's virtual, three weeks we're here covering Cube action, check out theCUBE experience zone, the partner experience. What is the difference between BigID and say Amazon's Macy? Let's think about that. So how do you compare and contrast, in Amazon they say we love partnering, but we promote our ecosystem. You guys sure have a similar thing. What's the difference? >> There's a big difference. Yes, there is some overlap because both a smallID and Macy can classify data in S3 buckets. And Macy does a pretty good job at it, right? I'm not arguing about it. But smallID is not only about scanning for sensitive data in S3. It also scans anything else you have in your AWS environment, like DynamoDB, like EMR, like Athena. We're also adding Redshift soon, Glue and other rare data sources as well. And it's not only about identifying and alerting on sensitive data, it's about building full catalog (indistinct) It's about giving you almost like a full registry of your data in AWS, where you can look up any type of data and see where it's found across structured, unstructured big data repositories that you're handling inside your AWS environment. So it's broader than just for security. Apart from the fact that they're used for privacy, I would say the biggest value of it is by building that catalog and making it accessible for data enablement, enabling your data across the board for other use cases, for analytics in Redshift, for Glue, for data integrations, for various other purposes. We have also integration into Kinesis to be able to scan and let you know which topics, use what type of data. So it's really a very, very robust full-blown catalog of the data that across the board that is dynamic. And also like you mentioned, accessible to APIs. Very much like the AWS tradition. >> Yeah, great stuff. I got to ask you a question while you're here. You're the co-founder and again congratulations on your success. Also the chief product officer of BigID, what's your advice to your colleagues and potentially new friends out there that are watching here? And let's take it from the entrepreneurial perspective. I have an application and I start growing and maybe I have funding, maybe I take a more pragmatic approach versus raising billions of dollars. But as you grow the pressure for AppSec reviews, having all the table stakes features, how do you advise developers or entrepreneurs or even business people, small medium-sized enterprises to prepare? Is there a way, is there a playbook to say, rather than looking back saying, oh, I didn't do with all the things I got to go back and retrofit, get BigID. Is there a playbook that you see that will help companies so they don't get killed with AppSec reviews and privacy compliance reviews? Could be a waste of time. What's your thoughts on all this? >> Well, I think that very early on when we started BigID, and that was our perspective is that we knew that we are a security and privacy company. So we had to take that very seriously upfront and be prepared. Security cannot be an afterthought. It's something that needs to be built in. And from day one we have taken all of the steps that were needed in order to make sure that what we're building is robust and secure. And that includes, obviously applying all of the code and CI/CD tools that are available for testing your code, whether it's (indistinct), these type of tools. Applying and providing, penetration testing and working with best in line kind of pen testing companies and white hat hackers that would look at your code. These are kind of the things that, that's what you get funding for, right? >> Yeah. >> And you need to take advantage of that and use them. And then as soon as we got bigger, we also invested in a very, kind of a very strong CSO that comes from the industry that has a lot of expertise and a lot of credibility. We also have kind of CSO group. So, each step of funding we've used extensively also to make RM kind of security poster a lot more robust and invisible. >> Final question for you. When should someone buy BigID? When should they engage? Is it something that people can just download immediately and integrate? Do you have to have, is the go-to-market kind of a new target the VP level or is it the... How does someone know when to buy you and download it and use the software? Take us through the use case of how customers engage with. >> Yeah, so customers directly have those requirements when they start hitting and having to comply with regulations around privacy and security. So very early on, especially organizations that deal with consumer information, get to a point where they need to be accountable for the data that they store about their customers and they want to be able to know their data and provide the privacy controls they need to their consumers. For our BigID product this typically is a kind of a medium size and up company, and with an IT organization. For smallID, this is a good fit for companies that are much smaller, that operate mostly out of their, their IT is basically their DevOps teams. And once they have more than 10, 20 data sources in AWS, that's where they start losing count of the data that they have and they need to get more visibility and be able to control what data is being stored there. Because very quickly you start losing count of data information, even for an organization like BigID, which isn't a bigger organization, right? We have 200 employees. We are at the point where it's hard to keep track and keep control of all the data that is being stored in all of the different data sources, right? In AWS, in Google Drive, in some of our other sources, right? And that's the point where you need to start thinking about having that visibility. >> Yeah, like all growth plan, dream big, start small and get big. And I think that's a nice pathway. So small gets you going and you lead right into the BigID. Great stuff. Final, final question for you while I gatchu here. Why the awards? Someone's like, hey, BigID is this cool company, love the founder, love the team, love the value proposition, makes a lot of sense. Why all the awards? >> Look, I think one of the things that was compelling about BigID from the beginning is that we did things differently. Our whole approach for personal data discovery is unique. And instead of looking at the data, we started by looking at the identities, the people and finally looking at their data, learning how their data looks like and then searching for that information. So that was a very different approach to the traditional approach of data discovery. And we continue to innovate and to look at those problems from a different perspective so we can offer our customers an alternative to what was done in the past. It's not saying that we don't do the basic stuffs. The Reg X is the connectivity that that is needed. But we always took a slightly different approach to diversify, to offer something slightly different and more comprehensive. And I think that was the thing that really attracted us from the beginning with the RSA Innovation Sandbox award that we won in 2018, the Gartner Cool Vendor award that we received. And later on also the other awards. And I think that's the unique aspect of BigID. >> You know you solve big problems than certainly as needed. We saw this early on and again I don't think that the problem is going to go away anytime soon, platforms are emerging, more tools than ever before that converge into platforms and as the logic changes at the top all of that's moving onto the underground. So, congratulations, great insight. >> Thank you very much. >> Thank you. Thank you for coming on theCUBE. Appreciate it Nimrod. Okay, I'm John Furrier. We are theCUBE virtual here for the partner experience APN virtual. Thanks for watching. (gentle music)

>>don't talk mhm, >>Okay, thing is my presentation on coherent nonlinear dynamics and combinatorial optimization. This is going to be a talk to introduce an approach we're taking to the analysis of the performance of coherent using machines. So let me start with a brief introduction to easing optimization. The easing model represents a set of interacting magnetic moments or spins the total energy given by the expression shown at the bottom left of this slide. Here, the signal variables are meditate binary values. The Matrix element J. I. J. Represents the interaction, strength and signed between any pair of spins. I. J and A Chive represents a possible local magnetic field acting on each thing. The easing ground state problem is to find an assignment of binary spin values that achieves the lowest possible value of total energy. And an instance of the easing problem is specified by giving numerical values for the Matrix J in Vector H. Although the easy model originates in physics, we understand the ground state problem to correspond to what would be called quadratic binary optimization in the field of operations research and in fact, in terms of computational complexity theory, it could be established that the easing ground state problem is np complete. Qualitatively speaking, this makes the easing problem a representative sort of hard optimization problem, for which it is expected that the runtime required by any computational algorithm to find exact solutions should, as anatomically scale exponentially with the number of spends and for worst case instances at each end. Of course, there's no reason to believe that the problem instances that actually arrives in practical optimization scenarios are going to be worst case instances. And it's also not generally the case in practical optimization scenarios that we demand absolute optimum solutions. Usually we're more interested in just getting the best solution we can within an affordable cost, where costs may be measured in terms of time, service fees and or energy required for a computation. This focuses great interest on so called heuristic algorithms for the easing problem in other NP complete problems which generally get very good but not guaranteed optimum solutions and run much faster than algorithms that are designed to find absolute Optima. To get some feeling for present day numbers, we can consider the famous traveling salesman problem for which extensive compilations of benchmarking data may be found online. A recent study found that the best known TSP solver required median run times across the Library of Problem instances That scaled is a very steep route exponential for end up to approximately 4500. This gives some indication of the change in runtime scaling for generic as opposed the worst case problem instances. Some of the instances considered in this study were taken from a public library of T SPS derived from real world Veil aside design data. This feels I TSP Library includes instances within ranging from 131 to 744,710 instances from this library with end between 6880 13,584 were first solved just a few years ago in 2017 requiring days of run time and a 48 core to King hurts cluster, while instances with and greater than or equal to 14,233 remain unsolved exactly by any means. Approximate solutions, however, have been found by heuristic methods for all instances in the VLS i TSP library with, for example, a solution within 0.14% of a no lower bound, having been discovered, for instance, with an equal 19,289 requiring approximately two days of run time on a single core of 2.4 gigahertz. Now, if we simple mindedly extrapolate the root exponential scaling from the study up to an equal 4500, we might expect that an exact solver would require something more like a year of run time on the 48 core cluster used for the N equals 13,580 for instance, which shows how much a very small concession on the quality of the solution makes it possible to tackle much larger instances with much lower cost. At the extreme end, the largest TSP ever solved exactly has an equal 85,900. This is an instance derived from 19 eighties VLSI design, and it's required 136 CPU. Years of computation normalized to a single cord, 2.4 gigahertz. But the 24 larger so called world TSP benchmark instance within equals 1,904,711 has been solved approximately within ophthalmology. Gap bounded below 0.474%. Coming back to the general. Practical concerns have applied optimization. We may note that a recent meta study analyzed the performance of no fewer than 37 heuristic algorithms for Max cut and quadratic pioneer optimization problems and found the performance sort and found that different heuristics work best for different problem instances selected from a large scale heterogeneous test bed with some evidence but cryptic structure in terms of what types of problem instances were best solved by any given heuristic. Indeed, their their reasons to believe that these results from Mexico and quadratic binary optimization reflected general principle of performance complementarity among heuristic optimization algorithms in the practice of solving heart optimization problems there. The cerise is a critical pre processing issue of trying to guess which of a number of available good heuristic algorithms should be chosen to tackle a given problem. Instance, assuming that any one of them would incur high costs to run on a large problem, instances incidence, making an astute choice of heuristic is a crucial part of maximizing overall performance. Unfortunately, we still have very little conceptual insight about what makes a specific problem instance, good or bad for any given heuristic optimization algorithm. This has certainly been pinpointed by researchers in the field is a circumstance that must be addressed. So adding this all up, we see that a critical frontier for cutting edge academic research involves both the development of novel heuristic algorithms that deliver better performance, with lower cost on classes of problem instances that are underserved by existing approaches, as well as fundamental research to provide deep conceptual insight into what makes a given problem in, since easy or hard for such algorithms. In fact, these days, as we talk about the end of Moore's law and speculate about a so called second quantum revolution, it's natural to talk not only about novel algorithms for conventional CPUs but also about highly customized special purpose hardware architectures on which we may run entirely unconventional algorithms for combinatorial optimization such as easing problem. So against that backdrop, I'd like to use my remaining time to introduce our work on analysis of coherent using machine architectures and associate ID optimization algorithms. These machines, in general, are a novel class of information processing architectures for solving combinatorial optimization problems by embedding them in the dynamics of analog, physical or cyber physical systems, in contrast to both MAWR traditional engineering approaches that build using machines using conventional electron ICS and more radical proposals that would require large scale quantum entanglement. The emerging paradigm of coherent easing machines leverages coherent nonlinear dynamics in photonic or Opto electronic platforms to enable near term construction of large scale prototypes that leverage post Simoes information dynamics, the general structure of of current CM systems has shown in the figure on the right. The role of the easing spins is played by a train of optical pulses circulating around a fiber optical storage ring. A beam splitter inserted in the ring is used to periodically sample the amplitude of every optical pulse, and the measurement results are continually read into a refugee A, which uses them to compute perturbations to be applied to each pulse by a synchronized optical injections. These perturbations, air engineered to implement the spin, spin coupling and local magnetic field terms of the easing Hamiltonian, corresponding to a linear part of the CME Dynamics, a synchronously pumped parametric amplifier denoted here as PPL and Wave Guide adds a crucial nonlinear component to the CIA and Dynamics as well. In the basic CM algorithm, the pump power starts very low and has gradually increased at low pump powers. The amplitude of the easing spin pulses behaviors continuous, complex variables. Who Israel parts which can be positive or negative, play the role of play the role of soft or perhaps mean field spins once the pump, our crosses the threshold for parametric self oscillation. In the optical fiber ring, however, the attitudes of the easing spin pulses become effectively Qantas ized into binary values while the pump power is being ramped up. The F P J subsystem continuously applies its measurement based feedback. Implementation of the using Hamiltonian terms, the interplay of the linear rised using dynamics implemented by the F P G A and the threshold conversation dynamics provided by the sink pumped Parametric amplifier result in the final state of the optical optical pulse amplitude at the end of the pump ramp that could be read as a binary strain, giving a proposed solution of the easing ground state problem. This method of solving easing problem seems quite different from a conventional algorithm that runs entirely on a digital computer as a crucial aspect of the computation is performed physically by the analog, continuous, coherent, nonlinear dynamics of the optical degrees of freedom. In our efforts to analyze CIA and performance, we have therefore turned to the tools of dynamical systems theory, namely, a study of modifications, the evolution of critical points and apologies of hetero clinic orbits and basins of attraction. We conjecture that such analysis can provide fundamental insight into what makes certain optimization instances hard or easy for coherent using machines and hope that our approach can lead to both improvements of the course, the AM algorithm and a pre processing rubric for rapidly assessing the CME suitability of new instances. Okay, to provide a bit of intuition about how this all works, it may help to consider the threshold dynamics of just one or two optical parametric oscillators in the CME architecture just described. We can think of each of the pulse time slots circulating around the fiber ring, as are presenting an independent Opio. We can think of a single Opio degree of freedom as a single, resonant optical node that experiences linear dissipation, do toe out coupling loss and gain in a pump. Nonlinear crystal has shown in the diagram on the upper left of this slide as the pump power is increased from zero. As in the CME algorithm, the non linear game is initially to low toe overcome linear dissipation, and the Opio field remains in a near vacuum state at a critical threshold. Value gain. Equal participation in the Popeo undergoes a sort of lazing transition, and the study states of the OPIO above this threshold are essentially coherent states. There are actually two possible values of the Opio career in amplitude and any given above threshold pump power which are equal in magnitude but opposite in phase when the OPI across the special diet basically chooses one of the two possible phases randomly, resulting in the generation of a single bit of information. If we consider to uncoupled, Opio has shown in the upper right diagram pumped it exactly the same power at all times. Then, as the pump power has increased through threshold, each Opio will independently choose the phase and thus to random bits are generated for any number of uncoupled. Oppose the threshold power per opio is unchanged from the single Opio case. Now, however, consider a scenario in which the two appeals air, coupled to each other by a mutual injection of their out coupled fields has shown in the diagram on the lower right. One can imagine that depending on the sign of the coupling parameter Alfa, when one Opio is lazing, it will inject a perturbation into the other that may interfere either constructively or destructively, with the feel that it is trying to generate by its own lazing process. As a result, when came easily showed that for Alfa positive, there's an effective ferro magnetic coupling between the two Opio fields and their collective oscillation threshold is lowered from that of the independent Opio case. But on Lee for the two collective oscillation modes in which the two Opio phases are the same for Alfa Negative, the collective oscillation threshold is lowered on Lee for the configurations in which the Opio phases air opposite. So then, looking at how Alfa is related to the J. I. J matrix of the easing spin coupling Hamiltonian, it follows that we could use this simplistic to a p o. C. I am to solve the ground state problem of a fair magnetic or anti ferro magnetic ankles to easing model simply by increasing the pump power from zero and observing what phase relation occurs as the two appeals first start delays. Clearly, we can imagine generalizing this story toe larger, and however the story doesn't stay is clean and simple for all larger problem instances. And to find a more complicated example, we only need to go to n equals four for some choices of J J for n equals, for the story remains simple. Like the n equals two case. The figure on the upper left of this slide shows the energy of various critical points for a non frustrated and equals, for instance, in which the first bifurcated critical point that is the one that I forget to the lowest pump value a. Uh, this first bifurcated critical point flows as symptomatically into the lowest energy easing solution and the figure on the upper right. However, the first bifurcated critical point flows to a very good but sub optimal minimum at large pump power. The global minimum is actually given by a distinct critical critical point that first appears at a higher pump power and is not automatically connected to the origin. The basic C am algorithm is thus not able to find this global minimum. Such non ideal behaviors needs to become more confident. Larger end for the n equals 20 instance, showing the lower plots where the lower right plot is just a zoom into a region of the lower left lot. It can be seen that the global minimum corresponds to a critical point that first appears out of pump parameter, a around 0.16 at some distance from the idiomatic trajectory of the origin. That's curious to note that in both of these small and examples, however, the critical point corresponding to the global minimum appears relatively close to the idiomatic projector of the origin as compared to the most of the other local minima that appear. We're currently working to characterize the face portrait topology between the global minimum in the antibiotic trajectory of the origin, taking clues as to how the basic C am algorithm could be generalized to search for non idiomatic trajectories that jump to the global minimum during the pump ramp. Of course, n equals 20 is still too small to be of interest for practical optimization applications. But the advantage of beginning with the study of small instances is that we're able reliably to determine their global minima and to see how they relate to the 80 about trajectory of the origin in the basic C am algorithm. In the smaller and limit, we can also analyze fully quantum mechanical models of Syrian dynamics. But that's a topic for future talks. Um, existing large scale prototypes are pushing into the range of in equals 10 to the 4 10 to 5 to six. So our ultimate objective in theoretical analysis really has to be to try to say something about CIA and dynamics and regime of much larger in our initial approach to characterizing CIA and behavior in the large in regime relies on the use of random matrix theory, and this connects to prior research on spin classes, SK models and the tap equations etcetera. At present, we're focusing on statistical characterization of the CIA ingredient descent landscape, including the evolution of critical points in their Eigen value spectra. As the pump power is gradually increased. We're investigating, for example, whether there could be some way to exploit differences in the relative stability of the global minimum versus other local minima. We're also working to understand the deleterious or potentially beneficial effects of non ideologies, such as a symmetry in the implemented these and couplings. Looking one step ahead, we plan to move next in the direction of considering more realistic classes of problem instances such as quadratic, binary optimization with constraints. Eso In closing, I should acknowledge people who did the hard work on these things that I've shown eso. My group, including graduate students Ed winning, Daniel Wennberg, Tatsuya Nagamoto and Atsushi Yamamura, have been working in close collaboration with Syria Ganguly, Marty Fair and Amir Safarini Nini, all of us within the Department of Applied Physics at Stanford University. On also in collaboration with the Oshima Moto over at NTT 55 research labs, Onda should acknowledge funding support from the NSF by the Coherent Easing Machines Expedition in computing, also from NTT five research labs, Army Research Office and Exxon Mobil. Uh, that's it. Thanks very much. >>Mhm e >>t research and the Oshie for putting together this program and also the opportunity to speak here. My name is Al Gore ism or Andy and I'm from Caltech, and today I'm going to tell you about the work that we have been doing on networks off optical parametric oscillators and how we have been using them for icing machines and how we're pushing them toward Cornum photonics to acknowledge my team at Caltech, which is now eight graduate students and five researcher and postdocs as well as collaborators from all over the world, including entity research and also the funding from different places, including entity. So this talk is primarily about networks of resonate er's, and these networks are everywhere from nature. For instance, the brain, which is a network of oscillators all the way to optics and photonics and some of the biggest examples or metal materials, which is an array of small resonate er's. And we're recently the field of technological photonics, which is trying thio implement a lot of the technological behaviors of models in the condensed matter, physics in photonics and if you want to extend it even further, some of the implementations off quantum computing are technically networks of quantum oscillators. So we started thinking about these things in the context of icing machines, which is based on the icing problem, which is based on the icing model, which is the simple summation over the spins and spins can be their upward down and the couplings is given by the JJ. And the icing problem is, if you know J I J. What is the spin configuration that gives you the ground state? And this problem is shown to be an MP high problem. So it's computational e important because it's a representative of the MP problems on NPR. Problems are important because first, their heart and standard computers if you use a brute force algorithm and they're everywhere on the application side. That's why there is this demand for making a machine that can target these problems, and hopefully it can provide some meaningful computational benefit compared to the standard digital computers. So I've been building these icing machines based on this building block, which is a degenerate optical parametric. Oscillator on what it is is resonator with non linearity in it, and we pump these resonate er's and we generate the signal at half the frequency of the pump. One vote on a pump splits into two identical photons of signal, and they have some very interesting phase of frequency locking behaviors. And if you look at the phase locking behavior, you realize that you can actually have two possible phase states as the escalation result of these Opio which are off by pie, and that's one of the important characteristics of them. So I want to emphasize a little more on that and I have this mechanical analogy which are basically two simple pendulum. But there are parametric oscillators because I'm going to modulate the parameter of them in this video, which is the length of the string on by that modulation, which is that will make a pump. I'm gonna make a muscular. That'll make a signal which is half the frequency of the pump. And I have two of them to show you that they can acquire these face states so they're still facing frequency lock to the pump. But it can also lead in either the zero pie face states on. The idea is to use this binary phase to represent the binary icing spin. So each opio is going to represent spin, which can be either is your pie or up or down. And to implement the network of these resonate er's, we use the time off blood scheme, and the idea is that we put impulses in the cavity. These pulses air separated by the repetition period that you put in or t r. And you can think about these pulses in one resonator, xaz and temporarily separated synthetic resonate Er's if you want a couple of these resonator is to each other, and now you can introduce these delays, each of which is a multiple of TR. If you look at the shortest delay it couples resonator wanted to 2 to 3 and so on. If you look at the second delay, which is two times a rotation period, the couple's 123 and so on. And if you have and minus one delay lines, then you can have any potential couplings among these synthetic resonate er's. And if I can introduce these modulators in those delay lines so that I can strength, I can control the strength and the phase of these couplings at the right time. Then I can have a program will all toe all connected network in this time off like scheme, and the whole physical size of the system scales linearly with the number of pulses. So the idea of opium based icing machine is didn't having these o pos, each of them can be either zero pie and I can arbitrarily connect them to each other. And then I start with programming this machine to a given icing problem by just setting the couplings and setting the controllers in each of those delight lines. So now I have a network which represents an icing problem. Then the icing problem maps to finding the face state that satisfy maximum number of coupling constraints. And the way it happens is that the icing Hamiltonian maps to the linear loss of the network. And if I start adding gain by just putting pump into the network, then the OPI ohs are expected to oscillate in the lowest, lowest lost state. And, uh and we have been doing these in the past, uh, six or seven years and I'm just going to quickly show you the transition, especially what happened in the first implementation, which was using a free space optical system and then the guided wave implementation in 2016 and the measurement feedback idea which led to increasing the size and doing actual computation with these machines. So I just want to make this distinction here that, um, the first implementation was an all optical interaction. We also had an unequal 16 implementation. And then we transition to this measurement feedback idea, which I'll tell you quickly what it iss on. There's still a lot of ongoing work, especially on the entity side, to make larger machines using the measurement feedback. But I'm gonna mostly focused on the all optical networks and how we're using all optical networks to go beyond simulation of icing Hamiltonian both in the linear and non linear side and also how we're working on miniaturization of these Opio networks. So the first experiment, which was the four opium machine, it was a free space implementation and this is the actual picture off the machine and we implemented a small and it calls for Mexico problem on the machine. So one problem for one experiment and we ran the machine 1000 times, we looked at the state and we always saw it oscillate in one of these, um, ground states of the icing laboratoria. So then the measurement feedback idea was to replace those couplings and the controller with the simulator. So we basically simulated all those coherent interactions on on FB g. A. And we replicated the coherent pulse with respect to all those measurements. And then we injected it back into the cavity and on the near to you still remain. So it still is a non. They're dynamical system, but the linear side is all simulated. So there are lots of questions about if this system is preserving important information or not, or if it's gonna behave better. Computational wars. And that's still ah, lot of ongoing studies. But nevertheless, the reason that this implementation was very interesting is that you don't need the end minus one delight lines so you can just use one. Then you can implement a large machine, and then you can run several thousands of problems in the machine, and then you can compare the performance from the computational perspective Looks so I'm gonna split this idea of opium based icing machine into two parts. One is the linear part, which is if you take out the non linearity out of the resonator and just think about the connections. You can think about this as a simple matrix multiplication scheme. And that's basically what gives you the icing Hambletonian modeling. So the optical laws of this network corresponds to the icing Hamiltonian. And if I just want to show you the example of the n equals for experiment on all those face states and the history Graham that we saw, you can actually calculate the laws of each of those states because all those interferences in the beam splitters and the delay lines are going to give you a different losses. And then you will see that the ground states corresponds to the lowest laws of the actual optical network. If you add the non linearity, the simple way of thinking about what the non linearity does is that it provides to gain, and then you start bringing up the gain so that it hits the loss. Then you go through the game saturation or the threshold which is going to give you this phase bifurcation. So you go either to zero the pie face state. And the expectation is that Theis, the network oscillates in the lowest possible state, the lowest possible loss state. There are some challenges associated with this intensity Durban face transition, which I'm going to briefly talk about. I'm also going to tell you about other types of non aerodynamics that we're looking at on the non air side of these networks. So if you just think about the linear network, we're actually interested in looking at some technological behaviors in these networks. And the difference between looking at the technological behaviors and the icing uh, machine is that now, First of all, we're looking at the type of Hamilton Ian's that are a little different than the icing Hamilton. And one of the biggest difference is is that most of these technological Hamilton Ian's that require breaking the time reversal symmetry, meaning that you go from one spin to in the one side to another side and you get one phase. And if you go back where you get a different phase, and the other thing is that we're not just interested in finding the ground state, we're actually now interesting and looking at all sorts of states and looking at the dynamics and the behaviors of all these states in the network. So we started with the simplest implementation, of course, which is a one d chain of thes resonate, er's, which corresponds to a so called ssh model. In the technological work, we get the similar energy to los mapping and now we can actually look at the band structure on. This is an actual measurement that we get with this associate model and you see how it reasonably how How? Well, it actually follows the prediction and the theory. One of the interesting things about the time multiplexing implementation is that now you have the flexibility of changing the network as you are running the machine. And that's something unique about this time multiplex implementation so that we can actually look at the dynamics. And one example that we have looked at is we can actually go through the transition off going from top A logical to the to the standard nontrivial. I'm sorry to the trivial behavior of the network. You can then look at the edge states and you can also see the trivial and states and the technological at states actually showing up in this network. We have just recently implement on a two D, uh, network with Harper Hofstadter model and when you don't have the results here. But we're one of the other important characteristic of time multiplexing is that you can go to higher and higher dimensions and keeping that flexibility and dynamics, and we can also think about adding non linearity both in a classical and quantum regimes, which is going to give us a lot of exotic, no classical and quantum, non innate behaviors in these networks. Yeah, So I told you about the linear side. Mostly let me just switch gears and talk about the nonlinear side of the network. And the biggest thing that I talked about so far in the icing machine is this face transition that threshold. So the low threshold we have squeezed state in these. Oh, pios, if you increase the pump, we go through this intensity driven phase transition and then we got the face stays above threshold. And this is basically the mechanism off the computation in these O pos, which is through this phase transition below to above threshold. So one of the characteristics of this phase transition is that below threshold, you expect to see quantum states above threshold. You expect to see more classical states or coherent states, and that's basically corresponding to the intensity off the driving pump. So it's really hard to imagine that it can go above threshold. Or you can have this friends transition happen in the all in the quantum regime. And there are also some challenges associated with the intensity homogeneity off the network, which, for example, is if one opioid starts oscillating and then its intensity goes really high. Then it's going to ruin this collective decision making off the network because of the intensity driven face transition nature. So So the question is, can we look at other phase transitions? Can we utilize them for both computing? And also can we bring them to the quantum regime on? I'm going to specifically talk about the face transition in the spectral domain, which is the transition from the so called degenerate regime, which is what I mostly talked about to the non degenerate regime, which happens by just tuning the phase of the cavity. And what is interesting is that this phase transition corresponds to a distinct phase noise behavior. So in the degenerate regime, which we call it the order state, you're gonna have the phase being locked to the phase of the pump. As I talked about non degenerate regime. However, the phase is the phase is mostly dominated by the quantum diffusion. Off the off the phase, which is limited by the so called shallow towns limit, and you can see that transition from the general to non degenerate, which also has distinct symmetry differences. And this transition corresponds to a symmetry breaking in the non degenerate case. The signal can acquire any of those phases on the circle, so it has a you one symmetry. Okay, and if you go to the degenerate case, then that symmetry is broken and you only have zero pie face days I will look at. So now the question is can utilize this phase transition, which is a face driven phase transition, and can we use it for similar computational scheme? So that's one of the questions that were also thinking about. And it's not just this face transition is not just important for computing. It's also interesting from the sensing potentials and this face transition, you can easily bring it below threshold and just operated in the quantum regime. Either Gaussian or non Gaussian. If you make a network of Opio is now, we can see all sorts off more complicated and more interesting phase transitions in the spectral domain. One of them is the first order phase transition, which you get by just coupling to Opio, and that's a very abrupt face transition and compared to the to the single Opio phase transition. And if you do the couplings right, you can actually get a lot of non her mission dynamics and exceptional points, which are actually very interesting to explore both in the classical and quantum regime. And I should also mention that you can think about the cup links to be also nonlinear couplings. And that's another behavior that you can see, especially in the nonlinear in the non degenerate regime. So with that, I basically told you about these Opio networks, how we can think about the linear scheme and the linear behaviors and how we can think about the rich, nonlinear dynamics and non linear behaviors both in the classical and quantum regime. I want to switch gear and tell you a little bit about the miniaturization of these Opio networks. And of course, the motivation is if you look at the electron ICS and what we had 60 or 70 years ago with vacuum tube and how we transition from relatively small scale computers in the order of thousands of nonlinear elements to billions of non elements where we are now with the optics is probably very similar to 70 years ago, which is a table talk implementation. And the question is, how can we utilize nano photonics? I'm gonna just briefly show you the two directions on that which we're working on. One is based on lithium Diabate, and the other is based on even a smaller resonate er's could you? So the work on Nana Photonic lithium naive. It was started in collaboration with Harvard Marko Loncar, and also might affair at Stanford. And, uh, we could show that you can do the periodic polling in the phenomenon of it and get all sorts of very highly nonlinear processes happening in this net. Photonic periodically polls if, um Diabate. And now we're working on building. Opio was based on that kind of photonic the film Diabate. And these air some some examples of the devices that we have been building in the past few months, which I'm not gonna tell you more about. But the O. P. O. S. And the Opio Networks are in the works. And that's not the only way of making large networks. Um, but also I want to point out that The reason that these Nana photonic goblins are actually exciting is not just because you can make a large networks and it can make him compact in a in a small footprint. They also provide some opportunities in terms of the operation regime. On one of them is about making cat states and Opio, which is, can we have the quantum superposition of the zero pie states that I talked about and the Net a photonic within? I've It provides some opportunities to actually get closer to that regime because of the spatial temporal confinement that you can get in these wave guides. So we're doing some theory on that. We're confident that the type of non linearity two losses that it can get with these platforms are actually much higher than what you can get with other platform their existing platforms and to go even smaller. We have been asking the question off. What is the smallest possible Opio that you can make? Then you can think about really wavelength scale type, resonate er's and adding the chi to non linearity and see how and when you can get the Opio to operate. And recently, in collaboration with us see, we have been actually USC and Creole. We have demonstrated that you can use nano lasers and get some spin Hamilton and implementations on those networks. So if you can build the a P. O s, we know that there is a path for implementing Opio Networks on on such a nano scale. So we have looked at these calculations and we try to estimate the threshold of a pos. Let's say for me resonator and it turns out that it can actually be even lower than the type of bulk Pip Llano Pos that we have been building in the past 50 years or so. So we're working on the experiments and we're hoping that we can actually make even larger and larger scale Opio networks. So let me summarize the talk I told you about the opium networks and our work that has been going on on icing machines and the measurement feedback. And I told you about the ongoing work on the all optical implementations both on the linear side and also on the nonlinear behaviors. And I also told you a little bit about the efforts on miniaturization and going to the to the Nano scale. So with that, I would like Thio >>three from the University of Tokyo. Before I thought that would like to thank you showing all the stuff of entity for the invitation and the organization of this online meeting and also would like to say that it has been very exciting to see the growth of this new film lab. And I'm happy to share with you today of some of the recent works that have been done either by me or by character of Hong Kong. Honest Group indicates the title of my talk is a neuro more fic in silica simulator for the communities in machine. And here is the outline I would like to make the case that the simulation in digital Tektronix of the CME can be useful for the better understanding or improving its function principles by new job introducing some ideas from neural networks. This is what I will discuss in the first part and then it will show some proof of concept of the game and performance that can be obtained using dissimulation in the second part and the protection of the performance that can be achieved using a very large chaos simulator in the third part and finally talk about future plans. So first, let me start by comparing recently proposed izing machines using this table there is elected from recent natural tronics paper from the village Park hard people, and this comparison shows that there's always a trade off between energy efficiency, speed and scalability that depends on the physical implementation. So in red, here are the limitation of each of the servers hardware on, interestingly, the F p G, a based systems such as a producer, digital, another uh Toshiba beautification machine or a recently proposed restricted Bozeman machine, FPD A by a group in Berkeley. They offer a good compromise between speed and scalability. And this is why, despite the unique advantage that some of these older hardware have trust as the currency proposition in Fox, CBS or the energy efficiency off memory Sisters uh P. J. O are still an attractive platform for building large organizing machines in the near future. The reason for the good performance of Refugee A is not so much that they operate at the high frequency. No, there are particular in use, efficient, but rather that the physical wiring off its elements can be reconfigured in a way that limits the funding human bottleneck, larger, funny and phenols and the long propagation video information within the system. In this respect, the LPGA is They are interesting from the perspective off the physics off complex systems, but then the physics of the actions on the photos. So to put the performance of these various hardware and perspective, we can look at the competition of bringing the brain the brain complete, using billions of neurons using only 20 watts of power and operates. It's a very theoretically slow, if we can see and so this impressive characteristic, they motivate us to try to investigate. What kind of new inspired principles be useful for designing better izing machines? The idea of this research project in the future collaboration it's to temporary alleviates the limitations that are intrinsic to the realization of an optical cortex in machine shown in the top panel here. By designing a large care simulator in silicone in the bottom here that can be used for digesting the better organization principles of the CIA and this talk, I will talk about three neuro inspired principles that are the symmetry of connections, neural dynamics orphan chaotic because of symmetry, is interconnectivity the infrastructure? No. Next talks are not composed of the reputation of always the same types of non environments of the neurons, but there is a local structure that is repeated. So here's the schematic of the micro column in the cortex. And lastly, the Iraqi co organization of connectivity connectivity is organizing a tree structure in the brain. So here you see a representation of the Iraqi and organization of the monkey cerebral cortex. So how can these principles we used to improve the performance of the icing machines? And it's in sequence stimulation. So, first about the two of principles of the estimate Trian Rico structure. We know that the classical approximation of the car testing machine, which is the ground toe, the rate based on your networks. So in the case of the icing machines, uh, the okay, Scott approximation can be obtained using the trump active in your position, for example, so the times of both of the system they are, they can be described by the following ordinary differential equations on in which, in case of see, I am the X, I represent the in phase component of one GOP Oh, Theo f represents the monitor optical parts, the district optical Parametric amplification and some of the good I JoJo extra represent the coupling, which is done in the case of the measure of feedback coupling cm using oh, more than detection and refugee A and then injection off the cooking time and eso this dynamics in both cases of CNN in your networks, they can be written as the grand set of a potential function V, and this written here, and this potential functionally includes the rising Maccagnan. So this is why it's natural to use this type of, uh, dynamics to solve the icing problem in which the Omega I J or the eyes in coping and the H is the extension of the icing and attorney in India and expect so. Not that this potential function can only be defined if the Omega I j. R. A. Symmetric. So the well known problem of this approach is that this potential function V that we obtain is very non convicts at low temperature, and also one strategy is to gradually deformed this landscape, using so many in process. But there is no theorem. Unfortunately, that granted conventions to the global minimum of There's even Tony and using this approach. And so this is why we propose, uh, to introduce a macro structures of the system where one analog spin or one D O. P. O is replaced by a pair off one another spin and one error, according viable. And the addition of this chemical structure introduces a symmetry in the system, which in terms induces chaotic dynamics, a chaotic search rather than a learning process for searching for the ground state of the icing. Every 20 within this massacre structure the role of the er variable eyes to control the amplitude off the analog spins toe force. The amplitude of the expense toe become equal to certain target amplitude a uh and, uh, and this is done by modulating the strength off the icing complaints or see the the error variable E I multiply the icing complaint here in the dynamics off air d o p. O. On then the dynamics. The whole dynamics described by this coupled equations because the e I do not necessarily take away the same value for the different. I thesis introduces a symmetry in the system, which in turn creates security dynamics, which I'm sure here for solving certain current size off, um, escape problem, Uh, in which the X I are shown here and the i r from here and the value of the icing energy showing the bottom plots. You see this Celtics search that visit various local minima of the as Newtonian and eventually finds the global minimum? Um, it can be shown that this modulation off the target opportunity can be used to destabilize all the local minima off the icing evertonians so that we're gonna do not get stuck in any of them. On more over the other types of attractors I can eventually appear, such as limits I contractors, Okot contractors. They can also be destabilized using the motivation of the target and Batuta. And so we have proposed in the past two different moderation of the target amateur. The first one is a modulation that ensure the uh 100 reproduction rate of the system to become positive on this forbids the creation off any nontrivial tractors. And but in this work, I will talk about another moderation or arrested moderation which is given here. That works, uh, as well as this first uh, moderation, but is easy to be implemented on refugee. So this couple of the question that represent becoming the stimulation of the cortex in machine with some error correction they can be implemented especially efficiently on an F B. G. And here I show the time that it takes to simulate three system and also in red. You see, at the time that it takes to simulate the X I term the EI term, the dot product and the rising Hamiltonian for a system with 500 spins and Iraq Spain's equivalent to 500 g. O. P. S. So >>in >>f b d a. The nonlinear dynamics which, according to the digital optical Parametric amplification that the Opa off the CME can be computed in only 13 clock cycles at 300 yards. So which corresponds to about 0.1 microseconds. And this is Toby, uh, compared to what can be achieved in the measurements back O C. M. In which, if we want to get 500 timer chip Xia Pios with the one she got repetition rate through the obstacle nine narrative. Uh, then way would require 0.5 microseconds toe do this so the submission in F B J can be at least as fast as ah one g repression. Uh, replicate pulsed laser CIA Um, then the DOT product that appears in this differential equation can be completed in 43 clock cycles. That's to say, one microseconds at 15 years. So I pieced for pouring sizes that are larger than 500 speeds. The dot product becomes clearly the bottleneck, and this can be seen by looking at the the skating off the time the numbers of clock cycles a text to compute either the non in your optical parts or the dog products, respect to the problem size. And And if we had infinite amount of resources and PGA to simulate the dynamics, then the non illogical post can could be done in the old one. On the mattress Vector product could be done in the low carrot off, located off scales as a look at it off and and while the guide off end. Because computing the dot product involves assuming all the terms in the product, which is done by a nephew, GE by another tree, which heights scarce logarithmic any with the size of the system. But This is in the case if we had an infinite amount of resources on the LPGA food, but for dealing for larger problems off more than 100 spins. Usually we need to decompose the metrics into ah, smaller blocks with the block side that are not you here. And then the scaling becomes funny, non inner parts linear in the end, over you and for the products in the end of EU square eso typically for low NF pdf cheap PGA you the block size off this matrix is typically about 100. So clearly way want to make you as large as possible in order to maintain this scanning in a log event for the numbers of clock cycles needed to compute the product rather than this and square that occurs if we decompose the metrics into smaller blocks. But the difficulty in, uh, having this larger blocks eyes that having another tree very large Haider tree introduces a large finding and finance and long distance start a path within the refugee. So the solution to get higher performance for a simulator of the contest in machine eyes to get rid of this bottleneck for the dot product by increasing the size of this at the tree. And this can be done by organizing your critique the electrical components within the LPGA in order which is shown here in this, uh, right panel here in order to minimize the finding finance of the system and to minimize the long distance that a path in the in the fpt So I'm not going to the details of how this is implemented LPGA. But just to give you a idea off why the Iraqi Yahiko organization off the system becomes the extremely important toe get good performance for similar organizing machine. So instead of instead of getting into the details of the mpg implementation, I would like to give some few benchmark results off this simulator, uh, off the that that was used as a proof of concept for this idea which is can be found in this archive paper here and here. I should results for solving escape problems. Free connected person, randomly person minus one spring last problems and we sure, as we use as a metric the numbers of the mattress Victor products since it's the bottleneck of the computation, uh, to get the optimal solution of this escape problem with the Nina successful BT against the problem size here and and in red here, this propose FDJ implementation and in ah blue is the numbers of retrospective product that are necessary for the C. I am without error correction to solve this escape programs and in green here for noisy means in an evening which is, uh, behavior with similar to the Cartesian mission. Uh, and so clearly you see that the scaring off the numbers of matrix vector product necessary to solve this problem scales with a better exponents than this other approaches. So So So that's interesting feature of the system and next we can see what is the real time to solution to solve this SK instances eso in the last six years, the time institution in seconds to find a grand state of risk. Instances remain answers probability for different state of the art hardware. So in red is the F B g. A presentation proposing this paper and then the other curve represent Ah, brick a local search in in orange and silver lining in purple, for example. And so you see that the scaring off this purpose simulator is is rather good, and that for larger plant sizes we can get orders of magnitude faster than the state of the art approaches. Moreover, the relatively good scanning off the time to search in respect to problem size uh, they indicate that the FPD implementation would be faster than risk. Other recently proposed izing machine, such as the hope you know, natural complimented on memories distance that is very fast for small problem size in blue here, which is very fast for small problem size. But which scanning is not good on the same thing for the restricted Bosman machine. Implementing a PGA proposed by some group in Broken Recently Again, which is very fast for small parliament sizes but which canning is bad so that a dis worse than the proposed approach so that we can expect that for programs size is larger than 1000 spins. The proposed, of course, would be the faster one. Let me jump toe this other slide and another confirmation that the scheme scales well that you can find the maximum cut values off benchmark sets. The G sets better candidates that have been previously found by any other algorithms, so they are the best known could values to best of our knowledge. And, um or so which is shown in this paper table here in particular, the instances, uh, 14 and 15 of this G set can be We can find better converse than previously known, and we can find this can vary is 100 times faster than the state of the art algorithm and CP to do this which is a very common Kasich. It s not that getting this a good result on the G sets, they do not require ah, particular hard tuning of the parameters. So the tuning issuing here is very simple. It it just depends on the degree off connectivity within each graph. And so this good results on the set indicate that the proposed approach would be a good not only at solving escape problems in this problems, but all the types off graph sizing problems on Mexican province in communities. So given that the performance off the design depends on the height of this other tree, we can try to maximize the height of this other tree on a large F p g a onda and carefully routing the components within the P G A and and we can draw some projections of what type of performance we can achieve in the near future based on the, uh, implementation that we are currently working. So here you see projection for the time to solution way, then next property for solving this escape programs respect to the prime assize. And here, compared to different with such publicizing machines, particularly the digital. And, you know, 42 is shown in the green here, the green line without that's and, uh and we should two different, uh, hypothesis for this productions either that the time to solution scales as exponential off n or that the time of social skills as expression of square root off. So it seems, according to the data, that time solution scares more as an expression of square root of and also we can be sure on this and this production show that we probably can solve prime escape problem of science 2000 spins, uh, to find the rial ground state of this problem with 99 success ability in about 10 seconds, which is much faster than all the other proposed approaches. So one of the future plans for this current is in machine simulator. So the first thing is that we would like to make dissimulation closer to the rial, uh, GOP oh, optical system in particular for a first step to get closer to the system of a measurement back. See, I am. And to do this what is, uh, simulate Herbal on the p a is this quantum, uh, condoms Goshen model that is proposed described in this paper and proposed by people in the in the Entity group. And so the idea of this model is that instead of having the very simple or these and have shown previously, it includes paired all these that take into account on me the mean off the awesome leverage off the, uh, European face component, but also their violence s so that we can take into account more quantum effects off the g o p. O, such as the squeezing. And then we plan toe, make the simulator open access for the members to run their instances on the system. There will be a first version in September that will be just based on the simple common line access for the simulator and in which will have just a classic or approximation of the system. We don't know Sturm, binary weights and museum in term, but then will propose a second version that would extend the current arising machine to Iraq off F p g. A, in which we will add the more refined models truncated, ignoring the bottom Goshen model they just talked about on the support in which he valued waits for the rising problems and support the cement. So we will announce later when this is available and and far right is working >>hard comes from Universal down today in physics department, and I'd like to thank the organizers for their kind invitation to participate in this very interesting and promising workshop. Also like to say that I look forward to collaborations with with a file lab and Yoshi and collaborators on the topics of this world. So today I'll briefly talk about our attempt to understand the fundamental limits off another continues time computing, at least from the point off you off bullion satisfy ability, problem solving, using ordinary differential equations. But I think the issues that we raise, um, during this occasion actually apply to other other approaches on a log approaches as well and into other problems as well. I think everyone here knows what Dorien satisfy ability. Problems are, um, you have boolean variables. You have em clauses. Each of disjunction of collaterals literally is a variable, or it's, uh, negation. And the goal is to find an assignment to the variable, such that order clauses are true. This is a decision type problem from the MP class, which means you can checking polynomial time for satisfy ability off any assignment. And the three set is empty, complete with K three a larger, which means an efficient trees. That's over, uh, implies an efficient source for all the problems in the empty class, because all the problems in the empty class can be reduced in Polian on real time to reset. As a matter of fact, you can reduce the NP complete problems into each other. You can go from three set to set backing or two maximum dependent set, which is a set packing in graph theoretic notions or terms toe the icing graphs. A problem decision version. This is useful, and you're comparing different approaches, working on different kinds of problems when not all the closest can be satisfied. You're looking at the accusation version offset, uh called Max Set. And the goal here is to find assignment that satisfies the maximum number of clauses. And this is from the NPR class. In terms of applications. If we had inefficient sets over or np complete problems over, it was literally, positively influenced. Thousands off problems and applications in industry and and science. I'm not going to read this, but this this, of course, gives a strong motivation toe work on this kind of problems. Now our approach to set solving involves embedding the problem in a continuous space, and you use all the east to do that. So instead of working zeros and ones, we work with minus one across once, and we allow the corresponding variables toe change continuously between the two bounds. We formulate the problem with the help of a close metrics. If if a if a close, uh, does not contain a variable or its negation. The corresponding matrix element is zero. If it contains the variable in positive, for which one contains the variable in a gated for Mitt's negative one, and then we use this to formulate this products caused quote, close violation functions one for every clause, Uh, which really, continuously between zero and one. And they're zero if and only if the clause itself is true. Uh, then we form the define in order to define a dynamic such dynamics in this and dimensional hyper cube where the search happens and if they exist, solutions. They're sitting in some of the corners of this hyper cube. So we define this, uh, energy potential or landscape function shown here in a way that this is zero if and only if all the clauses all the kmc zero or the clauses off satisfied keeping these auxiliary variables a EMS always positive. And therefore, what you do here is a dynamics that is a essentially ingredient descend on this potential energy landscape. If you were to keep all the M's constant that it would get stuck in some local minimum. However, what we do here is we couple it with the dynamics we cooperated the clothes violation functions as shown here. And if he didn't have this am here just just the chaos. For example, you have essentially what case you have positive feedback. You have increasing variable. Uh, but in that case, you still get stuck would still behave will still find. So she is better than the constant version but still would get stuck only when you put here this a m which makes the dynamics in in this variable exponential like uh, only then it keeps searching until he finds a solution on deer is a reason for that. I'm not going toe talk about here, but essentially boils down toe performing a Grady and descend on a globally time barren landscape. And this is what works. Now I'm gonna talk about good or bad and maybe the ugly. Uh, this is, uh, this is What's good is that it's a hyperbolic dynamical system, which means that if you take any domain in the search space that doesn't have a solution in it or any socially than the number of trajectories in it decays exponentially quickly. And the decay rate is a characteristic in variant characteristic off the dynamics itself. Dynamical systems called the escape right the inverse off that is the time scale in which you find solutions by this by this dynamical system, and you can see here some song trajectories that are Kelty because it's it's no linear, but it's transient, chaotic. Give their sources, of course, because eventually knowledge to the solution. Now, in terms of performance here, what you show for a bunch off, um, constraint densities defined by M overran the ratio between closes toe variables for random, said Problems is random. Chris had problems, and they as its function off n And we look at money toward the wartime, the wall clock time and it behaves quite value behaves Azat party nominally until you actually he to reach the set on set transition where the hardest problems are found. But what's more interesting is if you monitor the continuous time t the performance in terms off the A narrow, continuous Time t because that seems to be a polynomial. And the way we show that is, we consider, uh, random case that random three set for a fixed constraint density Onda. We hear what you show here. Is that the right of the trash hold that it's really hard and, uh, the money through the fraction of problems that we have not been able to solve it. We select thousands of problems at that constraint ratio and resolve them without algorithm, and we monitor the fractional problems that have not yet been solved by continuous 90. And this, as you see these decays exponentially different. Educate rates for different system sizes, and in this spot shows that is dedicated behaves polynomial, or actually as a power law. So if you combine these two, you find that the time needed to solve all problems except maybe appear traction off them scales foreign or merely with the problem size. So you have paranormal, continuous time complexity. And this is also true for other types of very hard constraints and sexual problems such as exact cover, because you can always transform them into three set as we discussed before, Ramsey coloring and and on these problems, even algorithms like survey propagation will will fail. But this doesn't mean that P equals NP because what you have first of all, if you were toe implement these equations in a device whose behavior is described by these, uh, the keys. Then, of course, T the continue style variable becomes a physical work off. Time on that will be polynomial is scaling, but you have another other variables. Oxidative variables, which structured in an exponential manner. So if they represent currents or voltages in your realization and it would be an exponential cost Al Qaeda. But this is some kind of trade between time and energy, while I know how toe generate energy or I don't know how to generate time. But I know how to generate energy so it could use for it. But there's other issues as well, especially if you're trying toe do this son and digital machine but also happens. Problems happen appear. Other problems appear on in physical devices as well as we discuss later. So if you implement this in GPU, you can. Then you can get in order off to magnitude. Speed up. And you can also modify this to solve Max sad problems. Uh, quite efficiently. You are competitive with the best heuristic solvers. This is a weather problems. In 2016 Max set competition eso so this this is this is definitely this seems like a good approach, but there's off course interesting limitations, I would say interesting, because it kind of makes you think about what it means and how you can exploit this thes observations in understanding better on a low continues time complexity. If you monitored the discrete number the number of discrete steps. Don't buy the room, Dakota integrator. When you solve this on a digital machine, you're using some kind of integrator. Um and you're using the same approach. But now you measure the number off problems you haven't sold by given number of this kid, uh, steps taken by the integrator. You find out you have exponential, discrete time, complexity and, of course, thistles. A problem. And if you look closely, what happens even though the analog mathematical trajectory, that's the record here. If you monitor what happens in discrete time, uh, the integrator frustrates very little. So this is like, you know, third or for the disposition, but fluctuates like crazy. So it really is like the intervention frees us out. And this is because of the phenomenon of stiffness that are I'll talk a little bit a more about little bit layer eso. >>You know, it might look >>like an integration issue on digital machines that you could improve and could definitely improve. But actually issues bigger than that. It's It's deeper than that, because on a digital machine there is no time energy conversion. So the outside variables are efficiently representing a digital machine. So there's no exponential fluctuating current of wattage in your computer when you do this. Eso If it is not equal NP then the exponential time, complexity or exponential costs complexity has to hit you somewhere. And this is how um, but, you know, one would be tempted to think maybe this wouldn't be an issue in a analog device, and to some extent is true on our devices can be ordered to maintain faster, but they also suffer from their own problems because he not gonna be affect. That classes soldiers as well. So, indeed, if you look at other systems like Mirandizing machine measurement feedback, probably talk on the grass or selected networks. They're all hinge on some kind off our ability to control your variables in arbitrary, high precision and a certain networks you want toe read out across frequencies in case off CM's. You required identical and program because which is hard to keep, and they kind of fluctuate away from one another, shift away from one another. And if you control that, of course that you can control the performance. So actually one can ask if whether or not this is a universal bottleneck and it seems so aside, I will argue next. Um, we can recall a fundamental result by by showing harder in reaction Target from 1978. Who says that it's a purely computer science proof that if you are able toe, compute the addition multiplication division off riel variables with infinite precision, then you could solve any complete problems in polynomial time. It doesn't actually proposals all where he just chose mathematically that this would be the case. Now, of course, in Real warned, you have also precision. So the next question is, how does that affect the competition about problems? This is what you're after. Lots of precision means information also, or entropy production. Eso what you're really looking at the relationship between hardness and cost of computing off a problem. Uh, and according to Sean Hagar, there's this left branch which in principle could be polynomial time. But the question whether or not this is achievable that is not achievable, but something more cheerful. That's on the right hand side. There's always going to be some information loss, so mental degeneration that could keep you away from possibly from point normal time. So this is what we like to understand, and this information laws the source off. This is not just always I will argue, uh, in any physical system, but it's also off algorithm nature, so that is a questionable area or approach. But China gets results. Security theoretical. No, actual solar is proposed. So we can ask, you know, just theoretically get out off. Curiosity would in principle be such soldiers because it is not proposing a soldier with such properties. In principle, if if you want to look mathematically precisely what the solar does would have the right properties on, I argue. Yes, I don't have a mathematical proof, but I have some arguments that that would be the case. And this is the case for actually our city there solver that if you could calculate its trajectory in a loss this way, then it would be, uh, would solve epic complete problems in polynomial continuous time. Now, as a matter of fact, this a bit more difficult question, because time in all these can be re scared however you want. So what? Burns says that you actually have to measure the length of the trajectory, which is a new variant off the dynamical system or property dynamical system, not off its parameters ization. And we did that. So Suba Corral, my student did that first, improving on the stiffness off the problem off the integrations, using implicit solvers and some smart tricks such that you actually are closer to the actual trajectory and using the same approach. You know what fraction off problems you can solve? We did not give the length of the trajectory. You find that it is putting on nearly scaling the problem sites we have putting on your skin complexity. That means that our solar is both Polly length and, as it is, defined it also poorly time analog solver. But if you look at as a discreet algorithm, if you measure the discrete steps on a digital machine, it is an exponential solver. And the reason is because off all these stiffness, every integrator has tow truck it digitizing truncate the equations, and what it has to do is to keep the integration between the so called stability region for for that scheme, and you have to keep this product within a grimace of Jacoby in and the step size read in this region. If you use explicit methods. You want to stay within this region? Uh, but what happens that some off the Eigen values grow fast for Steve problems, and then you're you're forced to reduce that t so the product stays in this bonded domain, which means that now you have to you're forced to take smaller and smaller times, So you're you're freezing out the integration and what I will show you. That's the case. Now you can move to increase its soldiers, which is which is a tree. In this case, you have to make domain is actually on the outside. But what happens in this case is some of the Eigen values of the Jacobean, also, for six systems, start to move to zero. As they're moving to zero, they're going to enter this instability region, so your soul is going to try to keep it out, so it's going to increase the data T. But if you increase that to increase the truncation hours, so you get randomized, uh, in the large search space, so it's it's really not, uh, not going to work out. Now, one can sort off introduce a theory or language to discuss computational and are computational complexity, using the language from dynamical systems theory. But basically I I don't have time to go into this, but you have for heart problems. Security object the chaotic satellite Ouch! In the middle of the search space somewhere, and that dictates how the dynamics happens and variant properties off the dynamics. Of course, off that saddle is what the targets performance and many things, so a new, important measure that we find that it's also helpful in describing thesis. Another complexity is the so called called Makarov, or metric entropy and basically what this does in an intuitive A eyes, uh, to describe the rate at which the uncertainty containing the insignificant digits off a trajectory in the back, the flow towards the significant ones as you lose information because off arrows being, uh grown or are developed in tow. Larger errors in an exponential at an exponential rate because you have positively up north spawning. But this is an in variant property. It's the property of the set of all. This is not how you compute them, and it's really the interesting create off accuracy philosopher dynamical system. A zay said that you have in such a high dimensional that I'm consistent were positive and negatively upon of exponents. Aziz Many The total is the dimension of space and user dimension, the number off unstable manifold dimensions and as Saddam was stable, manifold direction. And there's an interesting and I think, important passion, equality, equality called the passion, equality that connect the information theoretic aspect the rate off information loss with the geometric rate of which trajectory separate minus kappa, which is the escape rate that I already talked about. Now one can actually prove a simple theorems like back off the envelope calculation. The idea here is that you know the rate at which the largest rated, which closely started trajectory separate from one another. So now you can say that, uh, that is fine, as long as my trajectory finds the solution before the projective separate too quickly. In that case, I can have the hope that if I start from some region off the face base, several close early started trajectories, they kind of go into the same solution orphaned and and that's that's That's this upper bound of this limit, and it is really showing that it has to be. It's an exponentially small number. What? It depends on the end dependence off the exponents right here, which combines information loss rate and the social time performance. So these, if this exponents here or that has a large independence or river linear independence, then you then you really have to start, uh, trajectories exponentially closer to one another in orderto end up in the same order. So this is sort off like the direction that you're going in tow, and this formulation is applicable toe all dynamical systems, uh, deterministic dynamical systems. And I think we can We can expand this further because, uh, there is, ah, way off getting the expression for the escaped rate in terms off n the number of variables from cycle expansions that I don't have time to talk about. What? It's kind of like a program that you can try toe pursuit, and this is it. So the conclusions I think of self explanatory I think there is a lot of future in in, uh, in an allo. Continue start computing. Um, they can be efficient by orders of magnitude and digital ones in solving empty heart problems because, first of all, many of the systems you like the phone line and bottleneck. There's parallelism involved, and and you can also have a large spectrum or continues time, time dynamical algorithms than discrete ones. And you know. But we also have to be mindful off. What are the possibility of what are the limits? And 11 open question is very important. Open question is, you know, what are these limits? Is there some kind off no go theory? And that tells you that you can never perform better than this limit or that limit? And I think that's that's the exciting part toe to derive thes thes this levian 10.