Image Title

Search Results for ADF analytics:

Rahul Pathak, AWS | AWS re:Invent 2021


 

>>Hey, welcome back everyone. We're live here in the cube in Las Vegas Raiders reinvent 2021. I'm Jeffrey hosted the key we're in person this year. It's a hybrid event online. Great action. Going on. I'm rolling. Vice-president of ADF analytics. David is great to see you. Thanks for coming on. >>It's great to be here, John. Thanks for having me again. >>Um, so you've got a really awesome job. You've got serverless, you've got analytics. You're in the middle of all the action for AWS. What's the big news. What are you guys announcing? What's going on? >>Yeah, well, it's been an awesome reinvent for us. Uh, we've had a number of several us analytics launches. So red shift, our petabyte scale data warehouse, EMR for open source analytics. Uh, and then we've also had, uh, managed streaming for Kafka go serverless and then on demand for Kinesis. And then a couple of other big ones. We've got RO and cell based security for AWS lake formation. So you can get really fine grain controls over your data lakes and then asset transactions. You can actually have a inserts, updates and deletes on data lakes, which is a big step forward. >>Uh, so Swami on stage and the keynote he's actually finishing up now. But even last night I saw him in the hallway. We were talking about as much as about AI. Of course, he's got the AI title, but AI is the outcome. It's the application of all the data and this and a new architecture. He said on stage just now like, Hey, it's not about the old databases from the nineties, right? There's multiple data stores now available. And there's the unification is the big trend. And he said something interesting. Governance can be an advantage, not an inhibitor. This is kind of this new horizontally scalable, um, kind of idea that enables the vertical specialization around machine learning to be effective. It's not a new architecture, but it's now becoming more popular. People are realizing it. It's sort of share your thoughts on this whole not shift, but the acceleration of horizontally scalable and vertically integrated. Yeah, >>No, I think the way Swami put it is exactly right. What you want is the right tool for the right job. And you want to be able to deliver that to customers. So you're not compromising on performance or functionality of scale, but then you wanted all of these to be interconnected. So they're, well-integrated, you can stay in your favorite interface and take advantage of other technologies. So you can have things like Redshift integrated with Sage makers, you get analytics and machine learning. And then in Swami's absolutely right. Governance is actually an enabler of velocity. Once you've got the right guardrails in place, you can actually set people free because they can innovate. You don't have to be in the way, but you know that your data is protected. It's being used in the way that you expect by the people that you are allowing to use that data. And so it becomes a very powerful way for customers to set data free. And then, because things are elastic and serverless, uh, you can really just match capacity with demand. And so as you see spikes in usage, the system can scale out as those dwindle, they can scale back down, and it just becomes a very efficient way for customers to operate with data at scale >>Every year it reinvented. So it was kind of like a pinch me moment. It's like, well, more that's really good technology. Oh my God, it's getting easier and easier. As the infrastructure as code becomes more programmable, it's becoming easier, more Lambda, more serverless action. Uh, you got new offerings. How are customers benefiting for instance, from the three new offerings that you guys announced here? What specifically is the value proposition that you guys are putting out there? Yeah, so the, >>Um, you know, as we've tried to do with AWS over the years, customers get to focus on the things that really differentiate them and differentiate their businesses. So we take away in Redshift serverless, for example, all of the work that's needed to manage clusters, provision them, scale them, optimize them. Uh, and that's all been automated and made invisible to customers, the customers to think about data, what they want to do with it, what insights they can derive from it. And they know they're getting the most efficient infrastructure possible to make that a reality for them with high performance and low costs. So, uh, better results, more ability to focus on what differentiates their business and lower cost structure over time. >>Yeah. I had the essential guys on it's interesting. They had part of the soul cloud. Continuous is their word for what Adam was saying is clouds everywhere. And they're saying it's faster to match what you want to do with the outcomes, but the capabilities and outcomes kind of merging together where it's easy to say, this is what we want to do. And here's the outcome it supports that's right with that. What are some of the key trends on those outcomes that you see with the data analytics that's most popular right now? And kind of where's that, where's that going? >>Yeah. I mean, I think what we've seen is that data's just becoming more and more critical and top of mind for customers and, uh, you know, the pandemic has also accelerated that we found that customers are really looking to data and analytics and machine learning to find new opportunities. How can they, uh, really expand their business, take advantage of what's happening? And then the other part is how can they find efficiencies? And so, um, really everything that we're trying to do is we're trying to connect it to business outcomes for customers. How can you deepen your relationship with your customers? How can you create new customer experiences and how can you do that more efficiently, uh, with more agility and take advantage of, uh, the ability to be flexible. And you know, what is a very unpredictable world, as we've seen, >>I noticed a lot of purpose-built discussion going on in the keynote with Swami as well. How are you creating this next layer of what I call purpose-built platform like features? I mean, tools are great. You see a lot of tools in the data market tools are tools of your hammer. You want to look for a nail. We see people over by too many tools and you have ultimately a platform, but this seems to be a new trend where there's this connect phenomenon was showing me that you've got these platform capabilities that people can build on top of it, because there's a huge ecosystem of data tools out there that you guys have as partners that want to snap together. So the trend is things are starting to snap together, less primitive, roll your own, which you can do, but there's now more easier ways. Take me through that. Explain that, unpack that that phenomenon role rolling your own firm is, which has been the way now to here. Here's, here's some prefabricated software go. >>Yeah. Um, so it's a great observation and you're absolutely right. I mean, I think there's some customers that want to roll their own and they'll start with instances, they'll install software, they'll write their own code, build their own bespoke systems. And, uh, and we provide what the customers need to do that. But I think increasingly you're starting to see these higher level abstractions that take away all of that detail. And mark has Adam put it and allow customers to compose these. And we think it's important when you do that, uh, to be modular. So customers don't have to have these big bang all or nothing approaches you can pick what's appropriate, uh, but you're never on a dead end. You can always evolve and scale as you need to. And then you want to bring these ideas of unified governance and cohesive interfaces across so that customers find it easy to adopt the next thing. And so you can start off say with batch analytics, you can expand into real time. You can bring in machine learning and predictive capabilities. You can add natural language, and it's a big ecosystem of managed services as well as third parties and partners. >>And what's interesting. I want to get your thoughts while I got you here, because I think this is such an important trend and historic moment in time, Jerry chin, who one of the smartest VCs that we know from Greylock and coin castles in the cloud, which kind of came out of a cube conversation here in the queue years ago, where we saw the movement of that someone's going to build real value on AWS, not just an app. And you see the rise of the snowflakes and Databricks and other companies. And he was pointing out that you can get a very narrow wedge and get a position with these platforms, build on top of them and then build value. And I think that's, uh, the number one question people ask me, it's like, okay, how do I build value on top of these analytic packages? So if I'm a startup or I'm a big company, I also want to leverage these high level abstractions and build on top of it. How do you talk about that? How do you explain that? Because that's what people kind of want to know is like, okay, is it enabling me or do I have to fend for myself later? This is kind of, it comes up a lot. >>That's a great question. And, um, you know, if you saw, uh, Goldman's announcement this week, which is about bringing, building their cloud on top of AWS, it's a great example of using our capabilities in terms of infrastructure and analytics and machine learning to really allow them to take what's value added about Goldman and their position to financial markets, to build something value, add, and create a ton of value for Goldman, uh, by leveraging the things that we offer. And to us, that's an ideal outcome because it's a win-win for us in Goldman, but it's also a win for Goldman and their customers. >>That's what we call the Supercloud that's the opportunity. So is there a lot of Goldmans opportunities out there? Is that just a, these unicorns, are these sites? I mean, how do you, I mean, that's Goldman Sachs, they're huge. Is there, is this open to everybody? >>Absolutely. I mean, that's been one of the, uh, you know, one of the core ideas behind AWS was we wanted to give anybody any developer access to the same technology that the world's largest corporations had. And, uh, that's what you have today. The things that Goldman uses to build that cloud are available to anybody. And you can start for a few pennies scale up, uh, you know, into the petabytes and beyond >>When I was talking to Adams, Lipski when I met with him prior to re-invent, I noticed that he was definitely had an affinity towards the data, obviously he's Amazonia, but he spent time at Tableau. So, so as he's running that company, so you see that kind of mindset of the data advantage. So I have to ask you, because it's something that I've been talking about for a while and I'm waiting for it to emerge, but I'm not sure it's going to happen yet. But what infrastructure is code was for dev ops and then dev sec ops, there's almost like a data ops developing where data as code or programmable data. If I can connect the dots of what Swami's saying, what you're doing is this is like a new horizontal layer of data of freely available data with some government governance built in that's right. So it's, data's being baked into everything. So data is any ingredient, not a query to some database, it's gotta be baked into the apps, that's data as code that's. Right. So it's almost a data DevOps kind of vibe. >>Yeah, no, you're absolutely right. And you know, you've seen it with things like ML ops and so on. It's all the special case of dev ops. But what you're really trying to do is to get programmatic and systematic about how you deal with data. And it's not just data that you have. It's also publicly available data sets and it's customers sharing with each other. So building the ecosystem, our data, and we've got things like our open data program where we've got publicly hosted data sets or things like the AWS data exchange where customers can actually monetize data. So it's not just data as code, but now data as a monetizeable asset. So it's a really exciting time to be in the data business. >>Yeah. And I think it's so many too. So I've got to ask you while I got you here since you're an expert. Um, okay. Here's my problem. I have a lot of data. I'm nervous about it. I want to secure it. So if I try to secure it, I'm not making it available. So I want to feed the machine learning. How do I create an architecture where I can make it freely available, but yet maintain the control and the comfort that this is going to be secure. So what products do I buy? >>Yeah. So, uh, you know, a great place to start at as three. Um, you know, it's one of the best places for data lakes, uh, for all the reasons. That's why we talked about your ability scale costs. You can then use lake formation to really protect and govern that data so you can decide who's allowed to see it and what they're allowed to see, and you don't have to create multiple copies. So you can define that, you know, this group of partners can see a, B and C. This group can see D E and F and the system enforces that. And you have a central point of control where you can monitor what's happening. And if you want to change your mind, you can do that instantly. And all access can be locked down that you've got a variety of encryption capabilities with things like KMS. And so you can really lock down your data, but yet keep it open to the parties that you want and give them specifically the access that you want to give them. And then once you've done that, they're free to use that data, according to the rules that you defined with the analytics tools that we offer to go drive value, create insight, and do something >>That's lake formation. And then you got a Thena querying. Yes, we got all kinds of tooling on top of it. >>It's all right. You can have, uh, Athena query and your data in S3 lake formation, protecting it. And then SageMaker is integrated with Athena. So you can pull that data into SageMaker for machine learning, interrogate that data, using natural language with things like QuickSight Q a like we demoed. So just a ton of power without having to really think too deeply about, uh, developing expert skill sets in this. >>So the next question I want to ask you is because that first part of the great, great, great description, thank you very much. Now, 5g in the edges here, outpost, how was the analytics going on that as edge becomes more pervasive in the architecture? >>Yeah, it's going to be a key part of this ecosystem and it's really a continuum. So, uh, you know, we find customers are collecting data at the edge. They might be making local ML or inference type decisions on edge devices, or, you know, automobiles, for example. Uh, but typically that data with some point will come back into the cloud, into S3 will be used to do heavy duty training, and then those models get pushed back out to the edge. And then some of the things that we've done in Athena, for example, with federated query, as long as you have a network path, and you can understand what the data format or the database is, you can actually run a query on that data. So you can run real-time queries on data, wherever it lives, whether it's on an edge device, on an outpost, in a local zone or in your cloud region and combine all of that together in one place. >>Yeah. And I think having that data copies everywhere is a big thing deal. I've got to ask you now that we're here at reinvent, what's your take we're back in person last year was all virtual. Finally, not 60,000 people, like a couple of years ago, it's still 27,000 people here, all lining up for the sessions, all having a great time. Um, all good. What's the most important story from your, your area that people should pay attention to? What's the headline, what's the top news? What should people pay attention to? >>Yeah, so I think first off it is awesome to be back in person. It's just so fun to see customers and to see, I mean, you, like, we've been meeting here over the years and it's, it's great to so much energy in person. It's been really nice. Uh, you know, I think from an analytics perspective, there's just been a ton of innovation. I think the core idea for us is we want to make it easy for customers to use the right tool for the right job to get insight from all of their data as cost effectively as possible. And I think, uh, you know, I think if customers walk away and think about it as being, it's now easier than ever for me to take advantage of everything that AWS has to offer, uh, to make sense of all the data that I'm generating and use it to drive business value, but I think we'll have done our jobs. Right. >>What's the coolest thing that you're seeing here is that the serverless innovation, is it, um, the new abstraction layer with data high level services in your mind? What's the coolest thing. Got it. >>It's hard to pick the coolest that sticks like kicking the candies. I mean, I think the, uh, you know, the continued innovation in terms of, uh, performance and functionality in each of our services is a big deal. I think serverless is a game changer for customers. Uh, and then I think really the infusion of machine learning throughout all of these systems. So things like Redshift ML, Athena ML, Pixar, Q a just really enabling new experiences for customers, uh, in a way that's easier than it ever has been. And I think that's a, that's a big deal and I'm really excited to see what customers do with it. >>Yeah. And I think the performance thing to me, the coolest thing that I'm seeing is the graviton three and the gravitron progression with the custom stacks with all this ease of use, it's just going to be just a real performance advantage and the costs are getting lowered. So I think the ECE two instances around the compute is phenomenal. No, >>Absolutely. I mean, I think the hardware and Silicon innovation is huge and it's not just performance. It's also the energy efficiency. It's a big deal for the future reality. >>We're at an inflection point where this modern applications are being built. And in my history, I'm old, my birthday is today. I'm in my fifties. So I remember back in the eighties, every major inflection point when there was a shift in how things were developed from mainframe client server, PC inter network, you name it every time the apps change, the app owners, app developers all went to the best platform processing. And so I think, you know, that idea of system software applications being bundled together, um, is a losing formula. I think you got to have that decoupling large-scale was seeing that with cloud. And I think now if I'm an app developer, whether whether I'm in a large ISV in your ecosystem or in the APN partner or a startup, I'm going to go with my software runs the best period and where I can create value. That's right. I get distribution, I create value and it runs fast. I mean, that's, I mean, it's pretty simple. So I think the ecosystem is going to be a big action for the next couple of years. >>Absolutely. Right. And I mean, the ecosystem's huge and I think, um, and we're also grateful to have all these partners here. It's a huge deal for us. And I think it really matters for customers >>What's on your roadmap this year, what you got going on. What can you share a little bit of a trajectory without kind of, uh, breaking the rules of the Amazonian, uh, confidentiality. Um, what's, what's the focus for the year? What do you what's next? >>Well, you know, as you know, we're always talking to customers and, uh, I think we're going to make things better, faster, cheaper, easier to use. And, um, I think you've seen some of the things that we're doing with integration now, you'll see more of that. And, uh, really the goal is how can customers get value as quickly as possible for as low cost as possible? That's how we went to >>Yeah. They're in the longterm. Yeah. We've always say every time we see each other data is at the center of the value proposition. I've been saying that for 10 years now, it's actually the value proposition, powering AI. And you're seeing because of it, the rise of superclouds and then the superclouds are emerging. I think you guys are the under innings of these emerging superclouds. And so it's a huge treading, the Goldman Sachs things of validation. So again, more data, the better, sorry, cool things happening. >>It is just it's everywhere. And the, uh, the diversity of use cases is amazing. I mean, I think from, you know, the Australia swimming team to, uh, to formula one to NASDAQ, it's just incredible to see what our >>Customers do. We see the great route. Good to see you. Thanks for coming on the cube. >>Pleasure to be here as always John. Great to see you. Thank you. Yeah. >>Thanks for, thanks for sharing. All of the data is the key to the success. Data is the value proposition. You've seen the rise of superclouds because of the data advantage. If you can expose it, protect it and govern it, unleashes creativity and opportunities for entrepreneurs and businesses. Of course, you got to have the scale and the price performance. That's what doing this is the cube coverage. You're watching the leader in worldwide tech coverage here in person for any of us reinvent 2021 I'm John ferry. Thanks for watching.

Published Date : Dec 1 2021

SUMMARY :

David is great to see you. It's great to be here, John. What are you guys announcing? So you can get really fine grain controls over your data lakes and then asset transactions. It's the application of all the data and this and a new architecture. And so as you see spikes in usage, the system can scale out How are customers benefiting for instance, from the three new offerings that you guys announced the customers to think about data, what they want to do with it, what insights they can derive from it. And they're saying it's faster to match what you want to do with the outcomes, And you know, what is a very unpredictable world, as we've seen, tools out there that you guys have as partners that want to snap together. So customers don't have to have these big bang all or nothing approaches you can pick And he was pointing out that you can get a very narrow wedge and get a position And, um, you know, if you saw, uh, Goldman's announcement this week, Is there, is this open to everybody? I mean, that's been one of the, uh, you know, one of the core ideas behind AWS was we wanted to give so you see that kind of mindset of the data advantage. And it's not just data that you have. So I've got to ask you while I got you here since you're an expert. And so you can really lock down your data, but yet And then you got a Thena querying. So you can pull that data into SageMaker for machine learning, So the next question I want to ask you is because that first part of the great, great, great description, thank you very much. data format or the database is, you can actually run a query on that data. I've got to ask you now that we're here at reinvent, And I think, uh, you know, I think if customers walk away and think about it as being, What's the coolest thing that you're seeing here is that the serverless innovation, I think the, uh, you know, the continued innovation in terms of, uh, So I think the ECE two instances around the compute is phenomenal. It's a big deal for the future reality. And so I think, you know, And I think it really matters for customers What can you share a little bit of a trajectory without kind of, Well, you know, as you know, we're always talking to customers and, uh, I think we're going to make things better, I think you guys are the under innings of these emerging superclouds. I mean, I think from, you know, the Australia swimming team to, uh, to formula one to NASDAQ, Thanks for coming on the cube. Great to see you. All of the data is the key to the success.

SENTIMENT ANALYSIS :

ENTITIES

EntityCategoryConfidence
DavidPERSON

0.99+

GoldmanORGANIZATION

0.99+

Rahul PathakPERSON

0.99+

AWSORGANIZATION

0.99+

JohnPERSON

0.99+

Goldman SachsORGANIZATION

0.99+

AdamPERSON

0.99+

Jerry chinPERSON

0.99+

NASDAQORGANIZATION

0.99+

AthenaLOCATION

0.99+

JeffreyPERSON

0.99+

2021DATE

0.99+

60,000 peopleQUANTITY

0.99+

27,000 peopleQUANTITY

0.99+

10 yearsQUANTITY

0.99+

last yearDATE

0.99+

John ferryPERSON

0.99+

todayDATE

0.99+

threeQUANTITY

0.98+

KafkaTITLE

0.98+

SwamiPERSON

0.98+

oneQUANTITY

0.98+

firstQUANTITY

0.98+

ADF analyticsORGANIZATION

0.97+

AmazoniaORGANIZATION

0.97+

eightiesDATE

0.97+

PixarORGANIZATION

0.97+

fiftiesQUANTITY

0.97+

eachQUANTITY

0.97+

three new offeringsQUANTITY

0.96+

RedshiftTITLE

0.96+

first partQUANTITY

0.96+

this yearDATE

0.96+

last nightDATE

0.95+

LipskiPERSON

0.94+

couple of years agoDATE

0.94+

next couple of yearsDATE

0.94+

SageORGANIZATION

0.93+

GoldmansORGANIZATION

0.92+

pandemicEVENT

0.92+

this weekDATE

0.92+

DatabricksORGANIZATION

0.9+

one placeQUANTITY

0.9+

markPERSON

0.88+

SupercloudORGANIZATION

0.87+

TableauORGANIZATION

0.84+

S3TITLE

0.84+

5gQUANTITY

0.8+

Athena MLORGANIZATION

0.78+

AthenaORGANIZATION

0.78+

RaidersORGANIZATION

0.77+

years agoDATE

0.75+

ninetiesDATE

0.74+

MLORGANIZATION

0.73+

AdamsPERSON

0.73+

two instancesQUANTITY

0.72+

LambdaTITLE

0.71+

ThenaORGANIZATION

0.71+

SageMakerTITLE

0.66+

VegasLOCATION

0.66+

InventEVENT

0.64+

VicePERSON

0.63+

graviton threeOTHER

0.62+

AustraliaLOCATION

0.59+

LasORGANIZATION

0.59+

KinesisTITLE

0.57+

AmazonianORGANIZATION

0.56+