well hello everybody John Wallace here on thecube he's continuing our segments here on the AWS Global startup showcase we are at day three of Reinventing irking Zhang is joining us now he is the CEO co-founder of Jupiter one um first off before we get going talking about you know security and big world for you guys I know what's your take on the show what's been going on out here at re invent yeah yeah ring event has been one of my favorite shows there's a lot of people here there's a lot of topics of course it's not just cyber security a lot of cloud infrastructure and just technology in general so you get a lot you know if you go walk the floor you see a lot of vendors you look at us go into sessions you can learn a lot but you're the Hot Topic right everybody's focused on Cyber yeah big time and with good reason right because as we know the Bad actors are getting even smarter and even faster and even more Nimble so just paint the landscape for me here in general right now as you see uh security Cloud Security in particular and and kind of where we are in that battle well we are clearly not winning so I think that in itself is a bit of a uh interesting problem right so as a it's not just Cloud security if you think about cyber security in general as an industry it has it has not been around for that long right but if you just look at the history of it uh we haven't done that while so uh pick another industry say medicine which has been around forever and if you look at the history of Medicine well I would argue you has done tremendously well because people live longer right when you get sick you get access to health care and yeah exactly you have Solutions and and you can see the trend even though there are problems in healthcare of course right but the trend is is good it's going well but not in cyber security more breaches more attacks more attackers we don't know what the hell we're doing with that many solutions and you know that's been one of my struggles as a former CSO and security practitioner for many years you know why is it that we're not getting better all right so I'm going to ask you the question yeah okay why aren't we getting better you know how come we can't stay ahead of the curve on this thing that for some reason it's like whack-a-mole times a hundred every time we think we solve one problem we have a hundred more that show up over here exactly and we have to address that and and our attention keeps floating around yeah I think you said it right so because we're taking this guacamole approach and we're looking for the painkiller of the day and you know we're looking for uh the Band-Aids right so and then we ended up well I I think to be fair to be fair to your industry the industry moves so quickly technology in general moves so quickly and security has been playing catch-up over time we're still playing catch-up so when you're playing catch-up you you can almost only uh look at you know what's the painkiller of what's the band name of the day so I can stop the bleeding right but I do think that we're we're to a point or we have enough painkillers and Band-Aids and and we need to start looking at how can we do better fundamentally with the basics and do the basics well because a lot of times the basics that get you into trouble so fundamentally the foundation I if I hear you right what you're saying is um you know quick changing industry right things are moving rapidly but we're not blocking and tackling we're not doing the X's and O's and so forget changing and we we got to get back to the basis and do those things right exactly you can only seem so simple it seems so simple but it's so hard right so you can you can think about you know uh even in case of building a starter building a company and and in order at one point right so we're blocking uh blocking tackling and then when we grow to a certain size we have to scale we have to figure out how to scale the business this is the same problem that happens in security as an industry we've been blocking happening for so long you know we're the industry is so young but we're to a point that we got to figure out how to scale this scale this in a fundamentally different way and I'll give you some example right so so what when we say the basics now it's easy to to think that say users should have MFA enabled is one of the basics right or another Basics will be you have endpoint protection on your devices you know maybe it's Cloud strike or Sentinel one or carbon black or whatever but the question being how do you know it is working 100 of the time right how do you know that how do you know right you find out too exactly that's right and how do you know that you have 100 coverage on your endpoints those Solutions are not going to tell you because they don't know what they don't know right if it's not enabled if it's not you know what what's the negative that you are not seeing so that's one of the things that you know that's in the basic state that you're now covering so the fundamentals it really goes to these five questions that I think that nobody has a really good answer for until now so the five questions goes what do I have right is it important what's important out of all the things I have you have a lot right you could have millions of things what important now for those that are important does it have a problem and if it has a problem who can fix it because the reality is in most cases security teams are not the ones fixing the problems they're they're the ones identical they're very good at recognizing but not so good exactly identifying the owner who can fix it right right could be could be business owner could be Engineers so the the asset ownership identification right so so these four questions and and then over time you know whether it's over a week or a month or a quarter or a year am I getting better right and then you just keep asking these questions in different areas in different domains with a different lens right so maybe that's endpoints maybe that's Cloud maybe that's you know users maybe that's a product and applications right but it really boils down to these five questions that's the foundation for any good security program if you can do that well I think we cover a lot of bases and we're going to be in much better shape than we have been all right so where do you come in man Jupiter one in terms of what you're providing because obviously you've identified this kind of pyramid yes this hierarchy of addressing needs and I assume obviously knowing you as I do and knowing the company as I do you've got Solutions that's exactly right right and and we precisely answer those five questions right for uh any organization uh from a asset perspective right because all the the answers to all those these five questions are based in assets it starts with knowing what I have right right so the the overall challenge of cyber security being broke broken I I believe is fundamentally that people do not understand and cannot uh probably deal with the complexity that we have within our own environments so again like you know using uh medicine as an example right so in order to come up with the right medicine for either it's a vaccine for covid-19 or whether it is a treatment for cancer or whatever that case may be you have to start with the foundations of understanding both the pathogen and to the human body like DNA sequencing right without those you cannot effectively produce the right medicine in modern uh you know Medicine sure right so that is the same thing that's happening in cyber security you know we spend a lot of times you know putting band days in patches right and then we spend a lot of time doing attacker research from the outside but we don't fundamentally understand in a complete way what's the complexity within our own environment in terms of digital assets and that's that's almost like the DNA of your own work what is that kind of mind-blowing in a way that if again hearing you what you're talking about is saying that the first step is to identify what you have that's right so it seems just so basic that that I should know what I what's under my hood I should know what is valuable and what is not I should prioritize what I really need to protect and what maybe can go on the second shelf yeah it has been a tough problem since the beginning of I.T not just the beginning of cyber security right so in the history of I.T we have this thing called cmdb configuration management database it is supposed to capture the configurations of it assets now over time that has become a lot more complex and and there's a lot more than just it asset that we have to understand from a security and attack service perspective right so we have to understand I.T environments we have to understand Cloud environments and applications and users and access and data and as and all of those things then then we have to take a different approach of sort of a modern cmdb right so what is the way that we can understand all of those complexity within all of those assets but not just independently within those silos but rather in a connected way so we can not only understand the attack surface but only but also understand the attack path that connect the dots from one thing to another right because everything in the organization is actually connected if if there's any one thing that sits on an island right so if you say you have a a a a server or a device or a user that is on an island that is not connected to the rest of the organization then why have it right and it doesn't matter so it's the understanding of that connect connected tissue this entire map where this you know DNA sequencing equivalent of a digital organization is what Jupiter one provides right so that visibility of the fundamental you know very granular uh level of assets and resources to answer those five questions and how does that how do I get better at that then I mean I have you to help me but but internally within our organization um I mean I don't want to be rude but I mean do I have do I have the skill for that do I have um do I have the the internal horsepower for that or or is there some need to close that Gap and how do I do it you know I'll tell you two things right so so one you mentioned the worst skills right so let me start there so because this one is very interesting we also have a huge skills shortage in cyber security we will we've all heard that for years and and and and for a long time but if you dig deeper into it why is that why is that and you know we have a lot of you know talented people right so why do we still have a skills shortage now what's interesting is if you think about what we're asking security people to do is mind-boggling so if you if you get a security analyst to say hey I want to understand how to protect something or or how to deal with an incident and what you're asking the person to do is not only to understand the security concept and be a domain expert in security you're also asking the person to and understand at the same time AWS or other clouds or endpoints or code or applications so that you can properly do the analysis and the in the response it's it's impossible it's like you know if you have you have to have a person who's an expert in everything know everything about everything that's right it's impossible so so so that's that's one thing that we have to to resolve is how do we use technology like Jupiter one to provide an abstraction so that there's Automation in place to help the security teams be better at their jobs without having to be an expert in deep technology right just add the abstract level of understanding because you know we can we can model the data and and provide the analysis and visual visualization out of the box for them so they can focus on just the security practices so that's one and the second thing is we have to change the mindset like take vulnerability management as an example right so the mindset for vulnerability management has been how do I manage findings now we have to change it to the concept of more proactive and how to manage assets so let's think about uh you know say log4j right that that happened and uh you know when it happened everybody scrambles and said hey which which devices or which you know uh systems have log4j and you know it doesn't matter what's the impact we can fix it right going back to those questions that that I mentioned before right and then um and then they try to look for a solution at a time say well where's that silver bullet that can give me the answers now what what what we struggle with though is that you know I want to maybe ask the question where were you six months ago where were you six months ago where you could have done the due diligence and put something in place that help you understand all of these assets and connections so you can go to one place and just ask for that question when something like that you know hit the fan so so if we do not fundamentally change the mindset to say I have to look at things not from a reactive findings perspective but really starting from an asset-centric you know day one perspective to look at that and have this Foundation have this map build we can't get there right so it's like you know if I need direction I go to Google Maps right but the the reason that it works is because somebody has done the work of creating the map right right if you haven't if you don't have the map and you just at you know when the time you say I gotta go somewhere and you expect the map to magically happen to show you the direction it's not going to work right right I imagine there are a lot of people out there right now are listening to thinking oh boy you know and that's what Jupiter one's all about they're there to answer your oh boy thanks for the time of course I appreciate the insights as well it's nice to know that uh at least somebody is reminding us to keep the front door locked too that's just the back door the side doors keep that front door and that garage locked up too definitely um all right we'll continue our coverage here at AWS re invent 22 this is part of the AWS Global startup showcase and you're watching the cube the leader in high-tech coverage foreign

(uplifting music) >> The data lake we see is evolving and ChaosSearch has built some pretty cool tech to enable customers to get more value out of data that's in lakes so that it doesn't become stagnant. Time to dig deeper, dive deeper into the water. We're here with Kevin Miller, who's the vice president and general manager of S3 at Amazon Web Services. We're going to talk about activating S3 for analytics. Kevin, welcome. Good to see you again. >> Yeah, thanks, Dave. It's great to be here again. >> So S3 was the very first service offered by AWS 15 years ago. We covered that out in Seattle. It was a great event you guys had. It has become the most prominent and popular example of object storage in the marketplace. And for years, customers use S3 as simple, cheap data storage, but because there's so much data now stored in S3, customers are looking to do more with the platform. So, Kevin, as we look ahead to reinvent this year, we're super excited about that, what's new? What's got you excited when it comes to the AWS flagship storage offering? >> Yeah, Dave. Well, that's right. And we're definitely looking forward to reinvent. We have some fun things that we're planning to announce there, so stay tuned on those. But, I'd say that one of the things that's most exciting for me as customers do more with their data and look to store more, to capture more of the data that they're generating every day, is our storage class that we had an announce a few years ago. But we actually just announced some improvements to the S3, intelligent-tiering storage class. And this is really our storage class, the only one in the cloud at this point that delivers automatic storage cost savings for customers where the data access patterns change. And that can happen, for example, as customers have some data that they're collecting and then a team spins up and decides to try to do something more with that data, and that data that was very cool and sitting sort of idle is now being actively used. And so with intelligent tiering, we're automatically monitoring data. And then, for customers there's no retrieval costs and no tiering charges. We're automatically moving the data into an access tier that reduces their costs, though, when that that data is not being accessed. So we've announced some improvements to that just a few months ago. And I'll just say, I look forward to some more announcements at reinvent, that will continue to extend what we have in our intelligent-tiering storage class. >> That's cool, Kevin. I mean, you've seen, you know, that technology, that tiering concept had been around, you know. But since back in the mainframe days the problem was it was always inside a box. So you didn't have the scale of the cloud and you didn't have that automation. So, I want to ask you, as the leader of S3, that business, when you meet with customers, Kevin, what do they tell you that they're facing as challenges when they want to do more, get better insights out of all that data that they've moved into S3? >> Well, I think that's just it, Dave. I think that most customers I speak with, of course they have the things that they want to do with their storage costs, you know: reducing storage costs and just making sure they have capacity available. But increasingly I think the real emphasis is around business transformation. What can I do with this data, that's very unique and different that unlike, you know, prior optimizations where it would just reduce the bottom line, they're saying, what can I do that will actually drive my top line more by either, you know, generating new product ideas, allowing for faster closed-loop process for acquiring customers? And so it's really that business transformation and everything around it that I think is really exciting. And for a lot of customers, that's a pretty long journey. And helping them get started on that, including transforming their workforce, and up-skilling, you know, parts of their workforce to be more agile and more oriented around software development, developing new products using software. >> So when I first met the folks at ChaosSearch, Thomas took me through sort of the architecture with Ed as well. They had me at "you don't have to move your data." That was the grabber for me. And there are a number of public customers, Digital River, Blackboard, or Klarna, we're going to get the customer perspective little later on, and others, that use both AWS S3 and ChaosSearch. And they're trying to get more out of their S3 data and execute analytics at scale. So, wonder if you could share with us, Kevin, what types of activities and opportunities do you see for customers like these that are making the move to put their enterprise data in S3, in terms of capabilities and outcomes that they are trying to achieve and are able to achieve beyond using S3 as just a bit bucket? >> Right. Well, Dave, I think you hit the nail on the head when you talk about outcomes. 'Cause that, I think, is key here. Customers want to reduce the time it takes to get to a tangible result that affects their business, that improves their business. And so that's one of the things that excites me about what ChaosSearch is doing here, specifically is that automatic indexing. Being able to take the data, as it is, in their bucket, index it and keep that index fresh and then allow for the customers to innovate on top of that and to try to experiment with a new capability, see why it works and then double down on the things that really do work to drive that business. And so, I just think that that capability reduces the amount of what I might call undifferentiated heavy-lifting the work to just sort of index and organize and catalog data. And instead allow customers to really focus on here's the idea, let's try to get this into production or into a test environment as quickly as possible to see if this can really drive some value for our business. >> Yeah, so you're seeing that sort of value that you've mentioned, the non-differentiated heavy-lifting, moving up the stack, right? It used to just be provisioning and managing the storage. Now it's all the layers above that. And we're going beyond that. So my question to you, Kevin, is, how do you see the evolution of all this data at scale? I'm especially interested as it pertains to data that's, of course, in S3, which is your swim lane. When you talk to customers who want to do more with their data and analytics, and, by the way, even beyond analytics, you know, where it's having conversations now in the community about building data products and creating new value. But how do you respond and how do you see ChaosSearch fitting in to those outcomes? >> Well, I think that's it, Dave. It's about kind of going up the stack and instead of spending time organizing and cataloging data, particularly as the data volumes get much larger. When modern customers and modern data lakes that we're seeing, quickly go from a few petabytes to tens, to hundreds of petabytes or more. And, when you're reaching that kind of scale of data, a single person can't reasonably kind of wrap their head around all that data, you need tools. S3 provides a number of first party tools. And, you know, we're investing in things like our S3 batch operations to really help give the end users of that data, the business owners that leverage to manage their data at scale and apply their new ideas to the data and generate, you know, pilots and production work that really drives their business forward. And so I think that, you know, ChaosSearch, again, I would just say is a good example of, you know, the kind of software that I think helps go upstack, automate some of that data management, and just help customers focus really specifically on the things that they want to accomplish for their business. >> So this is really important. I mean, we've talked for well over a decade, how to get more value out of data, and it's been challenging for a lot of organizations. But we're seeing themes of scale, automation, fine-grain tooling ecosystem participating on top of that data and then extracting that data value. Kevin, I'm really excited to see you face to face at Reinventing, and learn more about some of the announcements that you're going to make. We'll see you there. >> Yeah. Stay tuned. Looking forward to seeing you in person. Absolutely. >> All right. Great to have Kevin on. Keep it right there because in a moment we're going to get the customer perspective on how a leading practitioner is applying ChaosSearch on top of S3 to create business value from data. You're watching The Cube, your leader at digital high-tech coverage. (uplifting music)

>>From around the globe. It's the cube with digital coverage of AWS reinvent 2020 sponsored by Intel and AWS. >>Welcome back to the cubes live coverage of AWS reinvent 2020 I'm Lisa Martin and I have with me a cube alumni back, please. Welcome Scott Mullins, the worldwide financial services business development leader at AWS. Scott. Welcome back. Great to have you joining us, >>Lisa. It's great to be back on the cube and to be visiting with you today from virtual re-invent 2020. >>Yes. Reinventing reinvent. The last show that I got to host in-person for the cube was reinvent last year. And here we have this three week virtual event that started last week. So lots more even going on. I think I even saw a hundred thousand or so registered, so massive event, lots of news. So walk us through some of the highlights that have been announced at reinvent this year and some of the things that you're seeing the most interest from customers in. >>Well, I think one of the big highlights is 500,000 registrants that are reinvented 50,000 attendees last year to reinvent or 50,000 or so to 500,000 re registered for the event. So that's, that's, that's worth talking about in its own. Right. But I think, you know, one of the things, and you mentioned this, you know, more re-invent three weeks, uh, this year, as opposed to the four days that we normally spend in Las Vegas together, physically, when you do, when you do it digitally, you have the ability to actually include more things and more leaders talking about things. And so when we think about the announcements that are having impacts, uh, with financial services customers specifically I'd point to a couple of things and, you know, they're obviously gonna mention Andy's keynote, but there's going to be some things that you might go wait a minute. >>I didn't even see that announcement. Uh, and then maybe I could point you and the viewers to some other, other, um, keynotes or some other sessions that were announced. So obviously I think, uh, first and foremost in Andy's keynote, uh, hybrid, uh, was something that was a very, uh, big focus for him and I for a very long time, we've had the messaging of the right tool for the right job when it comes to any of your services. I think you could alter that today to say it's the right tool for the right job at the right time and in the right place. That makes sense for you and especially for financial institutions. Um, you could look at the announcements around containers, the announcements around Amazon EKS, distro, Amazon EKS, anywhere, and then also Amazon ECS anywhere, which allows our customers to actually, uh, put AWS container technology anywhere they would like to put it. >>You could look also at the additions of the one you and two you form factors to outposts. So no longer do you have to do the, the, the large for you, uh, foreign factor for outposts, smaller outposts for smaller spaces, uh, that particular will play well in the financial service industry. You may not have necessarily as much room for a full cabinet. You could also look from the hybrid perspective in the announcement we made, um, around red hat OpenShift on AWS, all of are giving customers the ability to choose how they actually want to deploy, um, and pursue a hybrid. I'd also point to some announcements we made around management and governance in the financial services, industry governance, uh, is a very important topic. Uh, we announced the management and government lens for the AWS well architected, um, uh, program, uh, that is focused on breath practices for evolving governance for the cloud. >>It has recommended combination of AWS services integrations with our partner network and vetted reference architectures and guidance for addressing regulatory obligations as well. I'd also point to some things we made around audits. I was specifically in Steve Smith's, um, session today, he talked about AWS audit manager. That's a new tool for continually assessing areas and environments for controls or risk compliance. That includes prebuilt compliance frameworks for things like PCI DSS and GDPR, uh, two things that are very important in the financial services industry and last, but certainly not least I'd point to the announcement around the AWS audit Academy. This is training for auditors to actually be able to audit clouds from an agnostic perspective. Any cloud, not specifically AWS that's tree, uh, digital training to do that. And then also an instructor led course specifically on how to audit AWS. So some very key announcements, both from the standpoint of services, uh, as well as additional layers of helping customers in the financial services industry in regulated industries actually use our services. >>So typical, re-invent typical in a lot of news, a lot of announcements, the 500,000 Mark in terms of registering. I hadn't heard that. That's amazing. Let's talk that this has been an Andy. Jassy had an exclusive with John furrier just a couple of weeks ago before. I think it was last week, actually. And we've been talking about this acceleration of digital business transformation because of COVID we've been talking about it, the entire pandemic on the virtual cube, talking about how companies it's really about right now, surviving and thriving to be able to go forward and companies that haven't accelerated are probably in some trouble. Talk to me about how AWS has been working with your financial services customers to help them pivot and move to the cloud faster, really to not just help them survive now, but thrive in the long-term. >>Yeah. Immediately when COVID hit and it hit at different times in different, in different parts of the world. Immediately when COVID hit, we saw the conversation that we were having turning from, Hey, what's my digital strategy to immediately, what are my digital capabilities? And what that really means is what do I have the ability to do tomorrow? Because tomorrow is going to really matter. I don't have necessarily the time to plan for the next several quarters or the next several years, what can I do tomorrow to, um, really, uh, support my, my own workforce and support my own customers and the obligations I have as a financial institution. The first thing we saw people do was to try and make sure that those who financial services work can work. You can look at the adoption of Amazon workspaces, as well as our, uh, Amazon connect, uh, call centers as a service. >>As two examples there at the RBL bank in India was able to move to Amazon workspaces in just 10 days to enable its teams to actually work remotely from home. When they couldn't come into the office, you can look at Barclays. Barclays is actually a presenter at re-invent this year. They'll have a session on how they use Amazon connect, which again is our call center as a service offering to enable 25,000 contacts and our agents to work from home when they can no longer work out of the, out of their traditional contact center. The second thing we saw a financial institutions joining was making sure that customer engagements could still be meaningful when digital was the only option, um, specifically here in the U S you could look at the work that each of us did with FinTech companies like biz two X or fins Zack, or BlueVine Stripe and cabbage in support of the care act in the U S you might remember that the cares act, um, hasn't provisions for funding for small businesses. >>This small business administration had a program called the paycheck protection program, and those organizations were active in providing funding, uh, to small businesses. Uh, through that program. I'll give you an example of cabbage cabbage had previously not been an SBA lender, um, but they were able to, in two weeks build a fully automated system for small businesses to access PPP funding using Amazon text track, to extract information from documentation that those folks submitted to get alone. That reduced approval times from multiple days to about a median of four hours to actually get approval, to get funding through the PPP program. And then just four months cabbage became the second largest PPP lender. They lent over $7 billion in funding, which was twice the amount of funding that they went last year in 2019 loans. So we were happy to support organizations like cabbage and those other FinTech companies, as they help small businesses in the U S get access to funding, uh, during this critical time. >>And as we know, as you said, critical time, but really life or death for a lot of businesses. And as we continue to go through these ways, but it's interesting that you talked about that the speed of facilitation that during such unprecedented times, AWS and this massive machine was able to continue moving at full speed ahead and helping those customers to pivot. You talked about the cloud connect. I had a conversation with a guest on the queue last week about that. And, and I now think about if I have to call in a contact center and that person might be from home. So, you know, we're fortunate that the cloud computing technology and people like you and AWS, or are able to power that because it's, it's literally essential, which is probably one of the words of the year, but being able to keep the machinery going and innovate at the same time has been, make or break for a lot of businesses. >>Absolutely. And you, you look at, you know, kind of one of the last year is that I'll point to is, um, financial institutions. Uh, anti-virus, we're were very much focused on making sure that that cannot fail, that they scaled. And so you can look at the work we did with, uh, with the, with FINRA FINRA is the primary capital markets regulator here in the U S and on a daily basis frame or processes about 400 billion market events on every night to do surveillance on our markets, that when COVID hit, we had unprecedented volume and volatility in the market. And FINRA was, was, um, looking at processing, uh, anywhere from two to three times, their normal daily market volumes that's anywhere from 800 billion market events to 1.2 trillion a night. And if you look at how they were able to scale, they're actually able to scale up compute resources in AWS. We're on a nightly basis. They're able to automatically turn on and off up to a hundred thousand compute nodes in a single day. That automatic ability to scale is, is the power you're talking about. Being able to actually turn things up when you needed it and turn things down when you, when you don't need it based on the volumes. >>Well, and that's going to be something key going forward. As we know that there will be one thing I think that I always say we can count on right now is uncertainty and continued uncertainty, but we've also seen I'm calling them COVID catalysts. You know, the, what you talked about with cabbage, for example, and how that business pivoted quickly, because of the power of cloud computing and emerging technologies, what are some of the things that you think as we go into 2021 in the financial services arena, what are some of the big tech trends that you think were maybe born during COVID that are going to be critical going forward? >>Well, you know, you, you, you had Melanie Frank from capital one on cube a couple of days ago, and she was talking about, you know, their shift to cloud and what that's really enabled, and it, and she kind of sums it up nicely. She says, look, we want to give our customers experience that are real time, and that are intelligent. And you just can't do that with legacy technology. That's sitting in, you know, kind of a legacy data center. And so I think that's going to be kind of the, the, the all encompassing statement for what's happening in the financial services industry. As I mentioned, you know, organizations overnight said, okay, wait a minute, let's take that strategy. And then let's put it aside. Let's talk about capabilities. What can we do? And I think, you know, necessity is the mother of invention. Um, and when you're faced with limitations and challenges, like we all have been faced with around the world and not just in the financial services industry, it, it breeds, um, invention and the, and the desire and the need to actually meet those challenges head on, in very engineered of ways. >>And I think you're going to see more invention and specifically more invention from the established players in the financial services industry. Cloud use is not just experimental on the edges anymore. You're going to see more organizations coming out of COVID. Um, having had those experiences where they actually stood up a context center and scaled it. And, and just a matter of a few days to, to thousands of agents, you're going to find, um, organizations saying, wait a minute, we, we can do remote work. We could, we have access to things like Amazon workspaces. So I think you're, you're gonna, you're going to see that, uh, be a, be a trend. I think you're also gonna see, um, w what Lori beer said in the keynote with Andy, you know, she, she made a very, very astute statement, and I don't know if people caught it, cause it's kind of neat in the middle of her conversation. >>She said, look, we're trying to infuse analytics into everything that we do at JP Morgan. I think you're going to see more and more financial institutions looking to do that, to actually leverage the power of analytics, to power everything we do as a financial institution. So I think those, those are a couple of things that you're going to see. Um, and then, you know, looking, uh, you know, kind of around the corner, I think you're going to continue to see more re-invention within the industry. And what I mean by that is you've seen many financial institutions over the last week, uh, with, uh, re-invent making announcements, you saw bank and we towel saying, Hey, look, we are completely transforming ourselves with AWS. Uh, just a few weeks before we even saw standard charter, the same thing HSBC said, the same thing, global payments earlier in the year said the same thing. And you're going to see more and more organizations coming out and talking about these strategic decisions to reinvent everything that they do to make the financial systems of the world work. And so we're really pleased to be partnering with those organizations to make those transformations possible. We're seeing a lot of invention within the industry, and we're very pleased to be a part of the reinvention of the financial systems around the world. >>It's interesting to hear that you, you see, even the JP Morgan, some of those legacy, big houses are going to be really pivoting. They have to, to be competitive and to be able to utilize analytics, to deliver those real-time services. Because as we all know, as consumers, our patients is wearing thin these days, but I agree with you. I think there's a lot of opportunity there that innovation is exciting and there will have to be reinvention of entire industries, but I think there's a lot of silver linings there. Scott. I wish we had more time, cause I know we could keep talking, but thank you for sharing your insights on this reinvented reinvent this year. >>I appreciate it. Thank you, Lisa. It's always a pleasure to be on the cube. >>Chris Scott Mullins, I'm Lisa Martin. You're watching the cubes coverage of AWS reinvent 2020.

>> Introducer: From the CUBE studios in Palo Alto and in Boston connecting with thought leaders all around the world. This is a CUBE Conversation. >> Hi everybody this is Dave Volante. Welcome to the special CUBE Conversation. I'm really excited to invite in my mentor and friend. We go way back. Fred Moore is here. He's the president of Horizon Information Strategies. We going to talk about managing data in the zettabyte era. Fred, I think when we first met, we were talking about like the megabyte era. >> Right, exactly. I think back then we had, you know, maybe 10 bytes in our telephone and one on the wristwatch, you know, but now you can put a whole data center in a single cartridge of tape and take off. Things that really changed. >> It's pretty amazing. And of course, for those who don't know Fred, he was the first a systems engineer at Storage Tech. And as I said, somebody who taught me a lot in my early days, of course he's very famous for the term that everybody uses today. Backup is one thing, recovery is everything. And Fred just wrote, you know, this fantastic paper. He's done this year after year after year. He's just dug in, he's a clear thinker, strategic planner with a technical bent in a business bent. You're like one of those five tool baseball players, Fred. But tell me about this paper. Why, did you write it? >> Well, the reason I wrote that is there's been so much focus in the last year or so on the archive component of the storage hierarchy. And the thing that's happening, we're generating data lots faster than we're analyzing it. So it's piling up being unanalyzed and sitting basically untapped for years at a time. So that has posed a big challenge for people. The other thing that got me deeper into this last year was the Hyperscale market. They are, those people are so big in terms of footprint and infrastructure that they can no longer keep everything on disk. It's just economically not possible. The energy consumption per disk, the infrastructure costs, the frequency of, you know, taking a disc out every three, four or five years for just for replacement, has made it very difficult to do that. So Hyperscale has gone to tape in a big way, and it's kind of where most of the tape business in the future is going to wind up in these Hyperscale businesses. >> Right. >> We know tape doesn't exist in the home. It doesn't exist in a small data center. It's only a large scale data center technology, but that whole cosmos led me into the archive space and in a need for a new archive technology beyond tape. >> So, I want to set up the premise here. Just going to pull this out of your paper. It says a 60% of all data is archival, and could reach 80% or more by 2024, making archival data by far the largest storage class. And given this trajectory, the traditional storage hierarchy paradigm is going to to need to disrupt itself. And quickly we're going to talk about that. That really is the premise of your paper here, isn't it? >> It is, you know, to do all this with traditional technologies is going to get very painful for a variety of reasons. So the stage is set for a new tier and a new technology to appear in the next five years. Fortunately, I'm actually working with somebody who is after this in a big way, and in a different way than what you and I know. So I think there is some hope here that we can redefine and really add a new tier down at the bottom. You see it kind of emerging on that picture of the deep archive tier it's. Beginning to show up now and it's, you know, infinite storage. I mean, if you look at major league sports, the world series and Superbowl, you know, that data will never be deleted. It'll be here forever. It'll be used periodically based on circumstances. >> Yeah, well, we've got that pyramid chart up here. I mean, you invented this chart, essentially. At least you were the first person that ever showed it to me. I honestly think that you first created this concept where you had a high performance tier, and a high cost per bit, and then an archive tier. Maybe it wasn't this granular, you know, back in the '70s and '80s? But it's constantly been changing with different media types and different use cases. >> You know, you're right. I mean, and you all know this because you know, when storage deck introduced the nearline architecture, nearline set in between online and offline storage, we called it nearline, and trademarked that term. So that was the tape library concept to move data from offline status to online status, with a robotic library. So that brought up that third tier online, nearline, and offline, but you're right. This pyramid has evolved and morphed into several things. And, you know, I keep it alive. Somebody said, I'll have a pyramid on my tombstone instead of my name when I go down. (both chuckles) But it's really the heart and soul of the infrastructure for data. And then out of this comes all the management and security, the deletion, the immutable storage concepts, the whole thing starts here. So it's like your house, you got to have a foundation, then you can build everything on top of it. >> Well, and as you pointed out in your paper, a minute ago, it always comes down to economics. So I want to bring up the sort of 10 year expected cost of ownership the TCO for the three levels you got all disk, you got all cloud and you got LTO and you got the different aspects of the cost. The purple is always the biggest piece of cost. It's the labor costs. But of course, you know, in cloud, you've got the big media cost because they've done so much automation. I wonder if you could take us through this slide, what are the key takeaways there? >> Well, you know the thing that hurts here with all these technologies is, as you can see up on top up there, what the key issues are with this and the staff and personnel. So the less people you have to manage data, the better off you are. And then, you know, it's pretty high for disk compared to a lot of things to do on desk, but lack of manage a lot of, you know, sadly what you and I had to deal with years ago and provision kind of, I mean, a lot of this stuff is just labor intensive. The further you get, the further down the pyramid and you also get less labor intensive storage. And that helps then you get a lower cost for energy and cost of ownership. The TCO thing is kind of taking on a new meaning. I hate to put up a TCO chart in some regards, because it's all based on what your input variables are. So you can decide something different, but we've tried to normalize all kinds of pricing and come up with everything. And the cloud is a big question for most people as to how does it stack up. And if you don't ever touch the data in the cloud, you know, the price comes way down. If you want to start moving data in and out of the cloud, you're going to have to ante up in a big way like that. But, you know we're going to see dollar a terabyte storage prices down at the bottom of this pyramid here in the next five years. But hey, you can get down to four or five terabyte with drives media in libraries tape, just entire flash and certainly higher than that. But you know, we're going to have the race to a dollar a terabyte, total TCO cost here in 2025. >> So when Amazon announced, they just announced a glacier. Everybody said, okay, what is that? Is that tape is that, you know, this spun down disk, cause it took a while to get it back. But you're kind of seeing that tape technology as you said, really move into the Hyperscale space and that's going to accommodate this massive, you know, lower part of the pyramid, isn't it? >> Exactly. Yeah. And we don't have a spin down disk solution today. I was actually on the board of a company that started that called Copay and years ago, right up here near Boulder. >> You watch him (both chuckles) You absolutely right. And a few other people that, you know also, but the spin down disk never made it. And you know, you can spin up and down on a desk on your desktop computer, but doing that in a data center, then on a fiber channel drive never made it. So we don't have a spin down disk to do that. The archive space is kind of dominated by very high capacity disc and then tape. And most of the archive data in the world today, unfortunately sits on display. It's not used and spinning seven by 24, three 65 and not touch much. So that's a bad economic move, but customers just found that easier to handle by doing that then going back to tape. So we've got a lot of data stored in the wrong place from a total economics point of view. >> But the Hyperscalers are solving this problem, or they're not through automation. And, you know, you referenced storage, tiering, really trying to take the labor cost out. How are they doing? Are they doing a good job? >> They've done really well taking the labor costs down, I mean, they have optimized every screw, nut and bolt in the 42 chassis that you could imagine to make it as clean as possible to do that. So they've done a whole lot to bring that cost down, but still the magnitude of these data centers, we're going to finish the year 2020 with about 570 Hyperscale data centers. So it's going right now around the world. You know, each one of these things is 350 400,000 square feet, and up of race wars space. And the economics just don't allow you to keep putting inactive data on spinning disk. We don't have to spin down disk, tape You know, I feel like the only guy in the industry that says this sometimes, but, you know, tapes had a, you know, a renaissance. That people don't appreciate in terms of reliability, throughput, you know, tapes three orders of reliability higher than disc right now. And most people don't know this. So tape's viable, the Hyperscalers see that. And read one Hyperscalers or you know, by over a million pieces of LTO tape last year alone. Just to handle this, you know, be the pressure valve to take all of this inactive stuff off of the gigantic disc farms that they have. >> Well, so let's talk about that a little bit. So you just try to keep it simple. You've got, you know, flash disk and tape. It feels like disc is getting squeezed. We know what flash has done in terms of eating into disc. And you see in that, in the storage market generally, it's soft right now. And I've posited that a lot of that is the headroom that data centers have with flash, is they don't have to buy spindles anymore for performance reasons. And the market is soft. Only pure is showing consistent growth, and ends up a little bit, cause because of mainframe, you've got Dell popping back and forth, but generally speaking, the primary storage market is not a great place to be right now, all the actions and sort of secondary storage and data protection. And so just going to get squeezed, and you mentioned tape, you said that if your only person talking about it, but you said in your paper, you know, it's sequential. So time to first bite is, is sometimes problematic, but you can front end a tape with cash. You can use algorithms and, you know, smart scans and to really address that problem. And dramatically lower the cost. Plus you could do things like you tell me Fred, you're the technologists here, but you're going to have multiple heads things that you can't necessarily do in a hermetically sealed disc drive. >> (chuckles) You can. And what you just described is called the active archive layer in the pyramid. So when you front end a tape library with a disk array for a cash buffer, you create an active archive and that data will sit in there three or four or five days before it gets demoted based on inactivity. So, you know for repetitive use and you're going to get dislike performance for tape data, and that's the same cash in concept that deserve systems had 30 years ago. So that does work and the active archive has got a lot of momentum right now. There's right here near me, where I live in Boulder. We have the Active Archive Alliances headquarters, and I get to do their annual report every year. And this whole active archives thing is a big way to make and overcome that time, the first bike problem that we've had in tape. And we'll have for quite a while. >> In your paper, you've talked about some of the use cases and workloads and you laid out, you know basically taking the pyramid and saying, okay based on the workload, some certain percentage should be up at the top of the pyramid for the high performance stuff. And of course lower for the, you know, the less, you know, important traditional workloads, et cetera. And it was striking to see the Delta between annual, the highest performance we had 70% , I think was up in the top of the pyramid versus, you know the last use case. So in you're talking about what it costs to store a zettabyte in services is that if I talk about 108 million at the high end versus a about 11 or 12 million, so huge Delta 10 X Delta between the top and the bottom based on those, you know allocations based on the workload. >> Yeah, I tried to get at the value of tiered storage based on your individual workload in your business. So I looked at five different workloads, the top one that you referenced. That was in there at 108 million, you know, is the HPC market. I mean, when I visited a few of the HPC people, you know, their DOD agencies in many cases, you know that and I threw the pyramid up. The first thing they would say our permanents inverted. You know (chuckles), all of our archive data is about 10%. You know, we were all flash as much as we can. And we have a little bit archived, we're in constant. Simulation and compute mode and producing results like crazy from the data. So we do an IO, bring in maybe a whole file at a time and compute for minutes before we come up with an answer. So just the reverse. And then I got to look into all the different workloads talking to people, and that's how we develop these profiles. >> So let's pull up this future of the storage hierarchy, was again kind of of talks to the premise of your paper. Walk us through this like, what changes should we be expecting, and you got air gap in here. We're going to, I'm going to ask you about remastering and lifespan, but take us through this. >> Yeah, you know, the traditional chart that you had up on the first big year had four tiers, you know, two disturbs and solid state at the top. And then the big archive tier, which is kind of everything falling down into tape at this point. But you know again, tape has some challenges. You know time to first bite and sequential access on. And then when we couple using tape or disc as an archive, most of that data that's archival is captured as unstructured data. So we don't have, we don't have tags, we don't have metadata, we don't have indices, and that has led to the movement for object storage, to be a primary, maybe in the next five years, the primary format in store archived data, because it's got all that information inside of it. So now we have a way to search things and we can get to objects, but in the interim, you know, it's hard to find and search out things that are unstructured and, you know, most estimates would say 80% of the world's data is at least that much is unstructured. So archives are hard to find once you store it, there's one storing is one thing, retrieving it is another thing. And that's led to the formation of another layer in the story tier. It's going to be data that doesn't have to be remastered or converted to a new technology. in the case of the disc, every three, four or five years or tape drive every eight, maybe 10 years take large lost. Kate Media can go 30 years, but with all new modern tape media, but unfortunately, you know, the underlying drive doesn't go back that far, you can't support that many different versions. So the media life is actually longer than it needs to be. So the stage is set for a new technology to appear down here to deal with this archives. So it'll have faster access will not need to be remastered every five or 10 years, but you'll have, you know, a 50 year life in here. And I believe me, I've been looking for a long time to be able find something like this. And, you know we have a shot at this now, and I'm actually working with the technology that could pull this off. >> Well, it's interesting also as well, you calling out the air gap and the chart we go back to our mainframe guesses, is not a lot we haven't seen before, you know, maybe data D duplication, but you know, the adversary has become a lot more sophisticated. And so air gaps and, you know, ransomware on everybody's mind today, but you've sort of highlighted three layers of the pyramid that are actually candidates for that air gapping. >> Yeah. The active archive up there, of course, you know, with the disk and tape combined, then just pure tape. And then this new technology, which can be removable. You know, when you have removability you create an air gap. little did we know when you and I met that removability would be important to take. We thought we were trying to get rid of the Chevy truck access method, and now without electricity with a terrorist attack and pandemic or whatever. The fastest way to move data is put it on a truck and get it out of town. So that has got renewed life right now. Removability much to my shock from where we started. >> You talked about remastering and you said it's a costly labor intensive process that typically migrates previously archived data to new media every five to 10 years. First of all, explain why you have to do that and how a data center operators can solve that problem. >> Yeah. And let's start with data where most of it sits today on described, you know it describes useful life is four to five years before it either fails or is replaced. That's pretty much common now. So then they have to start replacing these things. And that means you have to copy, you know, read the data off the disk and write it somewhere else, big data move. And as the years go by that amount of data to revamp or gets bigger and bigger. So, I mean, you can do the math as you well know, you want to move, you know, 50 petabytes of data. It's going to take several weeks to do that electronically. So this gets to be a real time consuming effort. So most data centers that I've seen will keep about one fifth of their disposal every year migrating to a new technology, just kind of rolling forward as they go like that rather than do the whole thing every five years. So that's the new build in the disc world. And then for tape the drive stay in there longer, you know the LTO family drives a good read. You know two generations back from the current one that's been there. They cut that off a year ago. They'll go back to something like this soon. But you know, you can go into 10 years on a tape drive. The media life because of very unfair right media, which was already oxidized the last 30 years or more. The old media metal particle was not oxidized. So, you know, the oxidized flake, the particles would fall off people will say shit. I've had this in here eight years, you know, and it's kind flake it I put it back in. So that didn't work well. But now that we had various Verite Media, it was all oxidized, the media lives skyrocket. So that was the whole trick with tape to get into something that was preoxidized before time could cause it to decay. So the remastering is a lot, is less on tape by two to one to three to one, but still when you've got petabytes, maybe an exabyte sitting on tape in the future, that's going to take a long time to do that. >> Right. >> So remastering you'd love a way to scale capacity without having to continue to move the data to something new ever so often. >> So my last question is you've , you know, you went from a technical role into a strategic planning role, which of course the more technical you are in that role, the better off you're going to be. You don't understand that the guardrails, but you've always had a sort of telescope in the industry and you close the paper and it's kind of where I want to end here on, you know, what's ahead. And you talk about some of the technologies that obviously have legs, like three D NAND and obviously magnetic storage. You got optical in here, but then you've got all these other ones that you even mentioned, you know, don't hold your breath waiting for these multilayer photonics and dedic DNA. What class media, holographic storage, quantum storage we do a lot about quantum. What should we be thinking about and expecting as observers as to, you know, new technologies that might drive some innovation in the storage business? >> Well, I've listed the ones that are in the lab that have any life at all, right on this paper. So, you know can kind of take your pick at what goes on there. I mean, optical disk has not made it in the data center. We talked about it for 35 years. We invested in it in storage deck and never saw the light of day. You know, optical disk has remained an entertainment technology throughout the last 35 years. And the bigger rate is very low compared to data center technology. So, you know optical would have to take a huge step going forward. We got a lot of legs left in the solid state business. That's really active SSB, the whole nonvolatile memory spaces. Probably not 45% of the total disc shipments in terms of units, from what it was at it's high and in 2010. Unbelievable though. You know, in disc shipment 650 million drives a year announced just under 400, 35,400. So flashes has taken this stuff away, like crazy. Tape shouldn't be taking just away, but the tape industry doesn't do a very effective job of marketing itself. Most people still don't know what's going on with tape. They're still looking out of the roof, still looking out of the rear view mirror at a tape, as opposed to the front windshield. We see all the new things that have happened. So, you know they have bad memories of taping the past load stretch, edge damage tape, wouldn't work a tear or anything like that. It was a problem. Oh, that's pretty well gone away now. In a moderate tape is a whole different ball game, but most people don't know that. So, you know tapes going to have to struggle with access time and sequential reality. They've done a few things to come over excess time and the order request now to take the optimizer based on physical movement on the tape that can take out 50% of your access time for multiple requests on a cartridge. The one on here that's got the most promise right now would be a version of a multilayer photonic storage, which is. I would say sort like optical, but, you know, with data center, class characteristics, multi-layer recording capability on that random access, which tape doesn't have. And, you know, I would say that's probably the one that you would want to take some look at going forward like this. The others are highly specular. You know, we've been talking about DNA since we were kids. So we don't have a DNA product out here yet. You know, it's access times eight hours. It's probably not going to work for us. That's your, that's not your deep archive anymore. That's your time capsule storage. >> Yeah, right. >> Lock the earth. So, I mean, I think you kind of see what's here. I mean, the chances are it's still going to be the magnetic technologies tape disc, and then the solid state number and stuff. >> Right. >> But these are the ones that I'm tracking and looking at, trying to have worked with a few of the companies that are in this. Future list and I'd love to see something breakthrough out there, but it's like, we've always said about a holographic storage. For example, you know, there's been more written about it than there's ever been written on it. (both chuckles) >> Well, the paper's called Reinventing Archival Storage. You can get it on your website I presume Fredhorizon.com >> Yep, absolutely. >> Awesome. >> Fred Moore, great to see you again. Thanks so much for coming on the CUBE. >> My pleasure, Dave. Thanks a lot. Great job. >> All right. And thank you for watching everybody. This is Dave Volante for the CUBE. We'll see you next time. (upbeat music)