>>Yeah, yeah, hello again and welcome to our last session of the day before we head to the meat. The experts roundtables how T Mobile is building a data driven organization with thought spot and whip prone. Today we'll hear how T Mobile is leaving Excel hell by enabling all employees with self service analytics so they can get instant answers on curated data. We're lucky to be closing off the day with these two speakers. Evo Benzema, manager of business intelligence services at T Mobile Netherlands, and Sanjeev Chowed Hurry, lead architect AT T Mobile, Netherlands, from Whip Chrome. Thank you both very much for being with us today, for today's session will cover how mobile telco markets have specific dynamics and what it waas that T Mobile was facing. We'll also go over the Fox spot and whip pro solution and how they address T mobile challenges. Lastly, but not least, of course, we'll cover Team Mobil's experience and learnings and takeaways that you can use in your business without further ado Evo, take us away. >>Thank you very much. Well, let's first talk a little bit about T Mobile, Netherlands. We are part off the larger deutsche Telekom Group that ISS operating in Europe and the US We are the second largest mobile phone company in the Netherlands, and we offer the full suite awful services that you expect mobile landline in A in an interactive TV. And of course, Broadbent. Um so this is what the Mobile is appreciation at at the moment, a little bit about myself. I'm already 11 years at T Mobile, which is we part being part of the furniture. In the meantime, I started out at the front line service desk employee, and that's essentially first time I came into a touch with data, and what I found is that I did not have any possibility of myself to track my performance. Eso I build something myself and here I saw that this need was there because really quickly, roughly 2020 off my employer colleagues were using us as well. This was a little bit where my efficient came from that people need to have access to data across the organization. Um, currently, after 11 years running the BR Services Department on, I'm driving this transformation now to create a data driven organization with a heavy customer focus. Our big goal. Our vision is that within two years, 8% of all our employees use data on a day to day basis to make their decisions and to improve their decision. So over, tuition Chief. Now, thank >>you. Uh, something about the proof. So we prize a global I T and business process consulting and delivery company. Uh, we have a comprehensive portfolio of services with presents, but in 61 countries and maybe 1000 plus customers. As we're speaking with Donald, keep customers Region Point of view. We primary look to help our customers in reinventing the business models with digital first approach. That's how we look at our our customers toe move to digitalization as much as possible as early as possible. Talking about myself. Oh, I have little over two decades of experience in the intelligence and tell cope landscape. Calico Industries. I have worked with most of the telcos totally of in us in India and in Europe is well now I have well known cream feed on brownfield implementation off their house on big it up platforms. At present, I'm actively working with seminal data transform initiative mentioned by evil, and we are actively participating in defining the logical and physical footprint for future architectures for criminal. I understand we are also, in addition, taking care off and two and ownership off off projects, deliveries on operations, back to you >>so a little bit over about the general telco market dynamics. It's very saturated market. Everybody has mobile phones already. It's the growth is mostly gone, and what you see is that we have a lot of trouble around customer brand loyalty. People switch around from provider to provider quite easily, and new customers are quite expensive. So our focus is always to make customer loyal and to keep them in the company. And this is where the opportunities are as well. If we increase the retention of customers or reduce what we say turned. This is where the big potential is for around to use of data, and we should not do this by only offering this to the C suite or the directors or the mark managers data. But this needs to be happening toe all employees so that they can use this to really help these customers and and services customers is situated. This that we can create his loyalty and then This is where data comes in as a big opportunity going forward. Yeah. So what are these challenges, though? What we're facing two uses the data. And this is, uh, these air massive over our big. At least let's put it like that is we have a lot of data. We create around four billion new record today in our current platforms. The problem is not everybody can use or access this data. You need quite some technical expertise to add it, or they are pre calculated into mawr aggregated dashboard. So if you have a specific question, uh, somebody on the it side on the buy side should have already prepared something so that you can get this answer. So we have a huge back lock off questions and data answers that currently we cannot answer on. People are limited because they need technical expertise to use this data. These are the challenges we're trying to solve going forward. >>Uh, so the challenge we see in the current landscape is T mobile as a civil mentioned number two telco in Europe and then actually in Netherlands. And then we have a lot of acquisitions coming in tow of the landscape. So overall complexity off technical stack increases year by year and acquisition by acquisition it put this way. So we at this time we're talking about Claudia Irureta in for Matic Uh, aws and many other a complex silo systems. We actually are integrated where we see multiple. In some cases, the data silos are also duplicated. So the challenge here is how do we look into this data? How do we present this data to business and still ensure that Ah, mhm Kelsey of the data is reliable. So in this project, what we looked at is we curated that around 10% off the data of us and made it ready for business to look at too hot spot. And this also basically help us not looking at the A larger part of the data all together in one shot. What's is going to step by step with manageable set of data, obviously manages the time also and get control on cost has. >>So what did we actually do and how we did? Did we do it? And what are we going to do going forward? Why did we chose to spot and what are we measuring to see if we're successful is is very simply, Some stuff I already alluded to is usual adoption. This needs to be a tool that is useable by everybody. Eso This is adoption. The user experience is a major key to to focus on at the beginning. Uh, but lastly, and this is just also cold hard. Fact is, it needs to save time. It needs to be faster. It needs to be smarter than the way we used to do it. So we focused first on setting up the environment with our most used and known data set within the company. The data set that is used already on the daily basis by a large group. We know what it's how it works. We know how it acts on this is what we decided to make available fire talksport this cut down the time around, uh, data modeling a lot because we had this already done so we could go right away into training users to start using this data, and this is already going on very successfully. We have now 40 heavily engaged users. We go went life less than a month ago, and we see very successful feedback on user experience. We had either yesterday, even a beautiful example off loading a new data set and and giving access to user that did not have a training for talk sport or did not know what thoughts, what Waas. And we didn't in our he was actively using this data set by building its own pin boards and asking questions already. And this shows a little bit the speed off delivery we can have with this without, um, much investments on data modeling, because that's part was already done. So our second stage is a little bit more ambitious, and this is making sure that all this information, all our information, is available for frontline uh, employees. So a customer service but also chills employees that they can have data specifically for them that make them their life easier. So this is performance KP ice. But it could also be the beautiful word that everybody always uses customer Terry, 60 fuse. But this is giving the power off, asking questions and getting answers quickly to everybody in the company. That's the big stage two after that, and this is going forward a little bit further in the future and we are not completely there yet, is we also want Thio. Really? After we set up the government's properly give the power to add your own data to our curated data sets that that's when you've talked about. And then with that, we really hope that Oh, our ambition and our plan is to bring this really to more than 800 users on a daily basis to for uses on a daily basis across our company. So this is not for only marketing or only technology or only one segment. This is really an application that we want to set in our into system that works for everybody. And this is our ambition that we will work through in these three, uh, steps. So what did we learn so far? And and Sanjeev, please out here as well, But one I already said, this is no which, which data set you start. This is something. Start with something. You know, start with something that has a wide appeal to more than one use case and make sure that you make this decision. Don't ask somebody else. You know what your company needs? The best you should be in the driver seat off this decision. And this is I would be saying really the big one because this will enable you to kickstart this really quickly going forward. Um, second, wellness and this is why we introduce are also here together is don't do this alone. Do this together with, uh I t do this together with security. Do this together with business to tackle all these little things that you don't think about yourself. Maybe security, governance, network connections and stuff like that. Make sure that you do this as a company and don't try to do this on your own, because there's also again it's removes. Is so much obstacles going forward? Um, lastly, I want to mention is make sure that you measure your success and this is people in the data domain sometimes forget to measure themselves. Way can make sure everybody else, but we forget ourselves. But really try to figure out what makes its successful for you. And we use adoption percentages, usual experience, surveys and and really calculations about time saved. We have some rough calculations that we can calculate changes thio monetary value, and this will save us millions in years. by just automating time that is now used on, uh, now to taken by people on manual work. So, do you have any to adhere? A swell You, Susan, You? >>Yeah. So I'll just pick on what you want to mention about. Partner goes live with I t and other functions. But that is a very keating, because from my point of view, you see if you can see that the data very nice and data quality is also very clear. If we have data preparing at the right level, ready to be consumed, and data quality is taken, care off this feel 30 less challenges. Uh, when the user comes and questioned the gator, those are the things which has traded Quiz it we should be sure about before we expose the data to the Children. When you're confident about your data, you are confident that the user will also get the right numbers they're looking for and the number they have. Their mind matches with what they see on the screen. And that's where you see there. >>Yeah, and that that that again helps that adoption, and that makes it so powerful. So I fully agree. >>Thank you. Eva and Sanjeev. This is the picture perfect example of how a thought spot can get up and running, even in a large, complex organization like T Mobile and Sanjay. Thank you for sharing your experience on how whip rose system integration expertise paved the way for Evo and team to realize value quickly. Alright, everyone's favorite part. Let's get to some questions. Evil will start with you. How have your skill? Data experts reacted to thought spot Is it Onley non technical people that seem to be using the tool or is it broader than that? You may be on. >>Yes, of course, that happens in the digital environment. Now this. This is an interesting question because I was a little bit afraid off the direction off our data experts and are technically skilled people that know how to work in our fight and sequel on all these things. But here I saw a lot of enthusiasm for the tool itself and and from two sides, either to use it themselves because they see it's a very easy way Thio get to data themselves, but also especially that they see this as a benefit, that it frees them up from? Well, let's say mundane questions they get every day. And and this is especially I got pleasantly surprised with their reaction on that. And I think maybe you can also say something. How? That on the i t site that was experienced. >>Well, uh, yeah, from park department of you, As you mentioned, it is changing the way business is looking at. The data, if you ask me, have taken out talkto data rather than looking at it. Uh, it is making the interactivity that that's a keyword. But I see that the gap between the technical and function folks is also diminishing, if I may say so over a period of time, because the technical folks now would be able to work with functional teams on the depth and coverage of the data, rather than making it available and looking at the technical side off it. So now they can have a a fair discussion with the functional teams on. Okay, these are refute. Other things you can look at because I know this data is available can make it usable for you, especially the time it takes for the I t. G. When graduate dashboard, Uh, that time can we utilize toe improve the quality and reliability of the data? That's yeah. See the value coming. So if you ask me to me, I see the technical people moving towards more of a technical functional role. Tools such as >>That's great. I love that saying now we can talk to data instead of just looking at it. Um Alright, Evo, I think that will finish up with one last question for you that I think you probably could speak. Thio. Given your experience, we've seen that some organizations worry about providing access to data for everyone. How do you make sure that everyone gets the same answer? >>Yes. The big data Girlfriends question thesis What I like so much about that the platform is completely online. Everything it happens online and everything is terrible. Which means, uh, in the good old days, people will do something on their laptop. Beirut at a logic to it, they were aggregated and then they put it in a power point and they will share it. But nobody knew how this happened because it all happened offline. With this approach, everything is transparent. I'm a big I love the word transparency in this. Everything is available for everybody. So you will not have a discussion anymore. About how did you get to this number or how did you get to this? So the question off getting two different answers to the same question is removed because everything happens. Transparency, online, transparent, online. And this is what I think, actually, make that question moot. Asl Long as you don't start exporting this to an offline environment to do your own thing, you are completely controlling, complete transparent. And this is why I love to share options, for example and on this is something I would really keep focusing on. Keep it online, keep it visible, keep it traceable. And there, actually, this problem then stops existing. >>Thank you, Evelyn. Cindy, That was awesome. And thank you to >>all of our presenters. I appreciate your time so much. I hope all of you at home enjoyed that as much as I did. I know a lot of you did. I was watching the chat. You know who you are. I don't think that I'm just a little bit in awe and completely inspired by where we are from a technological perspective, even outside of thoughts about it feels like we're finally at a time where we can capitalize on the promise that cloud and big data made to us so long ago. I loved getting to see Anna and James describe how you can maximize the investment both in time and money that you've already made by moving your data into a performance cloud data warehouse. It was cool to see that doubled down on with the session, with AWS seeing a direct query on Red Shift. And even with something that's has so much scale like TV shows and genres combining all of that being able to search right there Evo in Sanjiv Wow. I mean being able to combine all of those different analytics tools being able to free up these analysts who could do much more important and impactful work than just making dashboards and giving self service analytics to so many different employees. That's incredible. And then, of course, from our experts on the panel, I just think it's so fascinating to see how experts that came from industries like finance or consulting, where they saw the imperative that you needed to move to thes third party data sets enriching and organizations data. So thank you to everyone. It was fascinating. I appreciate everybody at home joining us to We're not quite done yet. Though. I'm happy to say that we after this have the product roadmap session and that we are also then going to move into hearing and being able to ask directly our speakers today and meet the expert session. So please join us for that. We'll see you there. Thank you so much again. It was really a pleasure having you.

>> Narrator: From New York City It's theCUBE covering New Relic FutureStack 2019. Brought to you by New Relic. >> Hi, I'm Stu Minimen and we're here at New Relic's FutureStack 2019 at the Grand Hyatt, next to Grand Central Station, here in New York City. Happy to welcome to the program a first time guest, Roger Scott who's the Chief Customer Officer at New Relic. Roger, thanks so much for joining us. >> Thanks, Stu. Thanks for having me on. Good to be here. >> Alright so, I love this morning actually in addition to hearing all of the announcements, my first hand full of guests on theCUBE were customers. So I got to hear from them and we know your team is always excited about the announcements, but definitely enthusiasm from the customers, things in the keynote that got people. >> Fired up! Yeah. >> Clapping, and fired up. >> Great to see. >> Things like, oh wait! 10 terabytes of data, pressure thing, refresh for like a second, and >>oh my gosh! There's results. Yeah >> Pretty impressive so maybe give us a little bit of insight into customer engagement and how it's let to the bevy of announcements here at the show. >> Oh it's a great question actually and I think in my capacity as Chief Customer Officer and the functions I'm responsible for, we're continually engaging with customers as you can imagine. And one of the things we take a lot of pride in is being a proxy for the voice of the customer back into the organization. So we have a pretty rigid process. Not rigid, a pretty discipline process, I would argue, that allows us to get feedback from the field, listen to our customers, understand what's important to them, and reflect that in our product roadmap. And I'll let you know that's on a weekly cadence we do that. Now we're not doing that in a reactive fashion such that our roadmap diverts every single week in there, but we hear that constant feedback from the field as to what our customers are lacking. So lot of what you hear today, in terms of those six great announcements that we have were a combination of feedback that we've had over the last couple of years, I would argue. Because it's a dramatic shift to go from what we were previously, which was essentially six individual products that work really well together. But through the release of New Relic 1 in May earlier this year and what we announced today has truly developed us in to a observability platform. So monitoring with six different products to a true observably platform that's open, connected and programmable is a dramatic shift. And that's a combination of a bunch of feedback from our customers over the years. >> Yeah. I'm sure it's pretty much feedback from all customers. They're not asking for more tools and more interfaces and more things that they need to learn. >> Roger: Not at all, right. >> In many ways software can be a unifying feature especially that term platform who spend a bunch of time emphasizing what's needed from platform. >> Maybe, what were your costumers struggling with that kind of New Relic 1 in general is looking to solve as well as the observability piece? What went into that launch that was costumer pinpoints and things that they'd been asking for. >> Yeah maybe to stand back a little bit and understand some of the challenges that costumers had and then why they were asking for different solutions or evolution of our solution. If you think about today's world, there's this rapid development an deployment of software, so it's almost got to the point of continuous software deployment. And so your speed of needing to be able to react to problems in your environment, your costumer experience are degrading, ect. Being able to respond to that really quickly is essential, understanding the costumer experience is essential. You talked about operational efficiency of reducing the number of tooling sets or data sets that I'm looking at continually. So anything that we could provide to our costumers that allowed them to get to answers quicker, understand the why, and then be able to remediate that really easily so that the costumers have a greater experience. And at the same time reduces this friction that's unnecessarily introduced when you're going from one product to another, one tool to another and you're spending too much time rationalizing data sets across those tool sets. So consolidation is a big theme, ability to get to your answers really quickly is a big theme and that's really been the genesis of being able to create a platform. But not just a platform for consolidation, for better visibility, and observability but we believe it's not truly a platform until you can develop on it. If you think back in technology history of all the different peradams we've had throughout the history of technology, those who've won the platform wars over the years have been really good at being able to provide tools and ease of adoption of the platform by virtue of being able to build things on top of it. The ability to give people tools that allow them to build technology is really a therasense of the platform as well. >> You know, Roger, there's a certain trust level that costumers have to have if they're going to be building on top of your platform. >> When I've talked to costumers in New Relic they do talk about a partnership >> and the good back and forth but there's definitely a certain amount of stickiness once they've built something on your platform. >> Roger: Right, yeah. >> Any concerns from them as to, you know there's that term lock in out there as to the how do I know that this is going to work for me, and that I'm not going to have my pricing kind of crank up over time and be like oh my gosh, a year or two later, what did I get myself into? >> Right. It's a really important point that I'd like to start off by actually reemphasizing the point you made. I think we pride ourselves on the relationship we have with our costumers. It truly is the heart of everything at my organization does. We have this saying that we are because they are. In the realization that if we don't serve our costumers really well they have choices frequently, we're a saas vendor, the contracts come up for renewal frequently. And if you're unable to deliver on the promises that you made in the sales process, once they implement your solutions and try to use those in production, environments and everyday work if you can't deliver on those promises then you're going to breakdown that level of trust. And trust is at the center of all relationships as you know. Whether it's a personal relationship, you're playing on a sports team, whether you're working with your costumers. And so we want to make sure that we can deliver on those promises once we've sold them the product. So I haven't heard any specific concerns about lock in or anything, I think what they regularly come to us though with is they want us to have a really strong point of view, want us to be opinionated, tell them how this should work effectively together, what does best practice look like, what's the gold standard, what are some of the artifacts, tools, frameworks, reusable templates that we can share with them that accelerates their time to value. So I think the value significantly outweighs the concerns around lock in or reduction of the number of vendors that they're working with. >> If I look at really the enterprise space, you've got costumers working through their application modernization. They've got their modelist their going after micro services. I heard a stat that only about five to ten percent of apps are monitored at the app level today. >> Yeah, pretty scary, isn't it? >> Yeah, how many of your costumers are dealing with the installed state versus new deployments and what are some of the challenges you're hearing from costumers there? >> Yeah and I think it's important to pause that number because I think it's five to ten percent or growing to twenty percent as I think got indicated. If you look at those organizations Born In The Cloud or Born Digital it's significantly higher percentage of that which is possibly an indictment of the low level of instrumentation we see in a lot of legacy software technology stacks. And so I think in today's world we're tryna get that level of instrumentation observability up as much as possible. But maybe to link back to your previous question as well I think there's an important aspect here of when we move to a platform. When you're a product company your differentiation comes through product, comes through the capability of that product features and functions and we've certainly found ourselves in a significant number of those battles against competition where it's feature and function based. That's not a great comfort for the costumer. I think when you move to a platform it's very much around the networks differentiation. When I say network differentiation I think it's about getting the users of your service access to third party applications to third party data sources be they open source data emitters, opentelementry, open sensors, Zipkin any of those data sets that we are now in support for today. Giving them access to those data sets and being able to enrich the experience that we provide them that network effects and that's really where we see the opportunity to deliver significantly more value to our costumers with the ability to then build your own applications on top of the platform. That's second to none in the industry in my opinion. >> Roger, what's New Relic's role in helping costumers as really they're modernizing their work force? When I talk to so many companies it's like they need to retrain and they have to have new skill sets they need to make sure as certain cloud in automation changes where they focus on things and embrace devops and new ways of doing things. There are a lot of challenges there. Where does New Relic play in that modernization for costumers? >> You know what I think it's in a couple ways. The ways that we, my organization, can help the costumer in terms of just sheer understanding of the capability of the platform, what are best practices, how we can drive better accountability as you move to these new technology stacks and new ways of working much more agile environments. And so I think we can do a combination of that just sheer skills development, working really tightly with the likes of AWS you would've heard Dave McCann this morning talking about how when costumers migrate the application work goes to the AWS cloud environment. Hopefully they're not just doing that by way of compute lift and shift but they were actually looking at modernizing and refactoring those applications and when they do that, you heard Dave talk through a number of assets and frameworks and models and reusable best practices that we're trying to work with them on that we can give to our costumers that accelerate their journey 'cause it's not easy. We were talking to Chris Dillon this morning from Cox Automotive and when you think of an organization like that that's forty, fifty years old and has had to transform itself in terms of digital experience for it's costumer base, it's a significant cultural adjustment quite often to get teams to work in fundamentally different ways. So it's not an insignificant challenge but that's partly why we've invested so heavily in costumer success. Taking the costumers on the journey, thinking about their maturity over time, and constantly look for them to get better value from the platform. >> Roger, there are a number of things that have jumped out at me. Things like oh hey, we can save you potentially millions of dollars on your AWS cloud bill. You've already got costumers building on top of the platform, you had the future Haka event just a couple of weeks ago. Any other kind of interesting or exemplary costumer outcomes that you might be able to share? Either doesn't have to be about the new stuff but just that you've recently with your costumers. >> You know, one of the things that's most gratifying for me when talking to costumers is when we've been able to see when you work with older, more traditional companies that are undergoing some form of digital transformation and they're trying to shift a lot of the applications into a more modern stack and environment, become more agile, etc. they frequently sort of peel off part of the business and will have a digital division that will build some innovative, typically mobile based, apps. We've seen a number of different retailers that we've worked with. Number of different travel organizations where we've started out intrumenting the mobile application because they've built a new application to give their consumers or costumers access through to their services, and at some point that application is going to merge into the backend and have to connect back into older technology. And it's been the beauty of being able to connect those two different environments together. Not starting off at what we would've got as slightly easier place to start which was the more modern application environment where we are really well suited to. But then seeing the full value of being able to instrument the front end all the way through to the backend, link that back to the costumer's experience and to the impact on the business in terms of funnel analysis from number of people using the mobile application to actually ordering something to once they've ordered it, feeling satisfied in actually receiving the goods that they ordered. Being able to instrument all of that and understand the impact of performance and availability on the overall business arcam, that's when it's been truly transformational in working with costumers and that's certainly where we'd love to help more of our costumers in that fashion. >> Alright, Roger, want to give you the final word. Of course you bring together a number of costumers here at FutureStack in the U.S as well there's a few of those run in other geographical areas but throughout the year, any other key things you want to highlight as to how costumers can get engaged even more. >> Yeah, I mean, we've got a sort of what I would argue is a tiered approach to costumer success. At the very high end of our engagement model we have a significant number of resources. Solution architects, costumer success managers that we can deploy directly with our costumers. We typically do that in conjunction with them, build out success plans, etc. What we looking at investing Heavily at the moment is also having a good understanding of what the ideal costumer journey is like. Realizing that a costumer can come to an event like this and learn about our product but the best way for them to experience that is in the course of using the product. So heavy focus on product lead growth and how we actually deliver better value through the product itself, remove friction and adoption and getting to better value. We want to automate some of that costumer journey so that we know that if you've just signed up and, for instance, you've configured you're agent and you've done your learning policy but you haven't yet configured a custom apdex on that application or you haven't understood what your key transactions are, we've got all that data in the backend. So we're working really hard to understand how we get that information back out to costumers and go hey we know you haven't necessarily done this yet, here's some access to great assets. A short video clip, a self paced learn guide that somebody can get on demand from an LMS system. So trying to use a combination of direct resource investment, events like this where it's great to make announcements like we did about the six grade innovations and then increasingly using digital through the products but also through just the general costumer journey to say hey this is really important content and information, you should look at this now 'cause it's going to add value in what you're doing today. >> Alright, well Roger Scott, Chief Customer Officer at New Relic, thanks so much for joining us. >> Thanks so much, it's been great talking to you. >> All right. I'm Stu Minimen back with lots more here at New Relic FutureStack 2019 in New York City. Thanks for watching theCUBE. (outro music)