(upbeat music) >> Hi, today I'm going to talk about network analytics and what that means for telecommunications as we go forward, thinking about 5G, what the impact that's likely to have on network analytics and the data requirement, not just to run the network and to understand the network a little bit better, but also to inform the rest of the operation of the telecommunications business. So as we think about where we are in terms of network analytics and what that is over the last 20 years, the telecommunications industry has evolved its management infrastructure to abstract away from some of the specific technologies in the network. So what do we mean by that? Well, in the, when initial telecommunications networks were designed there were management systems that were built in. Eventually fault management systems, assurance systems, provisioning systems, and so on, were abstracted away. So it didn't matter what network technology had, whether it was a Nokia technology or Erickson technology or Huawei technology or whoever it happened to be. You could just look at your fault management system and understand where faults were happened. As we got into the last sort of 10, 15 years or so telecommunication service providers become, became more sophisticated in terms of their approach to data analytics and specifically network analytics and started asking questions about why and what if in relation to their network performance and network behavior. And so network analytics as a sort of an independent functioning was born and over time more and more data began to get loaded into the network analytics function. So today just about every carrier in the world has a network analytics function that deals with vast quantities of data in big data environments that are now being migrated to the cloud. As all telecommunications carriers are migrating as many IT workloads as possible to the cloud. So what are the things that are happening as we migrate to the cloud that drive enhancements in use cases and enhancements in scale in telecommunications network analytics? Well, 5G is the big thing, right? So 5G, it's not just another G in that sense. I mean, in some cases, in some senses it is 5G means greater bandwidth and lower latency and all those good things. So, you know, we can watch YouTube videos with less interference and, and less sluggish bandwidth and so on and so forth. But 5G is really about the enterprise and enterprise services transformation. 5G is a more secure kind of a network, but 5G is also a more pervasive network. 5G has a fundamentally different network topology than previous generations. So there's going to be more masts. And that means that you can have more pervasive connectivity. So things like IOT and edge applications, autonomous car, current smart cities, these kinds of things are all much better served because you've got more masts, that of course means that you're going to have a lot more data as well and we'll get to that. The second piece is immersive digital services. So with more masts, with more connectivity, with lower latency, with higher bandwidth with the potential is immense for services innovation. And we don't know what those services are going to be. We know that technologies like augmented reality, virtual reality, things like this have great potential, but we have yet to see where those commercial applications are going to be, but the innovation and the innovation potential for 5G is phenomenal. It certainly means that we're going to have a lot more edge devices. And that again is going to lead to an increase in the amount of data that we have available. And then the idea of pervasive connectivity when it comes to smart cities, autonomous cars, integrated traffic management systems, all of this kind of stuff, those kind of smart environments thrive where you've got this kind of pervasive connectivity, this persistent connection to the network. Again, that's going to drive more innovation. And again, because you've got these new connected devices, you're going to get even more data. So this rise, this exponential rise in data is really what's driving the change in network analytics. And there are four major vectors that are driving this increase in data in terms of both volume and in terms of speed. So the first thing is more physical elements. So we said already that 5G networks are going to have a different topology. 5G networks will have more devices, more masts. And so with more physical elements in the network, you're going to get more physical data coming off those physical networks. And so that needs to be aggregated and collected and managed and stored and analyzed and understood so that we can have a better understanding as to, you know, why things happened the way they do, why the network behaves in which they do in ways that it does and why devices that are connected to the network and ultimately of course, consumers, whether they be enterprises or retail customers behave in the way they do in relation to their interaction with the network. Edge nodes and devices. We're going to have an explosion in terms of the number of devices. We've already seen IOT devices with your different kinds of trackers and sensors that are hanging off the edge of the network, whether it's to make buildings smarter or car smart or people smarter in terms of having the measurements and the connectivity and all that sort of stuff. So the numbers of devices on the edge and beyond the edge are going to be phenomenal. One of the things that we've been trying to wrestle with as an industry over the last few years is where does a telco network end and where does the enterprise, or even the consumer network begin? It used to be very clear that, you know, the telco network ended at the router but now it's not that clear anymore because in the enterprise space, particularly with virtualized networking, which we're going to talk about in a second, you start to see end to end network services being deployed. And so are they being those services in some instances that are being managed by the service provider themselves, and in some cases by the enterprise client. Again, the line between where the telco network ends and where the enterprise or the consumer network begins is not clear. So those edge, the proliferation of devices at the edge, in terms of, you know, what those devices are, what the data yield is and what the policies are that need to govern those devices in terms of security and privacy and things like that, that's all going to be really, really important. Virtualized services, we just touched on that briefly. One of the big, big trends that's happening right now is not just the shift of IT operations onto the cloud, but the shift of the network onto the cloud, the virtualization of network infrastructure. And that has two major impacts. First of all, it means that you've got the agility and all of the scale benefits that you get from migrating workloads to the cloud, the elasticity and the growth and all that sort of stuff, but arguably more importantly for the telco it means that with a virtualized network infrastructure, you can offer entire networks to enterprise clients. So, you know, selling to a government department, for example, who's looking to stand up a system for, you know, certification of, you know, export certification, something like that. You can not just sell them the connectivity, but you can sell them the networking and the infrastructure in order to serve that entire end to end application. You could send, you could offer them in theory, an entire end-to-end communications network. And with 5G network slicing, they can even have their own little piece of the 5G bandwidth that's been allocated to gets a carrier and have a complete end to end environment. So the kinds of services that can be offered by telcos given virtualize network infrastructure are many and varied and it's an outstanding opportunity. But what it also means is that the number of network elements virtualized in this case is also exploding. And that means the amount of data that we're getting on, informing us as to how those network elements are behaving, how they're performing is going to go up as well. And then finally, AI complexity. So on the demand inside while historically network analytics, big data has been driven by returns in terms of data monetization, whether that's through cost avoidance or service assurance, or even revenue generation through data monetization and things like that. AI is transforming telecommunications and every other industry. The potential for autonomous operations is extremely attractive. And so understanding how the end-to-end telecommunication service delivery infrastructure works is essential as a training ground for AI models that can help to automate a huge amount of telecommunications operating processes. So the AI demand for data is just going through the roof. And so all of these things combined to mean that big data is getting explosive. It is absolutely going through the roof. So that's a huge thing that's happening. So as telecommunications companies around the world are looking at their network analytics infrastructure, which was initially designed for service insurance primarily and how they migrate that to the cloud. These things are impacting on those decisions because you're not just looking at migrating a workload to operate in the cloud that used to work in the data center. Now you're looking at migrating a workload but also expanding the use cases in that workload. And bear in mind many of those workloads are going to need to remain on-prem. So they'll need to be within a private cloud or at best a hybrid cloud environment in order to satisfy regulatory jurisdiction requirements. So let's talk about an example. So LG Uplus is fantastic service provider in Korea, huge growth in that business over the last, over the last 10, 15 years or so. And obviously most people would be familiar with LG, the electronics brand, maybe less so with, with LG Uplus, but they've been doing phenomenal work and were the first business in the world to launch commercial 5G in 2019. And so a huge milestone that they achieved. And at the same time they deployed the Network Real-time Analytics Platform or NRAP from a combination of Cloudera and our partner Caremark . Now, there were a number of things that were driving the requirement for the analytics platform at the time. Clearly the 5G launch was the big thing that they had in mind, but there were other things that were at play as well. So within the 5G launch, they were looking for a visibility of services and service assurance and service quality. So, you know, what services have been launched? How are they being taken up? What are the issues that are arising? Where the faults happening? Where are the problems? Because clearly when you launch a new service like that you want to understand and be on top of the issues as they arise. So that was really, really important. A second piece was and, you know, this is not a new story to any telco in the world, right? But there are silos in operation. And so it taking advantage of, or eliminating redundancies through the process of digital transformation it was really important. And so particular, the two silos between wired and the wireless sides of the business needed to come together so that there would be an integrated network management system for LG Uplus as they rolled out 5G. So eliminating redundancy and driving cost savings through the integration of the silos was really, really important. And that's a process and the people think every bit, as much as it is a systems and a data thing so another big driver. And the fourth one, you know, we've talked a little bit about some of these things, right? 5G brings huge opportunity for enterprise services innovation. So industry 4.0, digital experience, these kinds of use cases were very important in the South Korean market and in the business of LG Uplus And so looking at AI and how can you apply AI to network management? Again, there's a number of use cases, really, really exciting use cases that have gone live now in LG Uplus since we did this initial deployment and they're making fantastic strides there. Big data analytics for users across LG Uplus, right? So it's not just for, it's not just for the immediate application of 5G or the support or the 5G network, but also for other data analysts and data scientists across the LG Uplus business. Network analytics while primarily it's, it's primary use case is around network management. LG Uplus or network analytics has applications across the entire business, right? So, you know, for customer churn or next best offer for understanding customer experience and customer behavior really important there for digital advertising, for product innovation, all sorts of different use cases and departments within the business needed access to this information. So collaboration sharing across the network, the real-time network analytics platform it was very important. And then finally, as I mentioned, LG group is much bigger than just LG Uplus. It's got the electronics and other pieces, and they had launched a major group wide digital transformation program in 2019. And so being a part of that was important as well. Some of the seems that they were looking to address. So first of all, the integration of wired and wireless data sources, and so getting your assurance data sources, your network data sources and so on integration was really, really important. Scale was massive for them. You know, they're talking about billions of transactions in under a minute being processed and hundreds of terabytes per day. So, you know, phenomenal scale that needed to be, you know, available out of the box as it were. Real time indicators and alarms. And there was lots of KPIs and thresholds set that, you know, to make, made to meet certain criteria, certain standards. Customer specific real time analysis of 5G, services particularly for the launch, root cause analysis and AI based prediction on service anomalies and service issues was a core use case. As I talked about already the provision of service, of data services across the organization. And then support for 5G, served the business service impact was extremely important. So it's not just understand, well, you know, that you have an outreach in a particular network element, but what is the impact on the business of LG Uplus, but also what is the impact on the business of the customer from an outage or an anomaly or a problem on the network. So being able to answer those kinds of questions really, really important too. And as I said between Cloudera and Caremark and LG Uplus they have already, themselves an intrinsic part of the solution, this is what we ended up building. So a big, complicated architecture side. I really don't want to go into too much detail here. You can see these things for yourself, but let me skip through it really quickly. So, first of all, the key data sources. You have all of your wireless network information, other data sources, this is really important 'cause sometimes you kind of skip over this. There are other systems that are in place like the enterprise data warehouse that needed to be integrated as well. Southbound and northbound interfaces. So we get our data, yo know, from the network and so on and network management applications through both file interfaces, Kafka, NiFi are important technologies. And also the RDBMS systems that, you know, like the enterprise data warehouse that we're able to feed that into the system. And then northbound, you know, we spoke already about making network analytics services available across the enterprise. So, you know, having both a file and API interface available for other systems and other consumers across the enterprise is very important. Lots of stuff going on then in the platform itself. Two petabytes and persistent storage, Cloudera HDFS, 300 nodes for the raw data storage and then Kudu for real time storage for, you know, real-time indicator analysis around generation and other real time processes. So there was the core of the solution spark processes for ETL, key quality indicators and alarming, and also a bunch of work done around data preparation, data generation for transferal to, for party systems through the northbound interfaces. Impala API queries for real-time systems there on the right hand side and then a whole bunch of clustering classification, prediction jobs through the ML processes, the machine learning processes. Again, another key use case, and we've done a bunch of work on that, and I encourage you to have a look at the Cloudera website for more detail on some of the work that we did here. Some pretty cool stuff. And then finally, just the upstream services, some of these, there's lots more than simply these ones, but service assurance is really, really important so SQM, CEM and ACD right to the service quality management customer experience autonomous control is really, really important consumers of the real-time analytics platform and your conventional service assurance functions like faulted performance management. These things are as much consumers of the information and the network analytics platform as they are providers of data to the network analytics platform. So some of the specific use cases that have been stood up and that are delivering value to this day and there's lots of more besides, but these are just three that we pulled out. So, first of all, sort of specific monitoring and customer quality analysis care and response. So again, growing from the initial 5G launch, and then broadening into broader services, understanding where there are issues so that when people complain, when people have an issue that we can answer the concerns of the client in a substantive way. AI functions around root cause analysis understanding why things went wrong when they went wrong and also making recommendations as to how to avoid those occurrences in the future. So, you know, what preventative measures can be taken. And then finally, the collaboration function across LG Uplus extremely important and continues to be important to this day where data is shared throughout the enterprise, through the API Lira, through file interfaces and other things and through interface integrations with upstream systems. So that's kind of the real quick run through of LG Uplus. And the numbers are just staggering. You know, we've seen upwards of a billion transactions in under 40 seconds being tested. And we've gone through beyond those thresholds now already, and we're start, and this isn't just a theoretical sort of a benchmarking test or something like that. We're seeing these kinds of volumes of data and not too far down the track. So with those things that I mentioned earlier or with the proliferation of network infrastructure in the 5G context with virtualized elements, with all of these other bits and pieces are driving massive volumes of data towards the network analytics platform. So phenomenal scale. This is just one example. We work with service providers all over the world is over 80% of the top 100 telecommunication service providers run on Cloudera. They use Cloudera in the network and we're seeing those customers all migrating Legacy Cloudera platforms now onto CDP onto the Cloudera Data Platform. They're increasing the jobs that they do. So it's not just warehousing, not just ingestion of ETL and moving into things like machine learning. And also looking at new data sources from places like NW DAF the network data analytics function in 5G or the management and orchestration layer in software defined network function virtualization. So, you know, new use cases coming in all the time, new data sources coming in all the time, growth in the application scope from, as we say, from edge to AI. And so it's really exciting to see how the footprint is growing and how the applications in telecommunications are really making a difference in facilitating network transformation. And that's covering, that's me covered for today. I hope you found that helpful. By all means please reach out. There's a couple of links here. You can follow me on Twitter. You can connect to the telecommunications page. Reach out to me directly at Cloudera I'd love to answer your questions and talk to you about how big data is transforming networks and how network transformation is accelerating telcos throughout the world.