>> Live from Boston, it's theCUBE. Covering IBM Chief Data Officer Summit. Brought to you by IBM. >> Welcome back everyone to theCUBE's live coverage of the IBM Chief Data Summit here in Boston, Massachusetts. I'm your host, Rebecca Knight along with my co-host Paul Gillin. We're joined by Ash Dhupar, he is the Chief Analytics Officer at Publishers Clearing House. Thank you so much for coming on theCUBE. >> Thank you Rebecca for calling me here. >> So Publishers Clearing House is a billion-dollar company. We think of it as the sweepstakes company, we think of the giant checks and be the Prize Patrol surprising contestants, but it's a whole lot more than that. Tell our viewers a little bit, just explain all the vast amount of businesses that you're in. >> Sure, so, in a nutshell, we are a media and entertainment company with a large base of customers, about 100 million customers who are motivated with the chance to win. That's the sweepstakes angle to it. And we have, you can categorize the business into two buckets. One is our media and entertainment side, which is the publishing side. And then the other is our retail side which is where we sell merchandise to our customers. Think of us as a catalog and an e-commerce company. On the media and entertainment side, we have a very good engagement with our customers, we get about two billion page views on a monthly basis on our website. We, about 15 million unique customers on a monthly basis are coming to the site and they spend a considerable amount of time with us on an average, anywhere between 12 to 15 minutes, depending on, you know the type of the customers. Some of our very heavily-engaged customers can spend as much as about two hours a day with us. (Rebecca and John laughs) >> Trying to win that, that either the big prize or there are small prizes like, if you go on our site, there's a winner everyday, like there could be 1,000 dollar winner everyday playing a certain type of a game. So that's the media and the entertainment side of our business, that's completely ad-supported. And then we are the retail side of the business is we are in direct mail, so the traditional, we would send someone a direct mail package. And an e-commerce company as well. Just as a small nugget of information, we are. We send almost about 400 million pieces of physical mail which is including our packages that are sent and so on and so forth and though also still a large direct mail company. Still profitable and still growing. >> I'm sure the US Postal Service is grateful for your support. (laughs) They need all the help they can get. You collect, essentially, the prize money, is your cost of data acquisition and you have a huge database you told us earlier before we started filming of about 100 million people, that you have data on just in the US alone. Now what are you doing at the upper limits of what you're able to do with this data. How are you using this strategically other than just you know personalized email? >> Sure, so I think using data is a core asset for us. We are utilizing in giving our customers better experiences by utilizing the data we have on them. Marrying it with other data sources as well. So that we can personalize the experience. So that we can make your experience when you come on the site better. Or if we are sending something to you in mail, we give you products that are relevant to you. So to bring it down to a little more tactical level, in case of when you are on our site, then on our e-commerce site, there's a product recommendation engine, right? Which goes in and recommends products to you on what products to buy. Those product recommendation engines drive a significant amount of sales, almost about 40% of our sales are driven by the prior recommendation engines that is all understanding of the customer, what you're buying, what you're likely to buy and the algorithms behind it are built with that. >> Can you give another example though, of how, if I were, I mean you said all these customers are united by a common desire to win and to play a game and to win. >> Right. >> But what are some other ways beyond product recommendation engines, which are now sort of old hat. >> Right. >> What other ways are you enhancing the customers experience and personalizing it? >> Sure, sure. So, I'll give you a recent example of where we are utilizing some of the data to give a more relevant experience to the customer. So when a customer comes on our website, right when you're coming to register with us. So, as you register, as you fill in the form, after you give your name, address and your email address and you hit submit, at that very second, there are some algorithms that are running behind the scenes to understand how are you likely to engage with us. How are you going to, let's say, because we have a diverse business, are you likely to buy something from us? Or are you not likely to buy something from us? And if you're not likely to buy something from us, which means I can get you to, and you know not waste your time in showing you merchandise, but I can give you an experience of free-to-play games and you can, within free-to-play games, what type of games like understanding the persona of the person. We could say, hey, you probably are a lotto player or you are a word game puzzle player and we could give you and direct you to those experiences that are more relevant to you. In case of, if you're going to buy something from us, are you likely to buy, you know highly likely to buy or less likely to buy. Depending on that, should I show you just 10 or 15 products or should I show you like more than that? Are you more likely to buy a magazine? So making it more relevant for the customer experience is where it is all about. We use a lot of this data to, to make that happen. >> So analytics is really core to your business. It's the, completely strategic. Where do you sit in the organization, organizational layout, how is that reflected in the way your job is integrated into the organization? >> Sure, so, it is, I'm part of the C-Suite. And I think our CEO, he had this vision, thing he started. He loves data first of all. (laughs) >> Lucky for you. (laughs) >> Thank you. And he truly believes that data and analytics can drive growth and bring innovation from different areas if we utilize it in the best possible way. So A, I am part of that team. And work very closely with each of the business owners. That's the key, out here is like you know, it is, analytics is not in one corner but in the center of all the, all the business areas giving them either insights or building algorithms for them so that we can make either better decisions or we can power growth, depending on which way we are looking at it. >> You're the Chief Analytics Officer and we're here at the Chief Data Summit here, of here. How different are the roles in your mind and do they work together? I mean you have a CTO that is responsible for sort of Chief Data Officer. >> Yes. >> Responsibilities. How do you two collaborate and work together? >> It is a very tight collaboration. And they're two separate jobs but it is a very tight collaboration, we work hand in hand with each other. And the best part I would say is that you know, we're all focused and we're all driving towards how can we drive growth? That's the bottom line, that is where the bucks stops for all of us in the companies. Are we building projects? Are we doing things that is going to grow the company or not? So the collaboration with the CTO is A, a critical piece. They own the infrastructure, as well as the data and when you own the data, which is, in a way, is slightly, I would say, data governance I would say is a thankless job (laughs) believe it or not. But it is a critical job. It is if your data is not right, it is not going to work for whatever you're trying to do, it's the garbage in garbage out, we all know about that. And we work very closely. If there are CAPEX proposals that needs to be put in place because we're going after a certain big project, whether it's putting things together in one place or a 360 view of the customer. All of that is worked hand in hand. We work together in working towards that. >> What is your big data infrastructure like? Is it on the Cloud? Is it your own? Are you Adobe based? What do you use? >> All of the above. >> Oh. (laughter) No, so, what we have is because we are such an old company, you know we still have our legacy Db2 infrastructure. A lot of our backend databases, lot of our backend processes are all attached to that. We have a warehouse, a sequel server warehouse. We also, for our web analytics, we use Google's BigQuery. That's where you collect a lot of data on a daily basis. And recently, I think about three years ago, we went into the Cloud environment. We have a map, our cluster, which was cloud-based and now, we have brought in on prem very recently. >> Back from the Cloud. >> Back from the Cloud, on prem. And there was very good reasoning why we did that. I think frankly, it's cheaper on a longer term to bring that on prem and you are a lot more in control with all the issues with data privacy. So it is. >> Which, I hope you don't mind my interrupting but we have to wrap here and I need to get that question in. (laughs) >> Yes. >> You have data on 100 million consumers. What are you doing with all of the attention being paid for privacy right now? What are you doing to ensure the. >> We have a very, very I would say integrated infrastructure, data governance, data. There's a whole slew of, I would say, people and process around that to make sure that our date is not exposed. Now luckily, it's it's not like PII to the level that it's a health care data. So you are not really, you have information that is crazy but you still have the PII, the name and address of these customers. And as an example, none of the PII data is actually available to even to the analytics folks. It's all stripped, the PII's stripped off. You give us an ID to the customer and frankly the analytics team don't need the PII information to build any algorithms as well. So there is a whole process around keeping the data secure. >> Great, well Ash, thank you so much for coming on theCUBE, it was a pleasure having you. >> Thank you and thank you for inviting me. >> I'm Rebecca Knight for Paul Gillin. We will have more from IBM CDO Summit just after this. (techno music)

(upbeat music) >> Live from Boston. It's theCUBE. Covering IBM Chief Data Officer's Summit. Brought to you by IBM. >> Welcome back to theCUBE's live coverage of IBM's CDO here in beautiful Boston, Massachusetts. I'm your host, Rebecca Knight, along with my co-host Paul Gillen. We're joined by Beth Rudden. She is distinguished engineer, analytics at IBM. Thanks so much for coming on theCUBE, Beth. >> Thank you. >> So your background, you have a master's in anthropology and in classics and Greek. You're a former archeologist. And now here you are, distinguished engineer. There's only 672 in all of IBM. How did you, what are you doing here? (laughs) >> I think that I love data. I love that data represents human behavior and I think that understanding that puzzle and being able to tell that story is something that we need to do more of. And we need to understand how all of the data fits together and how all of the information is created and all the wisdom is created. And that takes a lot of effort from a lot of people and it involves storytelling. I think that 75,000 years of human history, we are always understanding conflict and resolution through storytelling. And I think that if we can have evidence for that using our data, and looking at our data in the business world, as it reflects our strategy, instead of force-fitting our data into our strategy. So I think that that's part of the change that we need to look at it, and I think with the, I would say, second or third hype curve of AI and what we're doing with AI and cognitive today, it really is being able to bind philosophy and psychology and look at it from a computer science perspective. And that's new. >> That's interesting. And we were just speaking with Inderpal earlier about the background that he sees CDOs coming from, and he talked very little about technology. It was all about human so-called soft skills. Are you finding that the CDO role is evolving in a less technical direction over time? >> Absolutely. And I think that, you know, when you're starting to look for outcomes, and as outcomes as they relate to our business, and as they relate to our clients and our customers, we have to be able to have a very diverse and inclusive viewpoint. And we have to be incredibly transparent. I think that is something that we are continuing to do within IBM, where we are really looking at how do we differentiate ourselves based on our expertise and based on our human capital. >> So it's differentiating yourself as an employer of choice. >> Absolutely. >> And attracting, recruiting or training the talent. >> 100%, yes. >> But then also being the expert that your clients come to. So yeah, just... >> So IBM has, you know, we have career frameworks. We have career paths. I was part of a team that created the data science profession at IBM and one of the things that we're looking at, as a differentiating feature, is that we really want people to continuously learn, continuously adapt, develop themselves, develop their skills, because that is our differentiating feature. And I think that, you know, when our clients meet our people they love our people. And this is such an amazing company to be a part of. We have a long history. 107 year history of being one of the most diverse and inclusive companies. 1899, we hired the first black and female person, and in 1953 we had equal opportunity rights 10 years before the Civil Rights Act. So I think that all of these things, you know, lead up to a company that shows that we can adapt and transform. And being an acting CDO for the largest IT system in IBM right now, we are doing amazing things because we are really investing in our people. We are investing in giving them that guidance, that career track. And allowing people to be themselves. Their true selves. >> You're speaking of Global Technology Services Operation, which is currently undergoing a transformation. >> That's right. >> What does the outcome look like? How do you envision the end point? >> I think that I envision the end point, we are in the process of developing our IBM Services Academy and it is a continuous learning platform for Bands Two through general managers. And, you know, one of the stories I like to tell is my general manager who I'm working for on this. You know, she believes so strongly in making sure that everybody has access to all of the available education and everybody is using that type of education and we are looking at transforming how we are measuring what we are doing to incentivize the behaviors that we want to see. And the behaviors that we are looking for are people who are helping other people, and making sure that we are continually being the premier leader of the intellectual brainchild of what we are doing for keeping us in the AI game, in the cognitive game, and making sure that we are understanding every single aspect of that as it relates to our transformation. >> So you're actually tracking and measuring how colleagues collaborate with each other. >> 100%. >> How do you do that? >> You look for words like we and team and you look for people who are enabling other people. And that's something that we can see in the data. Data are artifacts of human behavior. We can see that in our data. We are looking at unstructured data, we are looking at structured data. We are taking this in and we're taking it to what I would call a new level, so that we can see what we are doing, who our people are, and we are able to look at how many of the people are enabling other people and empowering other people. And sometimes this is called the glue, or glue work. I think that there is even a baseball reference for like a glue man. I think that we need to champion people who are enabling and empowering everybody to succeed. >> And are those typically the unsung heroes, would you say? >> Yes. 100%. And I want to sing the hero's song for those unsung heroes. And I want to make sure that those people are noticed and recognized, but I also want to make sure that people know that IBM is this amazing company with a very long history of making sure that we are singing the unsung hero's song. >> But how do you measure the outcome of that? There's got to be a big business bottom line benefit. What does that look like? >> Absolutely. I think that it always starts with our clients. Everything that we do starts with our clients. And in GTS, we have people, we have five to seven year relationships with our clients and customers. These are deep relationships, and they interact with our humans every single day. And we are the men and women who, you know, design and create and run and manage the foundational systems of the world. And every single person, like you cannot book an airline, you cannot pay your bill, you cannot do that, anything, without touching somebody in IBM. We are investing in those people, because those are who is interfacing with our clients and customers, and that is the most important thing to us right now. >> One of the things we were talking about earlier is bringing more women and underrepresented minorities and men into IBM, and into other industries too. So how, we know the technology industry has a very bad reputation, deservedly so, for being a bro culture. How are you personally combating it, and then how do you do it from an institutional perspective? >> Yeah. We have so many programs that are really looking at how we can take and champion diversity. I was very honored to walk into the Best of IBM with my husband a couple years ago, and he looks around and he goes, this is like a UN convention. He's like, you guys are so global. You have so much diversity. And you know, that viewpoint is something that, it's why I work for IBM. It's why I love IBM. I have the ability to understand different cultures, I have the ability to travel around the world. We have, you can work day and night. (laughs) You know, you can talk to India in morning and Australia in the afternoon. It is just, to me, you know, IBM operates in 172 different countries. We have the global infrastructure to be able to handle the type of global teams that we are building. >> When you look at the skills that will be needed in the future as organizations, big data becomes infused into the organization, how will the skill needs change? >> To me, I think that the skill needs are always going to continuously transform. We're always going to get new technology. Most of my data scientists, you know, I really push Python, I really push R, but I think that it's the will more than the skill. I think that it is how people have the attitude and how people collaborate. And that is more important, I think, than some of the skills. And a lot of people, you know, when they are performing data science or performing data engineering, they need to believe that they are doing something that is going to succeed. And that is will. And that's what we, we have seen a huge surge in oral and written communications which is not a hard skill. It's a soft skill. But to me, there's nothing soft about those skills. It takes courage and we have built resiliency because we have had the courage to really enable and empower people to get those types of skills. And that's a lot of where our education is going. >> So that's really an interesting point here. So are you hiring a self-selected group of people, or are you bringing in super smart people who maybe are not as skilled in those areas and bringing them into the culture. I mean, what's coming first here? >> Yeah. I think that we are, our culture is strong. IBM's brand is strong. Our culture is strong. We are investing in the people that we have. And we are investing in, you know, our humans in order to make sure that the people who already have that culture have the skills that they need in order to learn. And that understanding of going from disequilibrium to equilibrium to disequilibrium to learn, that's what we want to teach, so that any of the new technology, any of the new skills, or any of the new platforms that we need to learn, it's something that's inherent with people being able to learn how to learn. >> Beth, thank you so much for coming on theCUBE. >> Yes. >> It was great to have you. >> Thank you. >> I'm Rebecca Knight, for Paul Gillen, we will have more of theCUBE's live coverage of IBM's CDO coming up in just a little bit. (light music)

(light music) >> Live from Boston. It's theCUBE. Covering IBM Chief Data Officer Summit. Brought to you by IBM. >> Welcome back everyone, to theCUBE's live coverage of the IBM CDO Summit here in Boston, Massachusetts. I'm your host, Rebecca Night. And I'm joined by my co-host, Paul Gillen. We have two guests for this segment. We have Steven Eliuk, who is the Vice President of Deep Learning Global Chief Data Officer at IBM. And Christopher Bannocks, Group Chief Data Officer at IMG. Thanks so much for coming on theCUBE. >> My pleasure. >> Before we get started, Steve, I know you have some very important CUBE fans that you need-- >> I do. >> To give a shout out to. Please. >> For sure. So I missed them on the last three runs of CUBE, so I'd like to just shout out to Santiago, my son. Five years old. And the shortest one, which is Elana. Miss you guys tons and now you're on the air. (all laughing) >> Excellent. To get that important piece of business out. >> Absolutely. >> So, let's talk about Metadata. What's the problem with Metadata? >> The one problem, or the many (chuckles)? >> (laughing) There are a multitude of problems. >> How long ya got? The problem is, it's everywhere. And there's lots of it. And bringing context to that and understanding it from enterprise-wide perspective is a huge challenge. Just connecting to it finding it, or collecting centrally and then understanding the context and what it means. So, the standardization of it or the lack of standardization of it across the board. >> Yeah, it's incredibly challenging. Just the immense scale of metadata at the same time dealing with metadata as Chris mentioned. Just coming up with your own company's glossary of terms to describe your own data. It's kind of step one in the journey of making your data discoverable and governed. Alright, so it's challenging and it's not well understood and I think we're very early on in these stages of describing our data. >> Yeah. >> But we're getting there. Slowly but surely. >> And perhaps in that context it's not only the fact that it's everywhere but actually we've not created structural solutions in a consistent way across industries to be able to structure it and manage it in an appropriate way. >> So, help people do it better. What are some of the best practices for creating, managing metadata? >> Well you can look at diff, I mean, it's such a broad space you can look at different ones. Let's just take the work we do around describing our data and we do that for for the purposes of regulation. For the purposes of GDPR et cetera et cetera. It's really about discovering and providing context to the data that we have in the organization today. So, in that respect it's creating a catalog and making sure that we have the descriptions and the structures of the data that we manage and use in the organization and to give you perhaps a practical example when you have a data quality problem you need to know how to fix it. So, you store, so you create and structure metadata around well, where does it come from, first of all. So what's the journey it's taken to get to the point where you've identified that there's a problem. But also then, who do we go to to fix it? Where did it go wrong in the chain? And who's responsible for it? Those are very simple examples of the metadata around, the transformations the data might have come through to get to its heading point. The quality metrics associated with it. And then, the owner or the data steward that it has to be routed back to to get fixed. >> Now all of those are metadata elements >> All of those, yeah. >> Right? >> 'Cause we're not really talking about the data. The data might be a debit or a credit. Something very simple like that in banking terms. But actually it's got lots of other attributes associated with it which essentially describe that data. So, what is it? Who owns it? What are the data quality metrics? How do I know whether what it's quality is? >> So where do organizations make mistakes? Do they create too much metadata? Do they create poor, is it poorly labeled? Is it not federated? >> Yes. (all laughing) >> I think it's a mix of all of them. One of the things that you know Chris alluded to and you might of understood is that it's incredibly labor-intensive task. There's a lot of people involved. And when you get a lot of people involved in sadly a quite time-consuming, slightly boring job there's errors and there's problem. And that's data quality, that's GDPR, that's government owned entities, regulatory issues. Likewise, if you can't discover the data 'cause it's labeled wrong, that's potential insight that you've now lost. Because that data's not discoverable to a potential project that's looking for similar types of data. Alright, so, kind of step one is trying to scribe your metadata to the organization. Creating a taxonomy of metadata. And getting everybody on board to label that data whether it be short and long descriptions, having good tools et cetera. >> I mean look, the simple thing is... we struggle as... As a capability in any organization we struggle with these terms, right? Metadata, well ya know, if you're talking to the business they have no idea what you're talking about. You've already confused them the minute you mentioned meta. >> Hashtag. >> Yeah (laughs) >> It's a hashtag. >> That's basically what it is. >> Essentially what it is it's just data about data. It's the descriptive components that tell you what it is you're dealing with. If you just take a simple example from finance; An interest rate on it's own tells you nothing. It could be the interest rate on a savings account. It can the interest rate on a bond. But on its own you have no clue, what you're talking about. A maturity date, or a date in general. You have to provide the context. And that is it's relationships to other data and the contexts that it's in. But also the description of what it is you're looking at. And if that comes from two different systems in an organization, let's say one in Spain and one in France and you just receive a date. You don't know what you're looking at. You have not context of what you're looking at. And simply you have to have that context. So, you have to be able to label it there and then map it to a generic standard that you implement across the organization in order to create that control that you need in order to govern your data. >> Are there standards? I'm sorry Rebecca. >> Yes. >> Are there standards efforts underway industry standard why difference? >> There are open metadata standards that are underway and gaining great deal of traction. There are an internally use that you have to standardize anyway. Irrespective of what's happening across the industry. You don't have the time to wait for external standards to exist in order to make sure you standardize internally. >> Another difficult point is it can be region or country specific. >> Yeah. >> Right, so, it makes it incredibly challenging 'cause every region you might work in you might have to have a own sub-glossary of terms for that specific region. And you might have to control the export of certain data with certain terms between regions and between countries. It gets very very challenging. >> Yeah. And then somehow you have to connect to it all to be able to see what it all is because the usefulness of this is if one system calls exactly the same, maps to let's say date. And it's local definition of that is maturity date. Whereas someone else's map date to birthdate you know you've got a problem. You just know you've got a problem. And exposing the problem is part of the process. Understanding hey that mapping's wrong guys. >> So, where do you begin? If your mission is to transform your organization to be one that is data-centric and the business side is sort of eyes glazing over at the mention of metadata. What kind of communication needs to happen? What kind of teamwork, collaboration? >> So, I mean teamwork and collaboration are absolutely key. The communication takes time. Don't expect one blast of communication to solve the problem. It is going to take education and working with people to actually get 'em to realize the importance of things. And to do that you need to start something. Just the communication of the theory doesn't work. No one can ever connect to it. You have to have people who are working on the data for a reason that is business critical. And you need have them experience the problem to recognize that metadata is important. Until they experience the problem you don't get the right amount of traction. So you have to start small and grow. >> And you can use potentially the whip as well. Governance, the regulatory requirements that's a nice one to push things along. That's often helpful. >> It's helpful, but not necessarily popular. >> No, no. >> So you have to give-- >> Balance. >> We're always struggling with that balance. There's a lot of regulation that drives the need for this. But equally, that same regulation essentially drives all of the same needs that you need for analytics. For good measurement of the data. For growth of customers. For delivering better services to customers. All of these things are important. Just the web click information you have that's all essentially metadata. The way we interact with our clients online and through mobile. That's all metadata. So it's not all whip or stick. There's some real value that is in there as well. >> These would seem to be a domain that is ideal for automation. That through machine learning contextualization machines should be able to figure a lot of this stuff out. Am I wrong? >> No, absolutely right. And I think there's, we're working on proof of concepts to prove that case. And we have IBM AMG as well. The automatic metadata generation capability using machine learning and AI to be able to start to auto-generate some of this insight by using existing catalogs, et cetera et cetera. And we're starting to see real value through that. It's still very early days but I think we're really starting to see that one of the solutions can be machine learning and AI. For sure. >> I think there's various degrees of automation that will come in waves for the next, immediately right now we have certain degrees where we have a very small term set that is very high confidence predictions. But then you want to get specific to the specificity of a company which have 30,000 terms sometimes. Internally, we have 6,000 terms at IBM. And that level of specificity to have complete automation we're not there yet. But it's coming. It's a trial. >> It takes time because the machine is learning. And you have to give the machine enough inputs and gradually take time. Humans are involved as well. It's not about just throwing the machine at something and letting it churn. You have to have that human involvement. It takes time to have the machine continue to learn and grow and give it more terms. And give it more context. But over time I think we're going to see good results. >> I want to ask about that human-in-the-loop as IBM so often calls it. One of the things that Nander Paul Bendery was talking about is how the CDO needs to be a change engine in chief. So how are the rank and file interpreting this move to automation and increase in machine learning in their organizations? Is it accepted? It is (chuckles) it is a source of paranoia and worry? >> I think it's a mix. I think we're kind of blessed at least in the CDO at IBM, the global CDO. Is that everyone's kind of on board for that mission. That's what we're doing >> Right, right. >> There's team members 25, 30 years on IMBs roster and they're just as excited as I am and I've only been there for 16 months. But it kind of depends on the project too. Ones that have a high impact. Everyone's really gung ho because we've seen process times go from 90 days down to a couple of days. That's a huge reduction. And that's the governance regulatory aspects but more for us it's a little bit about we're looking for the linkage and availability of data. So that we can get more insights from that data and better outcomes for different types of enterprise use cases. >> And a more satisfying work day. >> Yeah it's fun. >> That's a key point. Much better to be involved in this than doing the job itself. The job of tagging and creating metadata associated with the vast number of data elements is very hard work. >> Yeah. >> It's very difficult. And it's much better to be working with machine learning to do it and dealing with the outliers or the exceptions than it is chugging through. Realistically it just doesn't scale. You can't do this across 30,000 elements in any meaningful way or a way that really makes sense from a financial perspective. So you really do need to be able to scale this quickly and machine learning is the way to do it. >> Have you found a way to make data governance fun? Can you gamify it? >> Are you suggesting that data governance isn't fun? (all laughing) Yes. >> But can you gamify it? Can you compete? >> We're using gamification in various in many ways. We haven't been using it in terms of data governance yet. Governance is just a horrible word, right? People have really negative connotations associated with it. But actually if you just step one degree away we're talking about quality. Quality means better decisions. And that's actually all governance is. Governance is knowing where your data is. Knowing who's responsible for fixing if it goes wrong. And being able to measure whether it's right or wrong in the first place. And it being better means we make better decisions. Our customers have better engagement with us. We please our customers more and therefore they hopefully engage with us more and buy more services. I think we should that your governance is something we invented through the need for regulation. And the need for control. And from that background. But realistically it's just, we should be proud about the data that we use in the organization. And we should want the best results from it. And it's not about governance. It's about us being proud about what we do. >> Yeah, a great note to end on. Thank you so much Christopher and Steven. >> Thank you. >> Cheers. >> I'm Rebecca Night for Paul Gillen we will have more from the IBM CDO Summit here in Boston coming up just after this. (electronic music)

>> Live from Boston, it's theCUBE! Covering IBM Chief Data Officers Summit. Brought to you by IBM. >> Welcome back to theCUBE's live coverage of the IBM CDO Summit here in Boston, Massachusetts. I'm your host Rebecca Knight, along with my co-host Paul Gillin. We're joined by Inderpal Bhandari, he is the Global Chief Data Officer at IBM. Thank you so much for coming back on theCUBE, Inderpal. >> It's my pleasure. >> It's great to have you. >> Thank you for having me. >> So I want to talk, I want to start by talking a little bit about your own career journey. Your first CDO job was in the early 2000s. You were one of the first CDOs, ever. In the history of Chief Data Officers. Talk a little bit about the evolution of the role and sort of set the scene for our viewers in terms of what you've seen, in your own career. >> Yes, no thank you, December 2006, I became a Chief Data Officer of a major healthcare company. And you know, it turned out at that time there were only four of us. Two in banking, one in the internet, I was the only one in healthcare. And now of course there are well over 1,999 of us and the professions taken off. And I've had the fortune of actually doing this four times now. So leading a legacy in four different organizations in terms of building that organizational capability. I think initially, when I became Chief Data Officer, the culture was one of viewing data's exhaust. Something that we had to discard, that came out of the transactions that you were, that your business was doing. And then after that you would discard this data, or you didn't really care about it. And over the course of time, people had begun to realize that data is actually a strategic asset and you can really use it to drive not just the data strategy, but the actual business strategy, and enable the business to go to the next level. And that transitions been tremendous to watch and to see. I've just been fortunate that I've been there for the full journey. >> Are you seeing any consensus developing around what background makes for a good CDO? What are the skills that a CDO needs? >> Yeah, no that's a very, very good question. My view has been evolving on that one too, over the last few years, right, as I've had these experiences. So, I'll jump to the conclusion, so that you kind of, to answer your question as opposed to what I started out with. The CDO, has to be the change agent in chief, for the organization. That's really the role of the CDO. So yes, there's the technical sharps that you have to have and you have to be able to deal with people who have advanced technical degrees and to get them to move forward. But you do have to change the entire organization and you have to be adept at going after the culture, changing it. You can't get frustrated with all the push back, that's inevitable. You have to almost develop it as an art, as you move forward. And address it, not just bottom up and lateral, but also top down. And I think that's probably where the art gets the most interesting. Because you've got to push a for change even at the top. But you can push just so far without really derailing everything that you are trying to do. And so, I think if I have to pick one attribute, it would be that the CDO has to be the change agent in chief and they have to be adept at addressing the culture of the organization, and moving it forward. >> You're laying out all of these sort of character traits that someone has to be indefatigable, inspirational, visionary. You also said during the keynote you have six months to really make your first push, the first six months are so important. When we talk about presidents, it's the first 100 days. Describe what you mean by that, you have six months? >> So if a new, and I'm talking here mainly about a large organization like an IBM, a large enterprise. When you go in, the key observation is it's a functioning organization. It's a growing concern. It's already making money, it's doing stuff like that. >> We hope. >> And the people who are running that organization, they have their own needs and demands. So very quickly, you can just become somebody who ends up servicing multiple demands that come from different business units, different people. And so that's kind of one aspect of it. The way the organization takes over if you don't really come in with an overarching strategy. The other way the organizations take over is typically large organizations are very siloed. And even at the lower levels you who have people who developed little fiefdoms, where they control that data, and they say this is mine, I'm not going to let anybody else have it. They're the only one's who really understand that curve. And so, pretty much unless you're able to get them to align to a much larger cause, you'll never be able to break down those silos, culturally. Just because of the way it's set up. So its a pervasive problem, goes across the board and I think, when you walk in you've got that, you call it honeymoon period, or whatever. My estimate is based on my experience, six months. If you don't have it down in six months, in terms of that larger cause that your going to push forward, that you can use to at least align everybody with the vision, or you're not going to really succeed. You'll succeed tactically, but not in a strategic sense. >> You're about to undertake the largest acquisition in IBM's history. And as the Chief Data Officer, you must be thinking right now about what that's going to mean for data governance and data integration. How are you preparing for an acquisition that large? >> Yeah so, the acquisition is still got to work through all the regulations, and so forth. So there's just so much we can do. It's much more from a planning stand point that we can do things. I'll give you a sense of how I've been thinking about it. Now we've been doing acquisitions before. So in that since we do have a set process for how we go about it, in terms of evaluating the data, how we're going to manage the data and so forth. The interesting aspect that was different for me on this one is I also talked back on our data strategy itself. And tried to understand now that there's going to be this big acquisition of move forward, from a planning standpoint how should I be prepared to change? With regard to that acquisition. And because we were so aligned with the overall IBM business strategy, to pursue cognition. I think you could see that in my remarks that when you push forward AI in a large enterprise, you very quickly run into this multi-cloud issue. Where you've got, not just different clouds but also unprime and private clouds, and you have to manage across all that and that becomes the pin point that you have to scale. To scale you have to get past that pin point. And so we were already thinking about that. Actually, I just did a check after the acquisition was announced, asking my team to figure out well how standardized are we with Red Hat Linux? And I find that we're actually completely standardized across with Red Hat Linux. We pretty much will have use cases ready to go, and I think that's the facet of the goal, because we were so aligned with the business strategy to begin with. So we were discovering that pinpoint, just as all our customers were. And so when the cooperation acted as it did, in some extent we're already ready to go with used cases that we can take directly to our clients and customers. I think it also has to do with the fact that we've had a partnership with Red Hat for some time, we've been pretty strategic. >> Do you think people understand AI in a business context? >> I actually think that that's, people don't really understand that. That's was the biggest, in my mind anyway, was the biggest barrier to the business strategy that we had embarked on several years ago. To take AI or cognition to the enterprise. People never really understood it. And so our own data strategy became one of enabling IBM itself to become an AI enterprise. And use that as a showcase for our clients and customers, and over the journey in the last two, three years that I've been with IBM. We've become more, we've been putting forward more and more collateral, but also technology, but also business process change ideas, organizational change ideas. So that our clients and customers can see exactly how it's done. Not that i'ts perfect yet, but that too they benefit from, right? They don't make the same mistakes that we do. And so we've become, your colleagues have been covering this conference so they will know that it's become more and more clear, exactly what we're doing. >> You made an interesting comment, in the keynote this morning you said nobody understands AI in a business context. What did you mean by that? >> So in a business context, what does it look like? What does AI look like from an AI enterprise standpoint? From a business context. So excuse me I just trouble them for a tissue, I don't know why. >> Okay, alright, well we can talk about this a little bit too while he-- >> Yeah, well I think we understand AI as an Amazon Echo. We understand it as interface medium but I think what he was getting at is that impacting business processes is a lot more complicated. >> Right. >> And so we tend to think of AI in terms of how we relate to technology rather than how technology changes the rules. >> Right and clearly its such, on the consumers side, we've all grasped this and we all are excited by its possibilities but in terms of the business context. >> I'm back! >> It's the season, yes. >> Yeah, it is the season, don't want to get in closer. So to your question with regard to how-- >> AI in a business context. >> AI in a business context. Consumer context everybody understands, but in a business context what does it really mean? That's difficult for people to understand. But eventually it's all around making decisions. But in my mind its not the big decisions, it's not the decisions we going to acquire Red Hat. It's not those decisions. It's the thousands and thousands of little decisions that are made day in and night out by people who are working the rank and file who are actually working the different processes. That's what we really need to go after. And if you're able to do that, it completely changes the process and you're going to get just such a lot more out of it, not just terms of productivity but also in terms of new ideas that lead to revenue enhancement, new products, et cetera, et cetera. That's what a business AI enterprise looks like. And that's what we've been bringing forward and show casing. In today's keynote I actually had Sonya, who is one of our data governance people, SMEs, who works on metadata generation. Really a very difficult manual problem. Data about data, specifically labeling data so that a business person could understand it. Its all been done manually but now it's done automatically using AI and its completely changed the process. But Sonya is the person who's at the forefront of that and I don't think people really understand that. They think in terms of AI and business and they think this is going to be somebody who's a data scientist, a technologist, somebody who's a very talented technical engineer, but it's not that. It's actually the rank and file people, who've been working these business processes, now working with an intelligent system, to take it to the next level. >> And that's why as you've said it's so important that the CDO is a change agent in chief. Because it is, it does require so much buy-in from, as you say, the rank and file, its not just the top decision makers that you're trying to persuade. >> Yes, you are affecting change at all levels. Top down, bottom up, laterally. >> Exactly. >> You have to go after it across the board. >> And in terms of talking about the data, it's not just data for data's sake. You need to talk about it in terms that a business person can understand. During the keynote, you described an earlier work that you were doing with the NBA. Can you tell our viewers a little bit about that? And sort of how the data had to tell a story? >> Yes, so that was in my first go 'round with IBM, from 1990 through '97. I was with IBM Research, at the Watson Research Lab, as a research staff member. And I created this program called Advanced Scout for the National Basketball Association. Ended up being used by every team on the NBA. And it would essentially suggest who to put in the line up, when you're matching lines up and so forth. By looking at a lot of game data and it was particularly useful during the Playoff games. The major lesson that came out of that experience for me, at that time, alright, this was before Moneyball, and before all this stuff. I think it was like '90, '93, '92. I think if you Google it you will still see articles about this. But the main lesson that came out for me was the first time when the program identified a pattern and suggested that to a coach during a playoff game where they were down two, zero, it suggested they start two backup players. And the coach was just completely flabbergasted, and said there's no way I'm going to do this. This is the kind of thing that would not only get me fired, but make me look really silly. And it hit me then that there was context that was missing, that the coach could not really make a decision. And the way we solved it then was we tied it to the snippets of video when those two players were on call. And then they made the decision that went on and won that game, and so forth. Today's AI systems can actually fathom all that automatically from the video itself. And I think that's what's really advanced the technology and the approaches that we've got today to move forward as quickly as they have. And they've taken hold across the board, right? In the sense of a consumer setting but now also in the sense of a business setting. Where we're applying it pretty much to every business process that we have. >> Exciting. Well Inderpal, thank you so much for coming back on theCUBE, it was always a pleasure talking to you. >> It's my pleasure, thank you. >> I'm Rebecca Knight for Paul Gillin, we will have more from theCUBE's live coverage of IBM CDO coming up in just a little bit. (upbeat music)

>> Live from Boston, it's theCUBE! Covering IBM Chief Data Officer Summit, brought to you by IBM. >> Welcome back everyone to the IBM CDO Summit and theCUBE's live coverage, I'm your host Rebecca Knight along with my co-host Paul Gillin. We have Joe Selle joining us. He is the Cognitive Solution Lead at IBM. And Thomas Ward, Supply Chain Cloud Strategist at IBM. Thank you so much for coming on the show! >> Thank you! >> Our pleasure. >> Pleasure to be here. >> So, Tom, I want to start with you. You are the author of Risk Insights. Tell our viewers a little bit about Risk Insights. >> So Risk Insights is a AI application. We've been working on it for a couple years. What's really neat about it, it's the coolest project I've ever worked on. And it really gets a massive amount of data from the weather company, so we're one of the biggest consumers of data from the weather company. We take that and we'd visualize who's at risk from things like hurricanes, earthquakes, things like IBM sites and locations or suppliers. And we basically notify them in advance when those events are going to impact them and it ties to both our data center operations activity as well as our supply chain operations. >> So you reduce your risk, your supply chain risk, by being able to proactively detect potential outages. >> Yeah, exactly. So we know in some cases two or three days in advance who's in harm's way and we're already looking up and trying to mitigate those risks if we need to, it's going to be a real serious event. So Hurricane Michael, Hurricane Florence, we were right on top of it and said we got to worry about these suppliers, these data center locations, and we're already working on that in advance. >> That's very cool. So, I mean, how are clients and customers, there's got to be, as you said, it's the coolest project you've ever worked on? >> Yeah. So right now, we use it within IBM right? And we use it to monitor some of IBM's client locations, and in the future we're actually, there was something called the Call for Code that happened recently within IBM, this project was a semifinalist for that. So we're now working with some non-profit groups to see how they could also avail of it, looking at things like hospitals and airports and those types of things as well. >> What other AI projects are you running? >> Go ahead. >> I can answer that one. I just wanted to say one thing about Risk Insights, which didn't come out from Tom's description, which is that one of the other really neat things about it is that it provides alerts, smart alerts out to supply chain planners. And the alert will go to a supply chain planner if there's an intersection of a supplier of IBM and a path of a hurricane. If the hurricane is vectored to go over that supplier, the supply chain planner that is responsible for those parts will get some forewarning to either start to look for another supplier, or make some contingency plans. And the other nice thing about it is that it launches what we call a Resolution Room. And the Resolution Room is a virtual meeting place where people all over the globe who are somehow impacted by this event can collaborate, share documents, and have a persistent place to resolve this issue. And then, after that's all done, we capture all the data from that issue and the resolution and we put that into a body of knowledge, and we mine that knowledge for a playbook the next time a similar event comes along. So it's a full-- >> It becomes machine learning. >> It's a machine learning-- >> Sort of data source. >> It's a full soup to nuts solution that gets smarter over time. >> So you should be able to measure benefits, you should have measurable benefits by now, right? What are you seeing, fewer disruptions? >> Yes, so in Risk Insights, we know that out of a thousand of events that occurred, there were 25 in the last year that were really the ones we needed to identify and mitigate against. And out of those we know there have been circumstances where, in the past IBM's had millions of dollars of losses. By being more proactive, we're really minimizing that amount. >> That's incredible. So you were going to talk about other kinds of AI that you run. >> Right, so Tom gave an overview of Risk Insights, and we tied it to supply chain and to monitoring the uptime of our customer data centers and things like that. But our portfolio of AI is quite broad. It really covers most of the middle and back and front office functions of IBM. So we have things in the sales domain, the finance domain, the HR domain, you name it. One of the ones that's particularly interesting to me of late is in the finance domain, monitoring accounts receivable and DSO, day sales outstanding. So a company like IBM, with multiple billions of dollars of revenue, to make a change of even one day of day sales outstanding, provides gigantic benefit to the bottom line. So we have been integrating disparate databases across the business units and geographies of IBM, pulling that customer and accounts receivable data into one place, where our CFO can look at an integrated approach towards our accounts receivable and we know where the problems are, and we're going to use AI and other advanced analytic techniques to determine what's the best treatment for that AI, for those customers who are at risk because of our predictive models, of not making their payments on time or some sort of financial risk. So we can integrate a lot of external unstructured data with our own structured data around customers, around accounts, and pull together a story around AR that we've never been able to pull before. That's very impactful. >> So speaking of unstructured data, I understand that data lakes are part of your AI platform. How so? >> For example, for Risk Insights, we're monitoring hundreds of trusted news sources at any given time. So we know, not just where the event is, what locations are at risk, but also what's being reported about it. We monitor Twitter reports about it, we monitor trusted news sources like CNN or MSNBC, or on a global basis, so it gives our risk analyst not just a view of where the event is, where it's located, but also what's being said, how severe it is, how big are those tidal waves, how big was the storm surge, how many people were affected. By applying some of the machine learning insights to these, now we can say, well if there are couple hundred thousand people without power then it's very likely there is going to be multimillions of dollars of impact as a result. So we're now able to correlate those news reports with the magnitude of impact and potential financial impact to the businesses that we're supporting. >> So the idea being that IBM is saying, look what we've done for our own business (laughs), imagine what we could do for you. As Inderpal has said, it's really using IBM as its own test case and trying to figure this all out and learning as it goes and he said, we're going to make some mistakes, we've already made some mistakes but we're figuring it out so you don't have to make those mistakes. >> Yeah that's right. I mean, if you think about the long history of this, we've been investing in AI, really, since, depending on how you look at it, since the days of the 90's, when we were doing Deep Blue and we were trying to beat Garry Kasparov at chess. Then we did another big huge push on the Jeopardy program, where we we innovated around natural language understanding and speed and scale of processing and probability correctness of answers. And then we kind of carry that right through to the current day where we're now proliferating AI across all of the functions of IBM. And there, then, connecting to your comment, Inderpal's comment this morning was around let's just use all of that for the benefit of other companies. It's not always an exact fit, it's never an exact fit, but there are a lot of pieces that can be replicated and borrowed, either people, process or technology, from our experience, that would help to accelerate other companies down the same path. >> One of the questions around AI though is, can you trust it? The insights that it derives, are they trustworthy? >> I'll give a quick answer to that, and then Tom, it's probably something you want to chime in on. There's a lot of danger in AI, and it needs to be monitored closely. There's bias that can creep into the datasets because the datasets are being enhanced with cognitive techniques. There's bias that can creep into the algorithms and any kind of learning model can start to spin on its own axis and go in its own direction and if you're not watching and monitoring and auditing, then it could be starting to deliver you crazy answers. Then the other part is, you need to build the trust of the users, because who wants to take an answer that's coming out of a black box? We've launched several AI projects where the answer just comes out naked, if you will, just sitting right there and there's no context around it and the users never like that. So we've understood now that you have to put the context, the underlying calculations, and the assessment of our own probability of being correct in there. So those are some of the things you can do to get over that. But Tom, do you have anything to add to that? >> I'll just give an example. When we were early in analyzing Twitter tweets about a major storm, what we've read about was, oh, some celebrity's dog was in danger, like uh. (Rebecca laughs) This isn't very helpful insight. >> I'm going to guess, I probably know the celebrity's dog that was in danger. (laughs) >> (laughs) actually stop saying that. So we learned how to filter those things out and say what are the meaningful keywords that we need to extract from and really then can draw conclusions from. >> So is Kardashian a meaningful word, (all laughing) I guess that's the question. >> Trending! (all laughing) >> Trending now! >> I want to follow up on that because as an AI developer, what responsibility do developers have to show their work, to document how their models have worked? >> Yes, so all of our information that we provided the users all draws back to, here's the original source, here's where the information was taken from so we can draw back on that. And that's an important part of having a cognitive data, cognitive enterprise data platform where all this information is stored 'cause then we can refer to that and go deeper as well and we can analyze it further after the fact, right? You can't always respond in the moment, but once you have those records, that's how you can learn from it for the next time around. >> I understand that building test models in some cases, particularly in deep learning is very difficult to build reliable test models. Is that true, and what progress is being made there? >> In our case, we're into the machine learning dimension yet, we're not all the way into deep learning in the project that I'm involved with right now. But one reason we're not there is 'cause you need to have huge, huge, vast amounts of robust data and that trusted dataset from which to work. So we aspire towards and we're heading towards deep learning. We're not quite there yet, but we've started with machine learning insights and we'll progress from there. >> And one of the interesting things about this AI movement overall is that it's filled with very energetic people that's kind of a hacker mindset to the whole thing. So people are grabbing and running with code, they're using a lot of open source, there's a lot of integration of the black box from here, from there in the other place, which all adds to the risk of the output. So that comes back to the original point which is that you have to monitor, you have to make sure that you're comfortable with it. You can't just let it run on its own course without really testing it to see whether you agree with the output. >> So what other best practices, there's the monitoring, but at the same time you do that hacker culture, that's not all bad. You want people who are energized by it and you are trying new things and experimenting. So how do you make sure you let them have, sort of enough rein but not free rein? >> I would say, what comes to mind is, start with the business problem that's a real problem. Don't make this an experimental data thing. Start with the business problem. Develop a POC, a proof of concept. Small, and here's where the hackers come in. They're going to help you get it up and running in six weeks as opposed to six months. And then once you're at the end of that six-week period, maybe you design one more six-week iteration and then you know enough to start scaling it and you scale it big so you've harnessed the hackers, the energy, the speed, but you're also testing, making sure that it's accurate and then you're scaling it. >> Excellent. Well thank you Tom and Joe, I really appreciate it. It's great to have you on the show. >> Thank you! >> Thank you, Rebecca, for the spot. >> I'm Rebecca Knight for Paul Gillin, we will have more from the IBM CDO summit just after this. (light music)

>> Live from Boston, it's theCUBE, covering IBM Chief Data Officer Summit, brought to you by IBM. >> Welcome back everyone to theCUBE's live coverage of the IBM CDO Summit here in Boston, Massachusetts. I'm your host Rebecca Knight*, and I'm joined by cohost, Paul Gillan*. We have a guest today, John Thomas. He is the Distinguished Engineer and Director* at IBM. Thank you so much for coming, returning to theCUBE. You're a CUBE veteran, CUBE alum. >> Oh thank you Rebecca, thank you for having me on this. >> So tell our viewers a little bit about, you're a distinguished engineer. There are only 672 in all of IBM. What do you do? What is your role? >> Well that's a good question. Distinguished Engineer is kind of a technical executive role, which is a combination of applying the technology skills, as well as helping shape IBM strategy in a technical way, working with clients, et cetera. So it is a bit of a jack of all trades, but also deep skills in some specific areas, and I love what I do (laughs lightly). So, I get to work with some very talented people, brilliant people, in terms of shaping IBM technology and strategy. Product strategy, that is part of it. We also work very closely with clients, in terms of how to apply that technology in the context of the client's use status. >> We've heard a lot today about soft skills, the importance of organizational people skills to being a successful Chief Data Officer, but there's still a technical component. How important is the technical side? What is, what are the technical skills that the CDOs need? >> Well, this is a very good question Paul. So, absolutely, so, navigating the organizational structure is important. It's a soft skill. You are absolutely right. And being able to understand the business strategy for the company, and then aligning your data strategy to the business strategy is important, right? But the underlying technical pieces need to be solid. So for example, how do you deal with large volumes of different types of data spread across a company? How do you manage that data? How do you understand the data? How do you govern that data? How do you then master leveraging the value of that data in the context of your business, right? So an understanding, a deep understanding of the technology of collecting, organizing, and analyzing that data is needed for you to be a successful CDO. >> So in terms of, in terms of those skillsets that you're looking for, and one of the things that Inderpal said earlier in his keynote, is that, there are just, it's a rare individual who truly understands the idea of how to collect, store, analyze, curatize, monetize the data, and then also have the soft skills of being able to navigate the organization, being able to be a change agent who is inspiring, inspiring the rank and file. How do you recruit and retain talent? I mean, this seems to be a major challenge. >> Expertise is, and getting the right expertise in place, and Inderpal talked about it in his keynote, which was the very first thing he did was bring in talent. Sometimes it is from outside of your company. Maybe you have a kind of talent that has grown up in your company. Maybe you have to go outside, but you've got to bring in the right skills together. Form the team that understands the technology, and the business side of things, and build this team, and that is essential for you to be a successful CDO. And to some extent, that's what Inderpal has done. That's what the analytic CDO's office has done. Seth Dobrin, my boss, is the analytics CDO , and he and the analytics CDO team actually hired people with different skills. Data engineering skills, data science skills, visualization skills, and then put this team together which understands the, how to collect, govern, curate, and analyze the data, and then apply them in specific situations. >> There's been a lot of talk about AI, at this conference, which seems to be finally happening. What do you see in the field, or perhaps projects that you've worked on, of examples of AI that are really having a meaningful business impact? >> Yeah Paul, that is a very good question because, you know, the term AI is overused a lot as you can imagine, a lot of hype around it. But I think we are past that hype cycle, and people are looking at, how do I implement successful use cases? And I stress the word use case, right? In my experience these, how I'm going to transform my business in one big boil the ocean exercise, does not work. But if you have a very specific bounded use case that you can identify, the business tells you this is relevant. The business tells you what the metrics for success are. And then you focus your attention, your efforts on that specific use case with the skills needed for that use case, then it's successful. So, you know, examples of use cases from across the industries, right? I mean everything that you can think of. Customer-facing examples, like, how do I read the customer's mind? So when, if I'm a business and I interact with my customers, can I anticipate what the customer is looking for, maybe for a cross-sell opportunity, or maybe to reduce the call handing time when a customer calls into my call center. Or trying to segment my customers so I can do a proper promotion, or a campaign for that customer. All of these are specific customer phasing examples. There also are examples of applying this internally to improve precesses, capacity planning for your infrastructure, can I predict when a system is likely to have an outage, or can I predict the traffic coming into my systems, into my infrastructure and provision capacity for that on demand, So all of these are interesting applications of AI in the enterprise. >> So when your trying, what are the things we keep hearing, is that we need to data to tell a story To, the data needs to be compelling enough so that the people, the data scientist get it but then also the other kinds of business decision makers get it to. >> Yep >> So, what are sort of, the best practices that have emerged from your experience? In terms of, being able to, for your data to tell a story that you want it to tell. >> Yeah, well I mean if the pattern doesn't exist in the data then no amount of fancy algorithms can help, you know? and sometimes its like searching for a needle in a haystack but assuming, I guess the first step is, like I said, What is the use case? Once you have a clear understanding of your use case and such metrics for your use case, do you have the data to support that use case? So for example if it's fraud detection, do you actually have the historical data to support the fraud use case? Sometimes you may have transactional data from your, transocular from your core enterprise systems but that may not be enough. You may need to alt mend it with external data, third party data, maybe unstructured data, that goes along with your transaction data. So the question is, can you identify the data that is needed to support the use case and if so can I, is that data clean, is that data, do you understand the lineage of the data, who has touched and modified the data, who owns the data. So then I can start building predictive models and machine learning, deep learning models with that data. So use case, do you have the data to support the use case? Do you understand how that sata reached you? Then comes the process of applying machine learning algorithms and deep learning algorithms against that data. >> What are the risks of machine learning and particularly deep learning, I think because it becomes kind of a black box and people can fall into the trap of just believing what comes back, regardless of whether the algorithms are really sound or the data is. What is the responsibility of data scientist to sort of show their work? >> Yeah, Paul this is fascinating and not completely solid area, right? So, bias detection, can I explain how my model behaved, can I ensure that the models are fair in their predictions. So there is a lot of research, a lot of innovation happening in the space. IBM is investing a lot into space. We call trust and transparency, being able to explain a model, it's got multiple levels to it. You need some level of AI governments itself, just like we talked about data governments that is the notion of AI governments. Which is what motion of the model was used to make a prediction? What were the imports that went into that model? What were the decisions that were, that were the features that were used to make a sudden prediction? What was the prediction? And how did that match up with ground truth. You need to be able to capture all that information but beyond that, we have got actual mechanisms in place that IBM Research is developing to look at bias detection. So pre processing during execution post processing, can I look for bias in how my models behave and do I have mechanisms to mitigate that? So one example is the open source Python library, called AIF360 that comes from IBM Research and has contributed to the open source community. You can look at, there are mechanisms to look at bias and provide some level of bias mitigation as part of your model building exercises. >> And the bias mitigation, does it have to do with, and I'm going to use an IMB term of art here, the human in the loop, is it how much are you actually looking at the humans that are part of this process >> Yeah, humans are at least at this point in time, humans are very much in the loop. This notion of Peoria high where humans are completely outside the loop is, we're not there yet so very much something that the system can for awhile set off recommendations, can provide a set of explanations and can someone who understands the business look at it and make a corrective, take corrective actions. >> There has been, however to Rebecca's point, some prominent people including Bill Gates, who have speculated that the AI could ultimately be a negative for humans. What is the responsibility of company's like IBM to ensure that humans are kept in the loop? >> I think at least at this point IBM's view is humans are an essential part of AI. In fact, we don't even use artificial intelligence that much we call it augmented intelligence. Where the system is pro sending a set of recommendations, expert advise to the human who can then make a decision. For example, you know my team worked with a prominent health care provider on you know, models for predicting patient death in the case of sepsis, sepsis-onset. This is, we are talking literally life and death decisions being made and this is not something you can just automate and throw into a magic black box, and have a decision be made. So this is absolutely a place where people with deep, domain knowledge are supported, are opt mended with, with AI to make better decisions, that's where I think we are today. As to what will happen five years from now, I can't predict that yet. >> Well I actually want to- >> But the question >> bring this up to both of you, the role, so you are helping doctor's make these decisions, not just this is what the computer program says about this patient's symptoms here but this is really, so you are helping the doctor make better decisions. What about the doctors gut, in the, his or her intuition to. I mean, what is the role of that, in the future? >> I think it goes away, I mean I think, the intuition really will be trumped by data in the long term because you can't argue with the facts. Some people do these days. (soft laughter) But I don't remember (everyone laughing) >> We have take break there for some laughter >> Intrested in your perspective onthat is there, will there, should there always be a human on the front line, who is being supported by the back end or would you see a scenario were an AI is making decisions, customer facing decisions that are, really are life and death decisions? >> So I think in the consumer invest way, I can definitely see AI making decisions on it's own. So you know if lets say a recommender system would say as you know I think, you know John Thomas, bought these last five things online. He's likely to buy this other thing, let's make an offer to him. You know, I don't need another human in the loop for that >> No harm right? >> Right. >> It's pretty straight forward, it's already happening, in a big way but when it comes to some of these >> Prepoping a mortgage, how about that one? >> Yeah >> Where bias creeps in a lot. >> But that's one big decision. >> Even that I think can be automated, can be automated if the threshold is set to be what the business is comfortable with, were it says okay, above this probity level, I don't really need a human to look at this. But, and if it is below this level, I do want someone to look at this. That's you know, that is relatively straight forward, right? But if it is a decision about you know life or death situation or something that effects the very fabric of the business that you are in, then you probably want a domain explore to look at it. In most enterprises, enterprises cases will fall, lean toward that category. >> These are big questions. These are hard questions. >> These are hard questions, yes. >> Well John, thank you so much for doing >> Oh absolutely, thank you >> On theCUBE, we really had a great time with you. >> No thank you for having me. >> I'm Rebecca Knight for Paul Gillan, we will have more from theCUBE's live coverage of IBM CDO, here in Boston, just after this. (Upbeat Music)

>> Live from Boston, it's the CUBE. Covering IBM Chief Data Officer Summit. Brought to you by IBM. >> Welcome to the CUBE's live coverage of IBM Chief Data Officer Summit here in Boston, Massachusetts. I'm your host, Rebecca Knight along with my co host, Paul Gillin. We're starting our coverage today. This is the very first day of the summit. We have two guests, Caitlin Halferty, she is the AI accelerator lead at IBM, and Sonia Mezzetta, the data governance technical product leader. Thank you both so much for coming on the CUBE >> Thanks for having us. >> So this is the ninth summit. Which really seems hard to belief. But we're talking about the growth of the event and just the kinds of people who come here. Just set the scene for our viewers a little bit, Caitlin. >> Sure, so when we started this event back in 2014, we really were focused on building the role of the chief data officer, and at that time, we know that there were just a handful across industries. Few in finance banking, few in health care, few in retail, that was about it. And now, you know, Gartner and Forrester, some industry analysts say there are thousands across industries. So it's not so much about demonstrating the value or the importance, now, it's about how are our Chief Data Officers going to have the most impact. The most business impact. And we're finding that they're really the decision-makers responsible for investment decisions, bringing cognition, AI to their organizations. And the role has grown and evolved. When we started the first event, we had about 20, 30 attendees. And now, we get 140, that join us in the Spring in San Francisco and 140 here today in Boston. So we've really been excited to see the growth of the community over the last four years now. >> How does that affect the relationship, IBM's relationship with the customer? Traditionally, your constituent has been the CIO perhaps the COO, but you've got this new C level executive. Now, what role do they play in the buying decision? >> There was really a lot of, I think back to, I co-authored a paper with some colleagues in 2014 on the rise of Chief Data Officer. And at that time, we interviewed 22 individuals and it was qualitative because there just weren't many to interview, I couldn't do a quantitative study. You know, I didn't have sample size. And so, it's been really exciting to see that grow and then it's not just the numbers grow, it's the impact they're having. So to you questions of what role are they playing, we are seeing that more and more their scope is increasing, their armed and equipped with teams that lead data science, machine learning, deep learning capabilities so they're differentiated from a technology perspective. And then they're really armed with the investment and budget decisions. How should we invest in technology. Use data as a strategic corporate asset to drive our progress forward in transformation. And so we've really seen a significant scope increase in terms of roles and responsibilities. And I will say though, there's still that blocking and tackling around data strategy, what makes a compelling data strategy. Is is the latest, greatest? Is it going to have an impact? So we're still working through those key items as well. >> So speaking of what makes this compelling strategy, I want to bring you into the conversation Sonia, because I now you're on the automated metadata generation initiative, which is a big push for IBM. Can you talk a little bit about what you're doing at IBM? >> Sure. So I am in charge of the data governance products internally within the company and specifically, we are talking today about the automated metadata generation tool. What we've tried to do with that particular product is to try to basically leverage automation and artificial intelligence to address metadata issues or challenges that we're facing as part of any traditional process that takes place today and trying to do curation for metadata. So specifically, what I would like to also point out is the fact that the metadata curation process in the traditional sense is something that's extremely time-consuming, very manual and actually tedious. So, one of the things that we wanted to do is to address those challenges with this solution. And to really focus in and hone in on leveraging the power of AI. And so one of the things that we did there was to basically take our traditional process, understand what were the major challenges and then focusing on how AI can address those challenges. And today at 4 p.m. I'll be giving a demo on that, so hopefully, everybody can understand the power of leveraging that. >> This may sound like a simple question, but I imagine for a lot of people outside of the CIO of the IT organization, their eyes glaze over when they hear terms like data governance. But it's really important. >> It is. >> So can you describe why it's important? >> Absolutely. >> And why metadata is important too. >> Absolutely. Well, I mean, metadata in itself is extremely critical for any data monetization position strategy, right. The other importance is in order to derive critical business insights that can lead to monetary value within a company. And the other aspect to that is data quality which Interpol talked about, right? So, in order for you to have the right data governance, you need to have right metadata in order for you to have high level of data quality can, if you don't and you're spending a lot of time cleaning dirty data and dealing with inefficiencies or perhaps making wrong business decisions based on bad data quality, it's all connected back to having the right level of data governance. >> So, I mean, I'm going to also go back to something you were talking about earlier and that's just the sheer number of CDOs that we have. We have statistic here, 90% of large global companies will have the CDO by 2019. That's really astonishing. Can you talk a little bit about what you see as sort of the top threats and opportunities that CDOs as grappling with right now. >> And let me make this tangible. I'll just describe my last two weeks, for example. I was with the CDO in person in Denver of a beer company, organization, and they were looking at some MNA opportunities and figuring out what their strategy was. I was at a bank in Chicago with the head of enterprise data government there, looking at it from a regular (mumbles) perspective. And then I was with a large multinational retail organization with their CDO and team figuring out how did they work at a sort of global scale and what did they centralize at enterprise data level. And what did they let markets and teams customize out in the field, out in the GOs. And so, that's just an example of, regardless of industry, regardless of these challenges, I'm seeing these individuals are increasingly responsible for those strategic decisions. And oftentimes, we start with the data strategy and have a good discussion about what is that organization's monetization strategy. What's the corporate business case? How are they going to make money in the future and how can we architect the data strategy that will accelerate their progress there? And again, regardless of product we're selling or retail, excuse me, our industry, those are the same types of challenges and opportunities we're grappling with. >> In the early days there was a lot of questions about the definition of the role and those CDOs set in different departments and reported to different people, are you seeing some commonality emerge now about how this role, where it sits in the organization, and what its responsibilities are? >> It's a great question, I get that all the time. And especially for organizations that recognize the need for enterprise data management. They want to invest in a senior level decision-maker. And then it's a question of where should they sit organizationally? For us internally, within IBM, we report to our Chief Financial Officer. And so, we find that to be quite a compelling fit in terms of budget. And visibility into some of those spend decisions. And we're on par in peers with our CIO, so I see that quite a bit where a Chief Data Officer is now on par and appear to the CIO. We tend to find that when it's potentially buried in the CIO's organization, you lose a little of that autonomy in terms of decision-making, so if you're able to position as partners and drive that transformation for your organization forward together, that can often work quite well. >> So that partnership, is it, I mean ideally, it is collaborative and collegial, but is it ever, are there ever tensions there and how do you recommend the companies get over, overcome those obstacles? >> Absolutely, in the fight for resources that we all have, especially talent and retaining some of our top talent, should that individual or those teams sit within a CIO's organization or a CDO's organization? How do we figure that out? I think there's always going to be the challenge of who owns what. We joke, sometimes, it feels like you own everything when you're in the data space, because you own all of the data that flows through, all your business processes, both CDO-owned and corporate HR's supply chain finance. Sometimes it feels you don't own anything. And so we joke that it's, you have to really carve that out. I think the important part is to really articulate what the data strategy is, what the CDO or enterprise data management office owns from a data perspective and then building up that platform and do it in partnership with your CIO team. And then you really start to be able to build and deploy those AI applications off that platform. That's what we've been able to see, so. >> I want to go back to something Sonia said this morning during the keynote, you talked about IBM's master metadata list catalog unifying your organization around a certain set of terms. There's 6,000 terms in that catalog. Now, how did you arrive at 6,000? And what are some rules for an organization trying to do something like that? How defined, how small should that sub-terms be? >> Sure. Well, we started off with a traditional approach which is probably something that most companies are familiar with these days. The traditional process was really just based on basically reaching out to a large number of subject matter experts across the enterprise that represent in many different data domains such as customer, offering, financial, etc. And essentially having them label this data, specifically with the business metadata that's used internally across a company. Now, another example to that is that there are different organizations across the company. We are a worldwide company. And so, what one business might call a particular piece of data, which is customer, another might call it client. Which really ended up being this very large list of 6,000 business terms which is what we're using internally. But one thing that we're trying to do to be able to kind to basically connect the different business terms is leverage knowledge management and specifically ontological relationships to be able to link the data together and make it more reasonable and provide better quality with that. >> What are the things that you were talking about, Interpol was talking about on the main stage too during the keynote, was making sure that the data is telling a story because getting by in is one of the biggest challenges. How do you recommend companies think about this and approach this very big daunting task? >> I'll start and then I'm sure you have a perspective as well. One of the things that we've seen internally and I work with my client on, is every project we initiate, we really want strong sponsorship from the business in terms of funding, making sure that the right decision-makers are involved. We've identified some projects for example, that we've been able to deploy around supply chains. So identifying the risk on our supply chain processes. Some of the risks in sites, we're going to demo a little bit later today. The AMG work that Sonia's leading. And all of those efforts are underway in partnership with the business. One of my favorite ones is around enabling our sellers to better understand information about, and data, about the customers. So like most organizations, customer data is housed in silo systems that don't necessarily talk well with each other, and so it's an effort to really pull that data together in partnership with our digital sellers and enable them to then pull up user interface, user-friendly, an app where they can identify and drill down to the types of information they need about their customers. And so our thought and recommendation based on our experience and then what I'm seeing is really having that strong partnership with the business. And the contribution funding, stakeholder involvement, engagement, and then you start to prioritize where you'll have the most impact. >> You did a program called the AI accelerator. What is that? >> We did, so when we stood up our first chief data office, it was three years ago now, we wanted to be quite transparent about the journey of driving cognition through our enterprise. And we were really targeting those CDO and processes around client master product data and then all of our enterprise processes. So that first six months was about writing the data strategy and implementing that, next we spent a year on all of our processes, really mapping out, we call it journey mapping, I think a lot of folks do that, by process. So HR, supply chain, identifying ways. How it's done today, how it will be done in a cognitive AI like future state. And then also, as we're driving out those efficiencies in automation, those reinvestment opportunities to free up that money for future initiatives. And so that was the first year, year and a half. And now, we're at the point where we've evolved far enough along that we think we're learned some lessons on the way and there's been some hurdles and stumbling blocks and obstacles. And so a year ago, we really start a cognitive enterprise blueprint and that was really intended to reflect all of our experiences, driving that transformation. A lot of customer engagements, lot of industry analysts feedback as well. And now we formalized that initiative. So now I have a really fantastic team of folks working with me. Subject matter domain expertise, really deep in different processes, solutions, folks, architects. And what we can do is pull together the right breadth and depth of IBM resources. Deploy it, customize it to customer need and really, hopefully, accelerate and apply a lot of what we've learned, lot of what the clients have learned, to accelerate their own AI transformation journey. >> But AI, IBM is the guinea pig and it showcase. And so you're learning as you go and helping customers do that too. >> Exactly and we've now built our platform, deployed that, as we mentioned, we've got about 30,000 active users, active users, using our platform. Plan to grow to 100,000. We're seeing about 600 million in business benefit internally from the work we've done. And so we want to really share that and do some good, best practice sharing and accelerate some of that process. >> IBM used the term cognitive rather than AI. What is the difference or is there one? >> I think we're starting actually to shift from cognitive to AI because of that exact perspective. AI, I think is better understood in the industry, in the market and that's what's resonating more so with clients and I think it's more reflective of what we're doing. And our particular approach is human in the loop. So we've always said rather than the black box sort of AI algorithms running behind the scenes, we want to make sure that we do that with trust and transparency, so there's a real transparency aspect to what we're doing. And the other thing I would notice, we talk about sort of your data is your data. Insights derive from that data is your insights. So we've worked quite closely with our legal teams to really articulate how your data is used. If you engage and partner with us to drive AI in your enterprise, making sure we have that trust and transparency (mumbles) clearly articulated is another important aspect for us. >> Getting right back to data governance. >> Right, right, exactly. Which is our we've come full circle. >> Well Caitlin and Sonia, thank you so much for coming on the CUBE, it was great. Great to kick off this summit together. >> Great to see you again, as always. >> I'm Rebecca Knight for Paul Gillin, stay tuned for more of the CUBE's live coverage of IBM CDO Summit here in Boston. (techno music)

>> Live from Boston, it's theCUBE. Covering IBM Chief Data Officer Summit. Brought to you by IBM. >> Welcome back, everyone, to theCUBE's live coverage of the IBM CDO Summit here in Boston, Massachusetts. I'm your host, Rebecca Knight, along with my co-host Paul Gillin. We have two guests for this session, we have Rebecca Shockley, she is executive consultant and IBM Global Business Services, and Alfred Essa, vice president analytics and R&D at McGraw-Hill Education. Rebecca and Alfred, thanks so much for coming on theCUBE. >> Thanks for having us. >> So I'm going to start with you, Rebecca. You're giving a speech tomorrow about the AI ladder, I know you haven't finished writing it-- >> Shh, don't tell. >> You're giving a speech about the AI ladder, what is the AI ladder? >> So, when we think about artificial intelligence, or augmented intelligence, it's very pervasive, we're starting to see it a lot more in organizations. But the AI ladder basically says that you need to build on a foundation of data, so that data and information architecture's your first rung, and with that data, then you can do analytics, next rung, move into machine learning once you're getting more comfortable, and that opens up the whole world of AI. And part of what we're seeing is organizations trying to jump to the top of the ladder or scramble up the ladder really quickly and then realize they need to come back down and do some foundational work with their data. I've been doing data and analytics with IBM for 21 years, and data governance is never fun. It's hard. And people would just as soon go do something else than do data governance, data security, data stewardship. Especially as we're seeing more business-side use of data. When I started my career, data was very much an IT thing, right. And part of my early career was basically just getting IT and business to communicate in a way that they were saying the same things. Well now you have a lot more self-service analytics, and business leaders, business executives, making software decisions and various decisions that impact the data, without necessarily understanding the ripples that their decisions can have throughout the data infrastructure, because that's not their forte. >> So what's the outcome, what's the result of this? >> Well, you start to see organizations, it's similar to what we saw when organizations first started making data lakes, right? The whole concept of a data lake, very exciting, interesting, getting all the data in together, whether it's virtual or physical. What ended up happening is without proper governance, without proper measures in place, you ended up with a data swamp instead of a data lake. Things got very messy very quickly, and instead of creating opportunities you were essentially creating problems. And so what we're advising clients, is you really have to make sure that you're focused on taking care of that first rung, right? Your data architecture, your information architecture, and treating the data with the respect as a strategic asset that it is, and making sure that you're dealing with that data in a proper manner, right? So, basically telling them, yes we understand that's fun up there, but come back down and deal with your foundation. And for a lot of organizations, they've never really stepped into data governance, because again, data isn't what they think makes the company run, right? So banks are bankers, not data people, but at the same time, how do you run a bank without data? >> Well exactly. And I want to bring you into this conversation, Alfred, as McGraw-Hill, a company that is climbing the ladder, in a more steady fashion. What's your approach? How do you think about bringing your teams of data scientists together to work to improve the company's bottom line, to enhance the customer experience? >> First I'd sort of like to start with laying some of the context of what we do. McGraw-Hill Education has been traditionally a textbook publisher, we've been around for over a hundred years, I started with the company over a hundred years ago. (all laughing) >> You've aged well. >> But we no longer think of ourselves as a textbook publisher. We're in the midst of a massive digital transformation. We started that journey over five years ago. So we think of ourselves as a software company. We're trying to create intelligent software based on smart data. But it's not just about software and AI and data, when it comes to education it's a tale of two cities. This is not just the U.S., but internationally. Used to be, we were born, went to school, got a job, raised a family, retired, and then we die. Well now, education is not episodic. People need to be educated, it's life-long learning. It's survival, but also flourishing. So that's created a massive problem and a challenge. It's a tale of two cities, by that I mean there's an incredible opportunity to apply technology, AI, we see a lot of potential in the new technologies. In that sense, it's the best of times. The worst of times is, we're faced with massive problems. There's a lot of inequity, we need to educate a people who have largely been neglected. That's the context. So I think in now answering your question about data science teams, first and foremost, we like to get people on the teams excited about the mission. It's like, what are we trying to achieve? What's the problem that we're trying to achieve? And I think the best employees, including data scientists, they like solving hard problems. And so, first thing that we try to do is, it's not what skills you have, but do you like solving really, really hard problems. And then taking it next step, I think the exciting thing about data science is it's an interdisciplinary field. It's not one skill, but you need to bring together a combination of skills. And then you also have to excel and have the ability to work in teams. >> You said that the AI has potential to improve the education process. Now, people have only so much capacity to learn, how can AI accelerate that process? >> Yeah, so if we stand back a little bit and look at the traditional model of education, there's nothing wrong with it but it was successful for a certain period of years, and it works for some people. But now the need for education is universal, and life long. So what our basic model, current model of education is lecture mode and testing. Now from a learning perspective, learning science perspective, all the research indicates that that doesn't work. It might work for a small group of people, but it's not universally applicable. What we're trying to do, and this is the promise of AI, it's not AI alone, but I think this is a big part of AI. What we can do is begin to customize and tailor the education to each individual's specific needs. And just to give you one quick example of that, different students come in with different levels of prior knowledge. Not everyone comes into a class, or a learning experience, knowing the same things. So what we can do with AI is determine, very, very precisely, just think of it as a brain scan, of what is it each student need to know at every given point in time, and then based on that we can determine also, this is where the models and algorithms are, what are you ready to learn next. And what you might be ready to learn next and what I might be ready to learn next is going to be very different. So our algorithms also help route delivery of information and knowledge at the right time to the right person, and so on. >> I mean, you're talking about these massive social challenges. Education as solving global inequity, and not every company has maybe such a high-minded purpose. But does it take that kind of mission, that kind of purpose, to unite employees? Both of you, I'm interested in your perspectives here. >> I don't think it takes, you know, a mission of solving global education. I do firmly agree with what Al said about people need a mission, they need to understand the outcome, and helping organizations see that outcome as being possible, gives them that rally point. So I don't disagree, I think everybody needs a mission to work towards but it doesn't have to be solving-- >> You want to extract that mission to a higher level, then. >> Exactly. >> Making the world a better place. >> Exactly, or at least your little corner of the world. Again what we're seeing, the difficulty is helping business leaders or consumers or whomever understand how data plays into that. You may have a goal of, we want better relationship with our customer, right? And at least folks of my age think that's a personal one-on-one kind of thing. Understanding who you are, I can find that much more quickly by looking at all your past transactions, and all of your past behaviors, and whether you clicked this or that. And you should expect that I remember things from one conversation to the next. And helping people understand that, you know, helping the folks who are doing the work, understand that the outcome will be that we can actually treat our customers the way that you want to be treated as a person, gives them that sense of purpose, and helps them connect the dots better. >> One of the big challenges that we hear CDOs face is getting buy-in, and what you're proposing about this new model really appending the old sage on the stage model, I mean, is there a lot of pushback? Is it difficult to get the buy-in and all stakeholders to be on the same page? >> Yeah, it is, I think it's doubly difficult. The way I think about it is, it's like a shift change in hockey, where you have one shift that's on the ice and another one that's about to come on the ice, that's a period of maximum vulnerability. That's where a lot of goals are scored, people get upset, start fighting. (all laughing) That's hockey. >> That's what you do. >> Organizations and companies are faced with the same challenge. It's not that they're resisting change. Many companies have been successful with one business model, while they're trying to bring in a new business model. Now you can't jettison the old business model because often that's paying the bills. That's the source of the revenue. So the real challenge is how are you going to balance out these two things at the same time? So that's doubly difficult, right. >> I want to ask you quickly, 'cause we have to end here, but there's a terrible shortage of cybersecurity professionals, data science professionals, the universities are simply not able to keep up with demand. Do you see the potential for AI to step in and fill that role? >> I don't think technology by itself will fill that role. I think there is a deficit of talented people. I think what's going to help fill that is getting people excited about really large problems that can be solved with this technology. I think, actually I think the talent is there, what I see is, I think we need to do a better job of bringing more women, other diverse groups, into the mix. There are a lot of barriers in diversity in bringing talented people. I think they're out there, I think we could do a much better job with that. >> Recruiting them, right. Alfred, Rebecca, thanks so much for coming on theCUBE, it was a pleasure. >> Thank you so much for having us. >> I'm Rebecca Knight, for Paul Gillin, we will have more from theCUBE's live coverage of the IBM CDO Summit here in Boston coming up in just a little bit.

>> Live from Boston, it's theCUBE covering IBM Chief Data Officer Summit. Brought to you by IBM. >> Welcome back, everyone. We are wrapping up a day of coverage, live coverage from theCUBE at the IBM CDO Summit on a very blustery cold day here in Boston, Massachusets. I'm your host, Rebecca Knight. I've been all day with Paul Gillan, my co-host. It's been a lot of fun co-hosting with you. >> It's been a pleasure. It's been a great day. >> Yeah. >> Great guests all day. >> Absolutely, high quality. This is not your father's IBM, not your mother's IBM, this is a very different company from when you started covering IBM way back when and to-- >> 1982. >> 82, so talk about a little bit about the changes. You grew up in an IBM town. >> I grew up in an IBM town. I grew up in an IBM world where everyone dressed the same, everyone had a set of talking points, it was a very closed, dark organization, dense organization, very little information got out. Of course the company at that time was operating under a consent to prove the justice department. Been attempting to be broken up. So they were understandably nervous. We began to see that change after IBM's crisis in the late 80s and began to open up the, began to celebrate individuals where previously individualism had been discouraged at IBM. And what you see at this conference here, I don't know, I'm always amazed when I go to IBM conferences at the differences I see between the people I meet at the conferences and sort of the corporate image that is represented which is of a company that is struggling to make some transitions. I see just lots of vibrant, intelligent, enthusiastic, forward-looking people. Very, very smart people at these conferences. I don't think that gets out enough to the IBM watchers out there. >> I would agree and what we're hearing too, about from the employees themselves talking about the soft skills that are needed to succeed here at IBM but also in all sorts of industries. I mean, our first guest, Inderpal was talking about, you asked him, "What do you need to succeed as a CDO?" Well, what are sort of the traits and ability-- >> He didn't talk about technology. >> And so it wasn't really on his list. >> He didn't talk about numbers or technology at all. He talked about managing relationships, about motivating organizations-- >> Inspiring people exactly. Exactly, so having those kinds of soft skills so necessary for success in the data world but also here at IBM. And then we've also had a lot of other people on talking about IBM as this very inclusive place where you bring your authentic self to work. I write for Harvard business reviews so these are really buzzy words right now. But really, so I don't know if a lot of employees would say that about their employer. >> And when you talk to IBMers, you hear very enthusiastic people, people who love the company, who love working here. With all the diversity, the way the company's been out front in promoting minorities, in promoting women, in all kinds of ways that it really was ahead of the game in the way he treated his workforce. You know, looking at the content of the conference, a couple things really stood out for me. I've been following this area for about five years now working at the MIT CDO event, on theCUBE for a number of years and really five years ago the CDO concept, we were asking questions like, does this job have a future, what does this job look like, what are the skills that are needed, where does it fit in the organization, is this a replacement for the CIO and conflict with the CIO, what's the responsibility, what is the job, we were asking. Really three or four years ago not hearing any of that anymore. There is a lot of unanimity of opinion. This position is important, it's critical. 90 percent of large organizations will have a CDO within the next couple of years, and the role appears to be well defined and is becoming more strategic and the issues of conflict with the CIO are largely being resolved. This is a main stream corporate C level position now, and it's amazing how quickly that's happened. Really over the last four years. >> Well and Andrew Paul said when he first started out, he was a CDO in 2006. He said, when I started out data was considered exhaust, so pollution and now we really know that it is a valuable asset. >> Now it's oil. >> Exactly, now it's gold and oil, and all the other. Yeah, no what about sort of this evolution from big data, big data was the buzz word a few years ago, now it's really all about AI. >> It is, and I've been an AI skeptic for a long time just because I've heard the term AI used for many years and when we didn't have it, when it didn't exist, I am now a believer. I believe that these systems that are being built are really exhibit signs of intelligence and we are going to much quicker in the future as Cloud comes into play, as software becomes more of an assembly process. We just had the discussion of the IBM risk analysis, supply chain risk analysis application. That was essentially assembled. It wasn't really written, it was assembled from components and it's a fantastic idea. We are going to see more of these powerful applications coming about and being built by people who are not extremely technical. So I think, I was amazed to see how the evolution of this program has gone from big data to AI. Today was all about AI and they're not talkin' trash anymore this stuff is really going to work. >> Are we cautious enough, would you think, as I mean, when you're thinking about all the industries here who are now playing in AI, sometimes scampering up the AI ladder a little too quickly because they want the shiny toys, when they really need to actually dig in deep with their data. But do you, as an analyst, where do you put-- >> Well, are we ever cautious enough with new technology? I mean look what Facebook is going through right now. We always go overboard and then we have to pull back and gaze at our navel and figure out, you know, how do we do this right. I'm sure there are a lot of mistakes being made with AI right now. Fortunately, I don't think the mistakes are being made in areas where it's going to meaningfully impact people's quality of life. It's not going to, we're not going to have medical, we had some people from the healthcare field on today. It was very clear that they take AI seriously, the role of AI seriously. I think we'll see a lot of stupid applications of AI, but that's always the way new technology is, right? So you have to experiment, you have to make some mistakes before you figure out what really works and I think we're just going through a natural cycle here. What's exciting is that these applications are the most transformational I've ever seen. >> Wow, and this is from someone who's been covering this industry for many decades. >> It's hard to maintain that wild-eyed enthusiasm after all these years, but it really is, boy, I wish I was 20 years younger, because this is going to be fun to stick around and watch how this develops. How about you? >> We got to raise our kids to grow up and be data scientists. >> I have every intention of doing that. (laughing) How about you? You were more focused on the workforce and the people side of the equation. We heard a lot about that today. >> Exactly, I mean, because frankly, what is all of this stuff doing, but making our work lives more easier, more satisfying, more interesting, less tedious, less boring, less onerous. So I think, frankly, when you put it all in terms of that is our goal is to help people do their jobs better and sometimes people's jobs are saving lives, sometimes people's jobs are, you know, helping people win at Publisher's Clearinghouse Sweepstakes. But that's what it really comes down to, so if it really is helping people do these things, I mean, it is as you said, very exciting. It's an exciting time to be looking at all of this stuff. >> And a time when I think people like you and me will increasingly be able to build these kinds of applications, because the tools are getting that easy to use. >> I hope so. I'm not that good. >> Well, maybe not you. (laughing) >> You can. My kids, definitely. Well, Paul it's been a real pleasure hosting, co-hosting this show with you. >> You too, it's been great. >> I'm Rebecca Knight for Paul Gillan. This has been theCUBE's live coverage of IBM CDO Summit, we will see you next time. (upbeat music)

>> Live from Boston, it's theCUBE covering IBM Chief Data Officer Summit. Brought to you by IBM. >> Welcome back everyone to theCUBE's live coverage of the IBM CDO Summit here in beautiful Boston, Massachusetts. I'm your host, Rebecca Knight, along with my co-host Paul Gillin. We have two guests for this segment, we have Susan Wagner, who is the VP Data Artificial Intelligence and Governance at Deutsche Telekom and Madhu Kochar, whose the Vice President Analytics Product Development at IBM. Thank you so much for coming on the show. >> Thank you. >> Happy to be here. Susan you're coming to us from Berlin, tell us a little bit about what you it's a relatively new job title and Paul was marveling before the cameras are rolling. Do you have artificial intelligence in your job title? Tell us a little bit about what you do at Deutsche Telekom. >> So we have a long history, working with data and this is a central role in the headquarter guiding the different data and artificial intelligence activities within Deutsche Telekom. So we have different countries, different business units, we have activities there. We have already use case catalog of 300,000 cases there and from a central point we are looking at it and saying, how are we able really to get the business benefit out of it. So we are looking at the different product, the different cases and looking for some help for the business units, how to scale things. For example, we have a case we implemented in one of our countries, it was about a call center to predict if someone calls the call center, if this is a problem, we would never have(laughing) at Deutsche Telekom but it could happen and then we open a ticket and we are working on it and then we're closing that ticket and but the problem is not solved, so the ticket comes again and the customer will call again and this is very bad for us bad for the customer and we did on AI project, there predicting what kind of tickets will come back in future and this we implemented in a way that we are able to use it not only in one country, but really give it to the next country. So our other business units other countries can take the code and use it in another country. That's one example. >> Wow. >> How would you define artificial intelligence? There's someone who has in your job-- (laughing) >> That's sometimes very difficult question I must admit. I'm normally if I would say from a scientific point, it's really to have a machine that works and feels and did everything like a human. If you look now at the hype, it's more about how we learn, how we do things and not about I would say it's about robotic and stuff like that but it's more how we are learning and the major benefit we are getting now out of artificial intelligence is really that we are able now to really work on data. We have great algorithm and a lot of progress there and we have the chips that develops so far that we are able to do that. It's far away from things like a little kid can do because little kid can just, you show them an apple and then it knows an apple is green. It's were-- >> A little kid can't open a support ticket. (laughing) >> Yeah, but that's very special, so in where we special areas, we are already very, very good in things, but this is an area, for example, if you have an (mumbles) who is able like we did to predict this kind of tickets this agreement is not able at the moment to say this as an apple and this is an orange, so you need another one. So we are far away from really having something like a general intelligence there. >> Madhu do I want to bring you into this conversation. (laughing) And a little bit just in terms of what Susan was saying the sort of the shiny newness of it all. Where do you think we are in terms of thinking about the data getting in the weeds of the data and then also sort of the innovations that we saw, dream about really impacting the bottom line and making the customer experience better and also the employee experience better? >> Yeah, so from IBM perspective, especially coming from data and analytics, very simple message, right? We have what we say your letter to AI. Everybody like Susan and every other company who is part of doing any digital transformation or modernization is talking about Ai. So our message is very simple, in order to get to the letter of AI, the most critical part is that you have access to data, right? You can trust your data, so this way you can start using it in terms of building models, not just predictive models but prescriptive and diagnostics. Everything needs to kind of come together, right? So that is what we are doing in data analytics. Our message is very, very simple. The innovations are coming in from the perspectives of machine learning, deep learning and making and to me that all equates to automation, right? A lot of this stuff data curation, I think you can Susan, tell how long and how manual the data curation aspects can be. Now with machine learning, getting to your latter of AI, You can do this in a matter of hours, right? And you can get to your business users, you can if your CHARM model, If your clients are not happy, your fraud, you have to detect in your bank or retail industry, it just applies to all the industry. So there is tons of innovation happening. We just actually announced a product earlier called IBM Cloud Private for Data. This is our the analytics platform which is ready with data built in governance to handle all your data curation and be building models which you can test it out, have all the DevOps and push it into production. Really, really trying to get clients like Deutsche Telekom to get their journey there faster. Very simple-- >> We've heard from many of our guests today about the importance of governance, of having good quality data before you can start building anything with it. What was that process like? How is the... what is the quality of data like at Deutsche Telekom and what work did it take to get it in that condition. >> So data quality is a major issue everywhere, because as Madhu that this is one of the essential things to really get into learning, if you want to learn, you need the data and we have in the different countries, different kind of majorities and what we are doing at the moment is that we are really doing it case by case because you cannot do everything from the beginning, so you start with one of the cases looking what to do there? How to define the quality? And then if the business asked for the next case, then you can integrate that, so you have the business impact, you have demand from the business and then you can integrate the data quality there and we are doing it really step by step because to bring it to the business from the beginning, it's very, very difficult. >> You mentioned, one of the new products that you announced just today, what are some of the-- (laughing) >> We announced it in may. >> Oh, okay, I'm sorry. >> It's okay still new. >> In terms of the other innovations in the pipeline, what I mean this is such a marvelous and exciting time for technology. What are some of the most exciting developments that you see? >> I think the most exciting, especially if I talk about what I do day out everything revolves around metadata, right? Used to be not a very sticky term, but it is becoming quite sexy all over again, right? And all the work in automatic metadata generation, understanding the lineage where the data is coming from. How easy, we can make it to the business users, then all the machine learning algorithms which we are doing in terms of our prescriptive models and predictive, right? Predictive maintenance is such a huge thing. So there's a lot of work going on there and then also one of the aspects is how do you build once and run anywhere, right? If you really look at the business data, it's behind the firewalls, Is in multicloud. How do you bring solutions which are going to be bringing all the data? Doesn't matter where it resides, right? And so there's a lot of innovation like that which we are working and bringing in onto our platform to make it really simple story make data easy access which you can trust. >> One of the remarkable things about machine learning is that the leading libraries have all been open source, Google, Facebook, eBay, others have open source their libraries. What impact do you think that has had on the speed with which machine learning is developed? >> Just amazing, right. I think that gives us that agility to quickly able to use it, enhance it, give it back to the community. That has been the one of the tenants for, I think that how everybody's out there, moving really really fast. Open source is going to play a very critical role for IBM, and we're seeing that with many of our clients as well. >> What tools are you using? >> We're using different kind of tools that depending on the departments, so the data scientists like to use our patents. (laughing) They are always use it, but we are using a lot like the Jupiter notebook, for example, to have different kind of code in there. We have in one of our countries, the classical things like thus there and the data scientists working with that one or we have the Cloud-R workbench to really bringing things into the business. We have in some business-- >> Data science experience. >> IBM, things integrated, so it it really depends a little bit on the different and that's a little bit the challenge because you really have to see how people working together and how do we really get the data, the models the sharing right. >> And then also the other challenges that all the CDOs face that we've been talking about today, the getting by in the-- >> Yes. >> The facing unrealistic expectations of what data can actually do. I mean, how would you describe how you are able to work with the business side? As a chief working in the chief data office. >> Yeah, so what I really like and what I'm always doing with the business that we are going to the business and doing really a joint approach having a workshop together like the design thinking workshop with the business and the demand has to come from the business. And then you have really the data scientists in there the data engineers best to have the operational people in there and even the controlling not all the time, but that it's really clear that all people are involved from the beginning and then you're really able to bring it into production. >> That's the term of DataOps, right? That's starting to become a big thing. DevOps was all about to agility. Now DataOps bring all these various groups together and yeah I mean that's how you we really move forward. >> So for organizations so that's both of you for organizations that are just beginning to go down the machine learning path that are excited by everything you've been hearing here. What advice would you have for them? They're just getting started. >> I think if you're just getting started to me, the long pole item is all about understanding where your data is, right? The data curation. I have seen over and over again, everybody's enthusiastic. They love the technology, but the... It just doesn't progress fast enough because of that. So invest in tooling where they have automation with machine learning where they can quickly understand it, right? Data virtualization, nobody's going to move data, right? They're sitting in bedrock systems access to that which I call dark data, is important because that is sometimes your golden nugget because that's going to help you make the decisions. So to me that's where I would focus first, everything else around it just becomes a lot easier. >> Great. >> So-- >> Do you have a best practice too? Yeah. >> Yeah. Focus on really bringing quick impact on some of the cases because they're like the management needs success, so you need some kind of quick access and then really working on the basics like Madhu said, you need to have access of the data because if you don't start work on that it will take you every time like half a year. We have some cases where we took finance department half a year to really get all that kind of data and you have to sharpen that for the future, but you need the fast equipments. You need to do both. >> Excellent advice. >> Right, well Susan and Madhu thank you so much for coming on theCUBE, it's been great having you. >> Thank you. >> Thank you. >> I'm Rebecca Knight for Paul Gillin we will have more from theCUBE's live coverage of the IBM CDO just after this. (upbeat music)

>> Live, from Boston, it's theCUBE, covering IBM Chief Data Officer Summit. Brought to you by IBM. >> Welcome back everyone to theCUBE's live coverage of the IBM CDO Summit here in Boston, Massachusetts. I'm your host Rebecca Knight, along with my co-host Paul Gillin. We're joined by Dr. Prakriteswar Santikary known as Dr Santi. He is the Vice President and Global Chief Data Officer at eResearch Technology. Thank you so much for coming back on theCUBE. >> Yeah, thank you for inviting me. >> So Dr Santi tell our viewers a little bit about eResearch Technology. You're based in Marlborough... >> Yeah, so we're in Boston, but ERT has been around since 1977 and we are a data and technology company that minimizes risks and uncertainties within clinical trial space and our customers are pharmaceutical companies, biotechnology companies, medical device companies, and where they really trust us in terms of running their clinical trials on our platform. So we have been around over 40 years, so we have seen a thing or two in the space. It's a very complex domain a very highly regulated as you know, because it's dealing with patients lives. So we take huge pride in what we do. >> We know how involved clinical trials can be long, very expensive, how are the new tools, big data impacting the cost? >> Well, that has been an age old problem within the clinical trials, usually a drug takes about eight to 12 years and costs about $2 billion from start to commercialization. So it's a very lengthy, manual and arduous process. So there are lots going on in this clinical trial domain that's tries to shorten the timeline and employing of big data technologies, modern data platform to expedite data processing, data collection from mobile devices and health technologies and all these. Artificial intelligence is playing a big role in terms of disrupting some of these domains, particularly if you see the protocol development down to patient selection, down to study design, then study monitoring. So you need to do all those things and each takes long long long time, so AI with the big data technologies is they're really making a difference. >> In what ways? >> For example, patient selection is one of the huge pin points in any clinical trial, because without patients there are no clinical trials. Particularly when you try to launch a drug, you will have to identify the patients, select the patients and not only select the patients, you have to make sure those patients stay with the clinical trials throughout the duration of the trial. So patient engagement is also a big deal. So with these big data technologies, like now you can see all this mobile health devices that patients are wearing using which you can monitor them. You can remind, send them a reminder, take your drug or you can send a text saying that there will be a clinical visit at that site come at seven o'clock, don't come at nine o'clock. So these kind of encouragement and constant feedback loop is really helping patients stay engaged. That is critical. Then matching patients with the given clinical trials is a very manual and arduous process, so that's where the algorithms is helping. So they are just cranking up real world evidence data for example claims data, prescription data and other type of genomic data and they're matching patients and the clinical trial needs. Instead of just fishing around in a big pond and find out, okay I need three patients. So go and fish around the world to get the three patients. That's why current process is very manual and these AI techniques and behind technologies and big data technologies are really disrupting this industry. >> So are the pharmaceutical companies finding that clinical trials are better today because patients are more engaged and they are getting as you said this constant reminder, take your drug, stay with us. Do you think that they are, in fact, giving them better insights into the efficacy of the drug? >> Yes because you will see their compliance rate is increasing, so because remember when they have to fill out all these diaries, like morning diaries evening diaries, when they are taking which medicine, when they are not taking. It used to be all manual paper driven, so they would forget and particularly think about a terminally ill patient, each day is so critical for them. So they don't have patience, nor do they have time to really maintain a manual diary. >> Nor do their caregivers have the time. Right. >> So this kind of automation is really helping and that is also encouraging them as well, that yeah somebody is really caring about me. We are not just a number, patient is not a number that somebody is really relating to them. So patient engagement, we have a product that specifically focuses around patient engagement. So we do all these phase one through phase four trials, one, two, three, four and then forced marketing, obviously, but through the entire process, we also do patient engagement, so that we help our customers like pharmaceutical companies and biotechnology companies so that they can run their trials with confidence. >> How about analyzing the data that you collect from the trials, are you using new techniques to gain insights more quickly? >> Yes, we are. We just recently launched a modern data platform, a data lake while we are consolidating all the data and anonymizing it and then really applying AI techniques on top of it and also it is giving us real time information for study monitoring. Like which side is not complying, with patients or not complying, so if the data quality is a big deal in clinical trials, because if the quality is good, then FDA approval, there is a chance that FDA may approve, but if the data quality is bad, forget about it, so that's why I think the quality of the data and monitoring of that trial real time to minimize any risks before they become risks. So you have to be preempted, so that's why this predictive algorithms are really helping, so that you can monitor the site, you can monitor individual patient through mHealth devices and all these and really pinpoint that, hey, your clinical trials are not going to end on time nor on budget. Because here you see the actual situation here, so, do something instead of waiting 10 years to find that out. So huge cost saving and efficiency gain. >> I want to ask about data in healthcare in general because one of the big tensions that we've talked about today is sort of what the data is saying versus what people's gut is saying and then in industry, it's the business person's gut but in healthcare it is the doctor, the caregivers' gut. So how are you, how have you seen data or how is data perceived and is that changing in terms of what the data shows that the physician about the patient's condition and what the patient needs right then and there, versus what the doctors gut is telling him that the patient needs? >> Yeah and that's where that augmentation and complementary nature, right? So AI and doctors, they're like complementing each other, So predictive algorithm is not replacing doctors the expertise, so you still need that. What AI and predictive algorithm is playing a big role is in expediting that process, so instead of sifting through manual document so sifting through this much amount of document, they would only need to do this much of document. So then that way it's minimizing that time horizon. It's all about efficiency again, so AI is not going to be replacing doctors anytime soon. We still need doctors, because remember a site is run by a primary investigator and primary investigator owns that site. That's the doctor, that's not a machine. That's not an AI algorithm, so his or her approval is the final approval. But it's all about efficiency cost cutting and bringing the drugs to the market faster. If you can cut down these 12 years by half, think about that not only are you saving lots of money, you are also helping patients because those drugs are going to get to the market six year earlier. So you're saving lots of patients in that regard as well. >> One thing that technologies like Watson can do is sort through, read millions of documents lab reports and medical journals and derive insights from them, is that helping in the process of perhaps avoiding some clinical trials or anticipating outputs earlier? >> Yes, because if you see Watson run a clinical study with Cleveland Clinic recently or Mayo Clinic I think or maybe both. While they reduce the patient recruitment time by 80%, 80%. >> How so? >> Because they sweep through all those documents, EMR results, claims data, all this data they combined-- >> Filter down-- >> Filter down and then say, for this clinical trial, here are the 10 patients you need. It's not going to recommend to who those 10 patients are but it will just tell you that, the goal is the average locations, this that, so that you just focus on getting those 10 patients quickly instead of wasting nine months to research on those 10 patients and that's a huge, huge deal. >> And how can you trust that, that is right? I mean I think that's another question that we have here, it's a big challenge. >> It is a challenge because AI is all about math and algorithm, right? So when you, so it's like, input black box, output. So that output may be more accurate than what you perceive it to be. >> But that black box is what is tripping me up here. >> So what is happening is sometimes, oftentimes, if it is a deep learning technique, so that kind of lower level AI techniques. It's very hard to interpret that results, so people will keep coming back to you and say, how did you arrive at that results? And that's where most of the, there are techniques like Machine Learning techniques that are easily interpretable. So you can convince FDA folks or other folks that here is how we've got to it, but there are a deep learning techniques that Watson uses for example, people will come and, how did you, how did you arrive at that? And it's very hard because those neural networks are multi-layers and all about math, but as I said, output may be way more accurate, but it's very hard to decipher. >> Right, exactly. >> That's the challenge. So that's a trust issue in that regard. >> Right, well, Dr. Santi, thank you so much for coming on theCUBE. It was great talking to you. >> Okay, thank you very much. Thanks for inviting. >> I'm Rebecca Knight for Paul Gillin we will have more from the IBM CDO Summit in just a little bit. (upbeat music)

live from Boston it's the cube covering IBM chief data officer summit brought to you by IBM welcome back everyone to the cubes live coverage of the IBM CDO summit here in Boston Massachusetts I'm your host Rebecca Knight and I'm joined by co-host Paul Gillan we have a guest today John Thomas he is the distinguished engineer and director at IBM thank you so much for coming returning to the cube you're a cube veteran so tell our viewers a little bit about your distinguished engineer there are only 672 in all of IBM what do you do what is your role that's a good question distinguished engineer is kind of a technical execute a role which is a combination of applying the technology skills as well as helping shape by the inscriber gene in a technical way working with clients etcetera right so it is it is a bit of a jack-of-all-trades but also deep skills in some specific areas and I love what I do so you get to work with some very talented people brilliant people in terms of shaping IBM technology and strategy products for energy that is part of it we also work very closely with clients in terms of how do you apply that technology in the context of the clients use cases we've heard a lot today about soft skills the importance of organizational people skills to being a successful chief data officer but there's still a technical component how important is the technical side what is what are the technical skills that the cdos need oh this is a very good question Paul so absolutely so navigating the organizational structure is important it's a soft skill you're absolutely right and being able to understand the business strategy for the company and then aligning your data strategy to the business strategy is important right but the underlying technical pieces need to be solid so for example how do you deal with large volumes of different types of data spread across the company how do you manage the data how do you understand the data how do you govern that data how do you then mast are leveraging the value of the data in the context of your business right so and understand deep understanding of the technology of collecting organizing and analyzing that data is needed for you to be a success for CBL so in terms of in terms of those skill sets that you're looking for and one of the things that Interpol said earlier in his keynote is that they're just it's a rare individual who truly understands the idea of how to collect store analyze curate eyes monetize the data and then also has the the soft skills of being able to navigate the organization being able to be a change agent is inspiring yeah inspiring the rank-and-file yeah how do you recruit and retain talent it seems to be a major tech expertise is not getting the right expertise in place and Interpol talked about it in his keynote which was the very first thing he did was bring in Terrence sometimes it is from outside of your company maybe you have a kind of talent that has grown up in your company maybe you have to go outside buddy God bring in the right skills together form the team that understands the technology and the business side of things and build esteem and that is essential for you to be a successful CTO and to some extent that's what Interpol has done that's what the analytic CEOs office has done a set up in my boss is the analytics EDF and he and the analytic CDO team actually engineering skills data science skills visualization skills and then put this team together which understands the how to collect govern curate and analyze the data and then apply them in specific situations a lot of talk about AI at this conference what seems to be finally happening what do you see in the field or perhaps projects that you've worked on examples of AI that are really having a meaningful business impact yeah Paul it's a very good question because you know the term AI is overused a lot as you can imagine a lot of hype around it but I think we are past that hype cycle and people are looking at how do i implement successful use cases and I stressed the word use case right in my experience these how I'm going to transform my business in one big boil the ocean exercise does not work but if you have a very specific bounded use case that you can identify the business tells you this is relevant the business tells you what the metrics for success are and then you focus your your attention your your efforts on that specific use case with the skills need for that use case then it's successful so you know examples of use cases from across the industries right I mean everything that you can think of customer-facing examples like how do I read the customers mind so when when if I'm a business and I interact with my customers can I anticipate what the customer is looking for maybe for a cross-sell opportunity or maybe to reduce the call handling time and a customer calls in to my call center or trying to segment my customer so I can do a proper promotion or a campaign for that customer all of these are specific customer facing examples there are also examples of applying this internally to improve processes capacity planning for your infrastructure can I predict when a system is likely to have an outage and or can I predict the traffic coming into my systems into my infrastructure and provision capacity that on-demand so all these are interesting applications of AI in the enterprise so when you're trying I mean one of the things we keep hearing is that we need data to tell a story the data needs to the data needs to be compelling enough so that the people the data scientists get it but then also that the other kinds of business decision makers get it - so what are sort of the best practices that have emerged from your experience in terms of being able to for your data to tell the story that you wanted to tell yeah well I mean if the pattern doesn't exist in the data then no amount of fancy algorithms can help you know so and sometimes it's like searching for a needle in a haystack but assuming I guess the first step is like I said what is the a use case once you have a clear understanding of your use case and success metrics for the use case do you have the data to support that use case so for example if it's fraud detection do you actually have the historical data to support the fraud use case sometimes you may have transactional data from your your transaction data from your current or PI systems but that may not be enough you may need to augment it with external data third party data may be unstructured data that goes along with the transaction data so question is can you identify the data that is needed to support the use case and if so can I do is that data clean is that is that data do you understand the lineage of the data who has touched and modified the data who owns the data so that I can then start building predictive models and machine learning be planning models with that data so use case do you have the data to support the use case do you understand how the data reached you then comes the process of applying machine learning algorithms and deep learning algorithms against that data one of the risks of machine learning and particularly deep learning I think is it becomes kind of a black box and people can fall into the trap of just believing what comes back regardless of whether the algorithms are really sound or the data is somewhat what is the responsibility of data scientists to sort of show their work yeah Paul this is a fascinating and not completely solved area right so bias detection can I explain how my model behaved can I ensure that the models are fair in their predictions so there's a lot of research lot of innovation happening in the space iBM is investing a lot in the space we call trust and transparency being able to explain a model it's got multiple levels to it you need some level of AI governments itself so just like we talked about data governance there is the notion of AI governance which is what version of the model was used to make a prediction what were the inputs that went into that model what were the decisions that are that what were the features that were used to make a certain prediction what was the prediction and how did that match up with ground truth you need to be able to capture all that information but beyond that we have got actual mechanisms in place that IBM Research is developing to look at bias detection so pre-processing during execution post-processing can I look for bias in how my models behave and do I have mechanisms to mitigate that so one example is the open source Python library called AI F 360 that comes from IBM's research on its contributor to the open source community you can look at there are mechanisms to look at bias and and and provide some level of bias mitigation as part of your model building exercises and is the bias mitigation does it have to do with and I'm gonna use an IBM term of art here at the human in the loop I mean is how much are you actually looking at the humans that are part of this process humans are at least at this point in time humans are very much in the loop this this notion of P or AI where humans are completely outside the loop is we're not there yet so very much something that the system can it provide a set of recommendations can it provide a set of explanations in can someone who understands the business look at it and make corrective take corrective action as needed there has been however to Rebecca's point some prominent people including Bill Gates who have have speculated that AI could ultimately be a negative for humans are what is the responsibility of companies like IBM to ensure that humans are kept in the loop I think at least at this point IBM's V was humans are an essential part of AI in fact we don't even use the term artificial intelligence that much we call it augmented intelligence where the system is presenting a set of recommendations expert advice to the human who can then make a decision so for example you know my team worked with a prominent healthcare provider on you know models for predicting patient death death in in the case of sepsis sepsis onset this is we're talking literally life and death decisions being made and this is not something that you can just automate and throw it into a magic black box and have a decision be made right so this is absolutely a place where people with deep domain knowledge are supported are augmented with with AI to make better decisions that's where that's where I think we are today as to what will happen five years from now I can't predict that yet the role so you are helping doctors make these decisions not just this is what the computer program says about this patients symptoms here but this is really you're helping the doctor make better decisions what about the doctors gut and the ease into his or her intuition too I mean what is what is the role of that in the future I think it goes away I mean I think the intuition really will be trumped by data in the long term because you can't argue with the facts much as some some people do these days the perspective on that is there will there all should there always be a human on the front lines who is being supported by the backend or would would you see a scenario where an AI is making decisions customer-facing decisions that are really are life and death so I think in the consumer industry I can definitely see AI making decisions on its own right so you know if let's say a recommender system which says you know I think you know John Thomas bought these last five things online he's likely to buy this other thing let's make an offer team you know I don't even in the loop for no harm it's it's it's it's pretty straightforward it's already happening in a big way but when it comes to some of these mortgage yeah about that one even that I think can be can be automated can be automated if the thresholds are said to be what the business is comfortable with where it says okay about this probability level I don't really need a human to look at this but and if it is below this level I do want someone to look at this that's you know that is relatively straightforward right but if it is a decision about you know life-or-death situations or something that affects the the very fabric of the business that you are in then you probably want to domain expert to look at it and most enterprises enterprise use cases will for lean towards that category these are big questions they're hard questions are questions yes well John thank you so much oh absolutely thank you we've really had a great time with you yeah thank you for having me I'm Rebecca night for Paul Gillen we will have more from the cubes live coverage of IBM CDO here in Boston just after this