>> From Union Square in downtown San Francisco, it's theCUBE, covering PagerDuty Summit '18. Now here's Jeff Frick. >> Hey welcome back, everybody, Jeff Frick here with theCUBE. We're at PagerDuty Summit 2018 at the Westin St. Francis in Union Square, San Francisco. Great event, 900 people, we're excited to be here, it's our second year, and now we get to talk to some customers, which we are always excited to do. And our next guest is Alan Alderson. He is the Director of IT Ops for William Hill. Great to see you. >> Afternoon, it's great to be here. >> Absolutely, so for people that aren't familiar with William Hill, what are you guys all about? >> So William Hill offer customers opportunities to place bets on sporting events, presidential elections, snow at Christmas, you name it. We present about a million opportunities every week for customers to have a bet on. >> A million opportunities a week? >> Yeah, so picking on football matches, you know the game of the ramble. So we have opportunities for people to bet playing up to the game, and then once the game kicks off, we transition into what's called in play, so people can then place a bet on who's going to score the next goal, and about another 120 markets within that one game whilst the game's in play. >> Wow, so what's the average duration of the window to put a bet down? >> So generally leading up to the match it's as much time as you want, as soon as the markets are out there you can place the bet before the game kicks off. >> Okay. >> But once the game kicks off, you can, right up until about towards the last few minutes of the game, there'll be markets available to have a bet on. >> Okay, and then what percentage is kind of things that I would guess easily, like sporting events or those types of things, versus you know, whether it's going to snow or not? >> Well we provide the opportunities on the website, so you can have a look and, you know it's snow on Christmas day is a popular bet. People do their research, and they like to have a bet on it. There is a lot of novelty bets. There used to be, you know, life being found on Mars, Elvis being found, et cetera. So there's a lot >> Still taking action on Elvis? >> I don't think so. >> I thought we'd find him. So we're here at PagerDuty Summit. What are you doing here at PagerDuty Summit? >> So I've just come back from a stint in Australia, working for the William Hill business over there. So we introduced PagerDuty over there to help out with just getting the right message out to the right support teams quickly. So we deployed it out there, and we just brought it in to do infrastructure to start with but once we deployed it, it's a bit of a ripple effect. So it was like dropping a pebble into a pool, the ripple effect, and everybody, they seem to be doing all right over there, they use it now for the support models and so those sorts of questions. It's very quick how the other teams decided to latch onto PagerDuty as well. So I since moved back to the UK. So I moved back in January, took on this role back in the Leeds office in the north of England, and one of the first things I said is, guys, start having a look at PagerDuty, we've deployed it successfully in Australia, so let's have a look at what it can do for us. And so management works at William Hill. So I'm not trying to fix anything that's broken. So, it works. But what we can do is increase its speed of how we deal with things. So there's a lot of manual tasks in there that PagerDuty will come in and automate. It will take the pressure off the incident analysts 'cause, you know if there's an incident at two o'clock in the morning, we have 24 by seven business, so if there's an incident overnight, we've got to get on it and start fixing, resolving the incident. And if there's one guy who's trying to call out a number of responders, calling out a duty manager, trying to get comms out, it's a lot of pressure on one person to do that, and when there's pressure mistakes happen. I want PagerDuty to take away the possibility of the mistakes, take the pressure of the incident analyst, so they can focus on resolving the incident and getting service back to our customers as quickly as possible. >> I'm curious though when you said that other people and other groups saw PagerDuty in action. What were some of the other tasks that were not the primary tasks that you brought it in, where people saw value and are implementing it for some other types of activities? >> So initially when we put it in, we put it in purely for service. So for looking at the CPU disk and memory alerts. And we were getting our acknowledgements down from minutes to seconds in Australia. So the other teams are watching in, and within their applications there was a lot of alerts just landing as an email and not getting actioned upon very quickly. So we brought PagerDuty in, they said, can this help out in this space, and they started integrating it into their applications. So through hooking it into their applications they could get the alerts directly from PagerDuty, rather than it going through knocks and service decks et cetera, so it's just a quicker response and get 'em onto the issue quicker. >> And do you have it integrated in with some of your other development tools so it's just kind of part of whole process, or is it more kind of standalone notification system? >> It was integrated straight into ServiceNow and PagerDuty. PagerDuty would integrate with ServiceNow, raise the ticket, and then the things started moving. But the big win was getting the guys the call straight away as that alert happened. Otherwise you're relying on people watching screens, watching queues, waiting for that to happen, and then make the call. So if the call's gone straight to the engineer, he's on it immediately. >> Right, right, right. So what are some of your impressions here? Seeing kind of the ecosystem, what's behind PagerDuty, some great keynotes earlier today, really in terms of, again, the mission it sounds like it's very much in line with what you're trying to do, which is to help teams be more effective. >> Yeah, and what I like about PagerDuty is their passion. You just get a sense of urgency about this place, and you get a sense of passion and commitment, and they want to help people out, and that's what's drawn me to PagerDuty. The guys I worked with in Australia, the guys I worked with in the UK, they just can't do enough for you, and they want to help you succeed as well. You know, you deals with some companies that, they just want to sell you something and move on. These guys are, you know, they look after you, they work with you and they make sure that you're getting the value out of their product. >> It's a pretty interesting culture, 'cause when I talked to Jennifer Tejada a couple of years ago, I used to tease her, I'm like, nobody here knows what a pager is, right? Nobody was born when pagers were >> I had one. >> the rage. >> You had one, yeah, I had one. Shell Oil upside down, I think it says hello, I can't remember, I have to check that. But it's an interesting, there's kind of culture around what a pager represents, and the work that they have duty in there as well, which is a very different kind of level of responsibility when you are the person with the pager on, and that seems to have really carried forward in the way that they deliver the services. >> Yeah, yeah. I mean, on-call has people running, doesn't it? When people, you know when they join a job and go, "Oh you might be expected to be on call", they run a mile, and they think that's not for me. But as we go down more of a DevOps transformation and we get a lot more down the we code it, we own it model, I think it'll change people's perceptions of being on call and just doing the right thing for the business, rather thank, you know, delivering something and expecting the Ops team to fix it all the time and call out the developers at a third line. We should be, we are heading towards being a team, where the alerts go to the right people at the right time, and we get issues resolved as soon as possible. >> Right. I'd just love to get your take on, a lot of talk about digital transformation, and the modernization of IT, and kind of expected behavior on apps going on. You're right in the middle of it. >> Massively in the middle of it. >> Massively in the middle of it, right. I'm sure, what percentage of your bets come in via mobile versus... >> On the digital platform, over 56%. >> A lot, right, a lot. >> And we've got, just said in the last session we had is, we've got competition. So if our app isn't performing, it isn't quick, or it's down, people will go elsewhere. They've got options, they've got choices, and they'll just go elsewhere. And the challenge is getting those customers back. We want to have a stack that just is available and is performing, so we don't drive customers away, or we make sure that things are available at peak times, so when they are wanting to bet on the Super Bowl, the Grand National, the three o'clock kickoffs on a Saturday afternoon in the UK, it's available for them and people can get the bet on as quickly as possible. >> Right. So do you have all your own infrastructure, or do you leverage public cloud? I'm just thinking as you're talking about Super Bowl and some of these other big events, you must have just crazy big spikes. >> You know we've, in the UK it's all on-premise, so we've got to build an infrastructure to cope with that one day of the year, which is Grand National. In the US, we've just opened up in New Jersey. The front end of that stack is in AWS, so we can scale, so when Super Bowl does turn round next January, February, we should be able to scale with the load. >> Right, last question before I let you go. What are your priorities next? What are some of the things that you're working on with your team, to kind of stay at the leading edge of this very competitive space? >> Yeah we're heading into AWS. So we're looking to move into Amazon next year, start migrating some applications in there, and we're looking to get some applications in there the back end of this year, but migrate the existing apps from the start of next year. We're going through a DevOps transformation. We've been doing an agile transformation as well over the last 12 to 18 months, so there's a huge amount of digital transformation going on at William Hill at the moment. It's a very, very exciting place to be. The US expansion, the place has just gone mad, you know. There's a lot going on, it's just a great place to be. >> Yeah, I mean significant changes obviously in the US attitude, I think you guys are a little more progressive on that side of the Atlantic. Big changes happening here. >> 14th of May was a big day, PASPA being repealed has opened up the betting opportunities in any state that wants to regulate. And we are leading the way in that charge at the moment, so it's very exciting. >> All right, well I'm going to let you go so you can get some sleep, 'cause I'm sure you're a very busy man. Alan, thanks for stopping by. >> Thank you very much. >> All right, he's Alan, I'm Jeff, you're watching theCUBE, we're at PagerDuty Summit 2018, thanks for watching.

(upbeat electronic music) >> Announcer: Live from Vancouver, Canada, it's theCUBE, covering OpenStack Summit North America 2018. Brought to you by Red Hat, the OpenStack Foundation, and its ecosystem partners. >> Welcome back to The Cube's coverage of OpenStack Summit 2018 in Vancouver. I'm Stu Miniman with my co-host John Troyer. Happy to welcome back a company we've spoken to a few times at events, Paddy Power Betfair. First time guest coming to us from across the pond, Dave Buckley who is the automation engineer with Paddy Power Betfair, thanks for joining us. >> Thank you for having me. >> Alright, so first of all, you've been to a couple summits and we've talked to Paddy Power about OpenStack. Before we get into your specific implementation, tell us about your experience here this week and any compare, contrast to previous years. >> Yeah so I'm very lucky, I got to come to the previous two summits in North America. I guess what I've enjoyed this week, it's kind of a slight tilt towards, it's away from being purely OpenStack, kind of towards this open infrastructure kind of thing, 'cause like I said, especially last year in Boston, Q and NEs was becoming a big thing. Yeah, and kind of, the OpenStack Foundation becoming kind of more, not that it wasn't before, but more community-based and being part of the ecosystem. So, yeah, I think it's been quite interesting seeing that. >> Not to put words in your mouth but, it was even, the last year or two, it's more aware of some of the complimentary things and adding pieces. You know, we had, one of the interviews we did this week was person who's the SIC lead for the Kubernetes stuff, that sits under another Foundation, things like that. Yeah, exactly. It's been quite interesting this week, I guess, sort the Kata Container project, which wasn't something I'd been aware of before Monday morning basically. I remember we were sitting in the keynotes, and they were like, you can have this container-like thing which has all the speed of a container, but it's as secure as a BM. And you're thinking, how, how is that even possible? So I've really enjoyed, I got to go to one of the sessions yesterday, one of the technical introductions on that. >> Yeah, I always love, there's certain things where, okay, this is what I'm going to do with my schedule, and turns into, this got announced, or I didn't know about this, and you knew, blow up my schedule, let me change everything else. Yeah, exactly, I think you always, you can't, you have to be flexible, right? Adaptable, and as the week goes on you just go to what you think is interesting. >> John: So Dave, you and your company have been working with OpenStack for quite a while. >> Dave: Yeah. >> And you obviously run a system that needs to be stable. Right, needs to, you take care of betting and people's money. >> Dave: Exactly. >> So that needs to be solid. But I understand you recently went though an upgrade and have some experiences talking about that? Can you talk a little bit about where you are with your OpenStack implementation and that sort of migration? >> Sure. So, I guess it's about three years ago, it was Betfair at the time, so this was before the merger of the two companies. So Betfair started using OpenStack, and I think it was actually the last time the summit was here, in Vancouver. So a couple of my colleagues who were kind of the technical leads at the time. Steve Armstrong and Steve Perera, they flew out here, to kind of get a feel for OpenStack, what it was, talk to people who'd had experiences with it. I actually think that conference back then was very informative of what the platform today now looks like. So some of the conversations they had there with people like New Age Networks and Arista, which we used for the switching, but conversations they had there kind of ended up being now what we're using in production. I guess over the past couple of years, so the big thing that happened obviously was this merger between Paddy Power and Betfair, following that they had an exercise which they called the single customer platform, which is annoyingly, for a sys-admin guy, kind of like me, they, it's always been abbreviated to SCP, but you have to ignore that. So that was to kind of consolidate and integrate the Paddy Power and Betfair co-bases and put it on a single platform, which was our OpenStack and Nuage platform. So that kind of completed in January this year, so that's live, so basically the Paddy Power sports book has an entirely new website, all running on OpenStack. A lot quicker and more efficient then the previous version. So that's been a real success. And as part of that, I should say that stability is really vital, so kind of in our business. If the site is down we don't make any money, and if it happens during a big sporting event you have a big problem. >> Do you have a metric around that? What a minute or an hour of down time would be? >> So I guess it always depends, so the nature of our traffic is very spikey. So obviously when you have a big, it's on a Saturday in Europe, the football, soccer, maybe I should say, is like a very big deal. >> We have a global audience, football's okay. >> I'll stick with football then. >> We were all watching the royal wedding. >> I don't want to talk about that. The football, if you, we just get peak traffic on that day. And, even within the year, there's a thing called the Grand National, which is a big event in the UK, big horse racing, I guess like the Kentucky Derby. It's kind of when we get our maximum traffic in the year. Yeah, you always need to be prepared for that. So one of the things as you mentioned, we kind of look into upgrade OpenStack from Kilo to Newton. So we've been on Kilo from the start. We're using Red Hat's distribution of OpenStack, so what Red Hat offer is this, they have like every three releases I think it is? They have this long release life-cycle. So that's kind of the reason we're going to Newton, cause we have kind of the, then the support will go to 2021. [Stu] - But if I remember, it's Red Hat the OpenStack Platform 10. >> Dave: Yeah. >> And 13 is going to be queened as their next one that's going to be released. >> Exactly, so I think they just announced that this week, right? So I think at some point in the next year or two we'd be going to queens. >> How do you determine when you make that jump and anything around the upgrade process, you know, good and bad that you could share. >> Dave: Yeah, so I guess going from, we were overdue an upgrade in this case, Kilos, you know, pretty old now. What we're lucky that we can do is because we have Nuage, it's like an external SDM provider, so the entire data plane is controlled by Nuage, and you can kind of plug as many OpenStacks as you like really into Nuage, and you offload all the networking to Nuage. So what's that's allowed us to do is basically we'd have had a lot of trouble if we'd had to do an in place upgrade, so I've actually been to one of the groups this week, quite a lot of people were talking about upgrades and just like all the nightmares it's caused. I know it's getting better as like the releases come out, but what we were able to do is kind of building new, an entirely new OpenStack cloud on the side of, so we've kind of turned it kind of an immutable OpenStack, so your OSB 7 cloud is there, we built this new OSB 10. But they're both circ into the same networking, so the same Nuage SDN. And the way our developers deploy their applications, I guess you want to see this in more detail, we've done presentations at these summits in the past, but kind of in short, every deployment we do immutable deployments as well, so for every deployment we'll create a new subnet within Nuage, and kind of do rolling update of your VMs that are on that new subnet into like a VIP which is kind of where the constant is, so all the traffic's come in to that VIP then you just flip things in and out below it when you do a deployment, so what that basically means is from a developers point of view, when they're migrating from OSB 7 to OSB 10 they'll essentially spin up new networks and new VMs in OSB 10 and that deployment pipeline will kind of just seamlessly, everything else will stay the same because the networking doesn't change. So we don't have to have any downtime on the data plane or the control plane. Which is really beneficial for us 'cause the way, I guess this is I'll just describe the way developers do deployments like we rely heavily on the OpenStack API being available. You pay a cost in that you, so you need extra hardware to do that I guess, but yeah we found it is something that's worked for us. >> John: Anything else with the networking and specifically that you all are running, the load balancing or resiliency that you need to have for your apps? >> Dave: Yeah so one of the things was, so it's kind of another problem there were trying to solve with this whole project, this new OpenStack platform is that historically Betfair, as it was at the time, had always run out of a single data-center. But we had another site, but it was mainly kind of a development environments right in there. So the company thought why don't we just have, we should just have both DCs for resiliency, try and run things in like an active-active configuration. Which is fine for external customer facing applications where we've had an external load balance server that can point traffic between the two DCs. But then the question is what do you do with internal apps? So this is what led us to use Avi Networks, which is kind of a cloud native load balancing technology, so we've been using to provide like GSLB internal laps, so basically we'll load balance traffic between the two data-centers so it gets deployed within your OpenStack environment, has a really neat integration with Nuage, the Nuage SDN layer, and will resolve you to whichever data-center is appropriate at that time. So if you have a full data-center outtage, you should be able to go "Okay, point stuff over there". >> John: So it makes you and the networking team or the IT team into the heroes not the villains, you're usually the people saying "No" or "We can't do that". >> I guess so, I guess so yeah you're probably right. It's cool technology though. I guess that we're very lucky and that we're given the opportunity by the people at the company to experiment with new things, so even though we're about stability but we're also about trying to push things forward in terms of what technology to use. >> Stu: Dave I'm curious how kind of the hybrid or multi-cloud type of environments fit into what you're doing today, give us the update there. >> Dave: Yeah so that's something very in our radar at the moment I guess it's, yeah it's what everybody's doing, looking to how you can have this hybrid cloud model. So I think, going back three years again, at that time, being like an online betting company, it's a highly regulated business and only at that point it was really possible to kind of put some of this stuff into the public cloud, it seems like things have come a long way, so it's something we're looking at at the moment, we're evaluating different solutions, different vendors like the Googles, AWSs, and seeing or even like some OpenStack public clouds and seeing maybe how could we migrate some workloads out into the public cloud, how do we want to that, to give us more resiliency, and also as I was saying about our spiky traffic, it just makes a lot of sense to be able to say burst out into whichever public cloud vendor on a Saturday when the football's on to deal with that peak load. So it's something we're very much looking at at the moment. But yeah no formal decisions as of yet. Unless they've done something while I've been away. >> John: With containers here at the show, lots of different threads right? Containers, Edge, the OpenDev track, things like that. Anything else, we've talked about Kata, anything else that came up that was interesting here that you just watch Kubernetes and container track as well? >> Dave: So I guess in terms of containers it's, sitting in the Keynotes on Monday you would, if you weren't watching if you were just listening, you probably wouldn't know you were at an OpenStack Summit right since there's as much Kubernetes container stuff as there is OpenStack. It's interesting so we've kind of been doing... Again, similar to the public cloud conversation, it's something that's very relevant to us at the moment, we've done kind of a few proof-of-concept ideas, evaluating different solutions, so we have like running a Cube cluster ourself, obviously we have a strong relationship with Red Hat that we've kind of explored to using OpenShift maybe, and then come the networking layer you can integrate with Nuage which would be really cool for us so it'll allow us to do kind of the all the networking, access control mechanisms as we do for our virtual machines. And again this is also something in the whole public cloud conversation is well if wanted to containers in the public cloud as well like you have all the different offerings, would we want to run our own, in like an AWS or something? Or maybe go to someone like Google where you have that supported self-service model I suppose. But yeah at the moment it's kind of at those stages so I think Steve did a presentation on the Kubernetes stuff like a PCO we done at the last Summit. But yeah still at the moment still want to make some firm decisions about which direction we're going to go but a lot of the developers a very keen for this and obviously for guys like us we all know the value of it so I think at the moment because we had that focus on stability we should now have a period of time where we're able to kind of look at all this stuff a bit more, hopefully get some container solutions into production which would be awesome. >> Stu: Dave Buckley we really appreciate you giving us the update, love to be able to do some of those longitudinal case studies as to where you've been where you're going, what you're thinking about. Be sure to check out thecube.net, you can actually search for Patty Power Betfair, see some of those previous interviews from Dave's peers. Loads more interviews there as well as all the shows we're going to be at in the future where hope you come by and say "Hi". For John Troyer I'm Stu Miniman, thanks so much for watching theCUBE. >> (electro-dance music) >> (soft piano)