>> From "theCube" Studios in Palo Alto in Boston bringing you data-driven insights from "theCube" and ETR. This is "Breaking Analysis" with Dave Vellante. >> As an independent pure play company, Palo Alto Networks has earned its status as the leader in security. You can measure this in a variety of ways. Revenue, market cap, execution, ethos, and most importantly, conversations with customers generally. In CISO specifically, who consistently affirm this position. The company's on track to double its revenues in fiscal year 23 relative to fiscal year 2020. Despite macro headwinds, which are likely to carry through next year, Palo Alto owes its position to a clarity of vision and strong execution on a TAM expansion strategy through acquisitions and integration into its cloud and SaaS offerings. Hello and welcome to this week's "Wikibon Cube Insights" powered by ETR and this breaking analysis and ahead of Palo Alto Ignite the company's user conference, we bring you the next chapter on top of the last week's cybersecurity update. We're going to dig into the ETR data on Palo Alto Networks as we promised and provide a glimpse of what we're going to look for at "Ignite" and posit what Palo Alto needs to do to stay on top of the hill. Now, the challenges for cybersecurity professionals. Dead simple to understand. Solving it, not so much. This is a taxonomic eye test, if you will, from Optiv. It's one of our favorite artifacts to make the point the cybersecurity landscape is a mosaic of stovepipes. Security professionals have to work with dozens of tools many legacy combined with shiny new toys to try and keep up with the relentless pace of innovation catalyzed by the incredibly capable well-funded and motivated adversaries. Cybersecurity is an anomalous market in that the leaders have low single digit market shares. Think about that. Cisco at one point held 60% market share in the networking business and it's still deep into the 40s. Oracle captures around 30% of database market revenue. EMC and storage at its peak had more than 30% of that market. Even Dell's PC market shares, you know, in the mid 20s or even over that from a revenue standpoint. So cybersecurity from a market share standpoint is even more fragmented perhaps than the software industry. Okay, you get the point. So despite its position as the number one player Palo Alto might have maybe three maybe 4% of the total market, depending on what you use as your denominator, but just a tiny slice. So how is it that we can sit here and declare Palo Alto as the undisputed leader? Well, we probably wouldn't go that far. They probably have quite a bit of competition. But this CISO from a recent ETR round table discussion with our friend Eric Bradley, summed up Palo Alto's allure. We thought pretty well. The question was why Palo Alto Networks? Here's the answer. Because of its completeness as a platform, its ability to integrate with its own products or they acquire, integrate then rebrand them as their own. We've looked at other vendors we just didn't think they were as mature and we already had implemented some of the Palo Alto tools like the firewalls and stuff and we thought why not go holistically with the vendor a single throat to choke, if you will, if stuff goes wrong. And I think that was probably the primary driver and familiarity with the tools and the resources that they provided. Now here's another stat from ETR's Eric Bradley. He gave us a glimpse of the January survey that's in the field now. The percent of IT buyers stating that they plan to consolidate redundant vendors, it went from 34% in the October survey and now stands at 44%. So we fo we feel this bodes well for consolidators like Palo Alto networks. And the same is true from Microsoft's kind of good enough approach. It should also be true for CrowdStrike although last quarter we saw softness reported on in their SMB market, whereas interestingly MongoDB actually saw consistent strength from its SMB and its self-serve. So that's something that we're watching very closely. Now, Palo Alto Networks has held up better than most of its peers in the stock market. So let's take a look at that real quick. This chart gives you a sense of how well. It's a one year comparison of Palo Alto with the bug ETF. That's the cyber basket that we like to compare often CrowdStrike, Zscaler, and Okta. Now remember Palo Alto, they didn't run up as much as CrowdStrike, ZS and Okta during the pandemic but you can see it's now down unquote only 9% for the year. Whereas the cyber basket ETF is off 27% roughly in line with the NASDAQ. We're not showing that CrowdStrike down 44%, Zscaler down 61% and Okta off a whopping 72% in the past 12 months. Now as we've indicated, Palo Alto is making a strong case for consolidating point tools and we think it will have a much harder time getting customers to switch off of big platforms like Cisco who's another leader in network security. But based on the fragmentation in the market there's plenty of room to grow in our view. We asked breaking analysis contributor Chip Simington for his take on the technicals of the stock and he said that despite Palo Alto's leadership position it doesn't seem to make much difference these days. It's all about interest rates. And even though this name has performed better than its peers, it looks like the stock wants to keep testing its 52 week lows, but he thinks Palo Alto got oversold during the last big selloff. And the fact that the company's free cash flow is so strong probably keeps it at the one 50 level or above maybe bouncing around there for a while. If it breaks through that under to the downside it's ne next test is at that low of around one 40 level. So thanks for that, Chip. Now having get that out of the way as we said on the previous chart Palo Alto has strong opinions, it's founder and CTO, Nir Zuk, is extremely clear on that point of view. So let's take a look at how Palo Alto got to where it is today and how we think you should think about his future. The company was founded around 18 years ago as a network security company focused on what they called NextGen firewalls. Now, what Palo Alto did was different. They didn't try to stuff a bunch of functionality inside of a hardware box. Rather they layered network security functions on top of its firewalls and delivered value as a service through software running at the time in its own cloud. So pretty obvious today, but forward thinking for the time and now they've moved to a more true cloud native platform and much more activity in the public cloud. In February, 2020, right before the pandemic we reported on the divergence in market values between Palo Alto and Fort Net and we cited some challenges that Palo Alto was happening having transitioning to a cloud native model. And at the time we said we were confident that Palo Alto would make it through the knot hole. And you could see from the previous chart that it has. So the company's architectural approach was to do the heavy lifting in the cloud. And this eliminates the need for customers to deploy sensors on prem or proxies on prem or sandboxes on prem sandboxes, you know for instance are vulnerable to overwhelming attacks. Think about it, if you're a sandbox is on prem you're not going to be updating that every day. No way. You're probably not going to updated even every week or every month. And if the capacity of your sandbox is let's say 20,000 files an hour you know a hacker's just going to turn up the volume, it'll overwhelm you. They'll send a hundred thousand emails attachments into your sandbox and they'll choke you out and then they'll have the run of the house while you're trying to recover. Now the cloud doesn't completely prevent that but what it does, it definitely increases the hacker's cost. So they're going to probably hit some easier targets and that's kind of the objective of security firms. You know, increase the denominator on the ROI. All right, the next thing that Palo Alto did is start acquiring aggressively, I think we counted 17 or 18 acquisitions to expand the TAM beyond network security into endpoint CASB, PaaS security, IaaS security, container security, serverless security, incident response, SD WAN, CICD pipeline security, attack service management, supply chain security. Just recently with the acquisition of Cider Security and Palo Alto by all accounts takes the time to integrate into its cloud and SaaS platform called Prisma. Unlike many acquisitive companies in the past EMC was a really good example where you ended up with a kind of a Franken portfolio. Now all this leads us to believe that Palo Alto wants to be the consolidator and is in a good position to do so. But beyond that, as multi-cloud becomes more prevalent and more of a strategy customers tell us they want a consistent experience across clouds. And is going to be the same by the way with IoT. So of the next wave here. Customers don't want another stove pipe. So we think Palo Alto is in a good position to build what we call the security super cloud that layer above the clouds that brings a common experience for devs and operational teams. So of course the obvious question is this, can Palo Alto networks continue on this path of acquire and integrate and still maintain best of breed status? Can it? Will it? Does it even have to? As Holger Mueller of Constellation Research and I talk about all the time integrated suites seem to always beat best of breed in the long run. We'll come back to that. Now, this next graphic that we're going to show you underscores this question about portfolio. Here's a picture and I don't expect you to digest it all but it's a screen grab of Palo Alto's product and solutions portfolios, network cloud, network security rather, cloud security, Sassy, CNAP, endpoint unit 42 which is their threat intelligence platform and every imaginable security service and solution for customers. Well, maybe not every, I'm sure there's more to come like supply chain with the recent Cider acquisition and maybe more IoT beyond ZingBox and earlier acquisition but we're sure there will be more in the future both organic and inorganic. Okay, let's bring in more of the ETR survey data. For those of you who don't know ETR, they are the number one enterprise data platform surveying thousands of end customers every quarter with additional drill down surveys and customer round tables just an awesome SaaS enabled platform. And here's a view that shows net score or spending momentum on the vertical axis in provision or presence within the ETR data set on the horizontal axis. You see that red dotted line at 40%. Anything at or over that indicates a highly elevated net score. And as you can see Palo Alto is right on that line just under. And I'll give you another glimpse it looks like Palo Alto despite the macro may even just edge up a bit in the next survey based on the glimpse that Eric gave us. Now those colored bars in the bottom right corner they show the breakdown of Palo Alto's net score and underscore the methodology that ETR uses. The lime green is new customer adoptions, that's 7%. The forest green at 38% represents the percent of customers that are spending 6% or more on Palo Alto solutions. The gray is at that 40 or 8% that's flat spending plus or minus 5%. The pinkish at 5% is spending is down on Palo Alto network products by 6% or worse. And the bright red at only 2% is churn or defections. Very low single digit numbers for Palo Alto, that's a real positive. What you do is you subtract the red from the green and you get a net score of 38% which is very good for a company of Palo Alto size. And we'll note this is based on just under 400 responses in the ETR survey that are Palo Alto customers out of around 1300 in the total survey. It's a really good representation of Palo Alto. And you can see the other leading companies like CrowdStrike, Okta, Zscaler, Forte, Cisco they loom large with similar aspirations. Well maybe not so much Okta. They don't necessarily rule want to rule the world. They want to rule identity and of course the ever ubiquitous Microsoft in the upper right. Now drilling deeper into the ETR data, let's look at how Palo Alto has progressed over the last three surveys in terms of market presence in the survey. This view of the data shows provision in the data going back to October, 2021, that's the gray bars. The blue is July 22 and the yellow is the latest survey from October, 2022. Remember, the January survey is currently in the field. Now the leftmost set of data there show size a company. The middle set of data shows the industry for a select number of industries in the right most shows, geographic region. Notice anything, yes, Palo Alto up across the board relative to both this past summer and last fall. So that's pretty impressive. Palo Alto network CEO, Nikesh Aurora, stressed on the last earnings call that the company is seeing somewhat elongated deal approvals and sometimes splitting up size of deals. He's stressed that certain industries like energy, government and financial services continue to spend. But we would expect even a pullback there as companies get more conservative. But the point is that Nikesh talked about how they're hiring more sales pros to work the pipeline because they understand that they have to work harder to pull deals forward 'cause they got to get more approvals and they got to increase the volume that's coming through the pipeline to account for the possibility that certain companies are going to split up the deals, you know, large deals they want to split into to smaller bite size chunks. So they're really going hard after they go to market expansion to account for that. All right, so we're going to wrap by sharing what we expect and what we're going to probe for at Palo Alto Ignite next week, Lisa Martin and I will be hosting "theCube" and here's what we'll be looking for. First, it's a four day event at the MGM with the meat of the program on days two and three. That's day two was the big keynote. That's when we'll start our broadcasting, we're going for two days. Now our understanding is we've never done Palo Alto Ignite before but our understanding it's a pretty technically oriented crowd that's going to be eager to hear what CTO and founder Nir Zuk has to say. And as well CEO Nikesh Aurora and as in addition to longtime friend of "theCube" and current president, BJ Jenkins, he's going to be speaking. Wendy Whitmore runs Unit 42 and is going to be several other high profile Palo Alto execs, as well, Thomas Kurian from Google is a featured speaker. Lee Claridge, who is Palo Alto's, chief product officer we think is going to be giving the audience heavy doses of Prisma Cloud and Cortex enhancements. Now, Cortex, you might remember, came from an acquisition and does threat detection and attack surface management. And we're going to hear a lot about we think about security automation. So we'll be listening for how Cortex has been integrated and what kind of uptake that it's getting. We've done some, you know, modeling in from the ETR. Guys have done some modeling of cortex, you know looks like it's got a lot of upside and through the Palo Alto go to market machine, you know could really pick up momentum. That's something that we'll be probing for. Now, one of the other things that we'll be watching is pricing. We want to talk to customers about their spend optimization, their spending patterns, their vendor consolidation strategies. Look, Palo Alto is a premium offering. It charges for value. It's expensive. So we also want to understand what kind of switching costs are customers willing to absorb and how onerous they are and what's the business case look like? How are they thinking about that business case. We also want to understand and really probe on how will Palo Alto maintain best of breed as it continues to acquire and integrate to expand its TAM and appeal as that one-stop shop. You know, can it do that as we talked about before. And will it do that? There's also an interesting tension going on sort of changing subjects here in security. There's a guy named Edward Hellekey who's been in "theCube" before. He hasn't been in "theCube" in a while but he's a security pro who has educated us on the nuances of protecting data privacy, public policy, how it varies by region and how complicated it is relative to security. Because securities you technically you have to show a chain of custody that proves unequivocally, for example that data has been deleted or scrubbed or that metadata does. It doesn't include any residual private data that violates the laws, the local laws. And the tension is this, you need good data and lots of it to have good security, really the more the better. But government policy is often at odds in a major blocker to sharing data and it's getting more so. So we want to understand this tension and how companies like Palo Alto are dealing with it. Our customers testing public policy in courts we think not quite yet, our government's making exceptions and policies like GDPR that favor security over data privacy. What are the trade-offs there? And finally, one theme of this breaking analysis is what does Palo Alto have to do to stay on top? And we would sum it up with three words. Ecosystem, ecosystem, ecosystem. And we said this at CrowdStrike Falcon in September that the one concern we had was the pace of ecosystem development for CrowdStrike. Is collaboration possible with competitors? Is being adopted aggressively? Is Palo Alto being adopted aggressively by global system integrators? What's the uptake there? What about developers? Look, the hallmark of a cloud company which Palo Alto is a cloud security company is a thriving ecosystem that has entries into and exits from its platform. So we'll be looking at what that ecosystem looks like how vibrant and inclusive it is where the public clouds fit and whether Palo Alto Networks can really become the security super cloud. Okay, that's a wrap stop by next week. If you're in Vegas, say hello to "theCube" team. We have an unbelievable lineup on the program. Now if you're not there, check out our coverage on theCube.net. I want to thank Eric Bradley for sharing a glimpse on short notice of the upcoming survey from ETR and his thoughts. And as always, thanks to Chip Symington for his sharp comments. Want to thank Alex Morrison, who's on production and manages the podcast Ken Schiffman as well in our Boston studio, Kristen Martin and Cheryl Knight they help get the word out on social and of course in our newsletters, Rob Hoof, is our editor in chief over at Silicon Angle who does some awesome editing, thank you to all. Remember all these episodes they're available as podcasts. Wherever you listen, all you got to do is search "Breaking Analysis" podcasts. I publish each week on wikibon.com and silicon angle.com where you can email me at david.valante@siliconangle.com or dm me at D Valante or comment on our LinkedIn post. And please do check out etr.ai. They've got the best survey data in the enterprise tech business. This is Dave Valante for "theCube" Insights powered by ETR. Thanks for watching. We'll see you next week on "Ignite" or next time on "Breaking Analysis". (upbeat music)

(upbeat music) >> Welcome back to Boston, Massachusetts. We're here at the Seaport. You're watching theCUBE's coverage of Red Hat Summit 2022. My name is Dave Vellante and Paul Gillin is here. He's my cohost for the next day. We are going to dig in to the famous RHEL, Red Hat Enterprise Linux. Gunnar Hellekson is here, he's the Vice President and General Manager of Red Hat Enterprise Linux. Gunnar, welcome to theCUBE. Good to see you. >> Thanks for having me. Nice to be here, Dave, Paul. >> RHEL 9 is, wow, nine, Holy cow. It's been a lot of iterations. >> It's the highest version of RHEL we've ever shipped. >> And now we're talking edge. >> Yeah, that's right. >> And so, what's inside, tell us. to keep happy with a new RHEL release. to keep happy with a new RHEL release. The first is the hardware partners, right, because they rely on RHEL to light up all their delicious hardware that they're making, then you got application developers and the ISVs who rely on RHEL to be that kind of stable platform for innovation, and then you've got the operators, the people who are actually using the operating system itself and trying to keep it running every day. So we've got on the, I'll start with the hardware side, So we've got on the, I'll start with the hardware side, which is something, as you know, RHEL success, and I think you talked about this with Matt, just in a few sessions earlier that the success of RHEL is really, hinges on our partnerships with the hardware partners and in this case, we've got, let's see, in RHEL 9 we've got all the usual hardware suspects and we've added, just recently in January, we added support for ARM servers, as general ARM server class hardware. And so that's something customers have been asking for, delighted to be shipping that in RHEL 9. So now ARM is kind of a first-class citizen, right? Alongside x86, PowerZ and all the other usual suspects. And then of course, working with our favorite public cloud providers. So making sure that RHEL 9 is available at AWS and Azure and GCP and all our other cloud friends, right? >> Yeah, you mentioned ARM, we're seeing ARM in the enterprise. We're obviously seeing ARM at the edge. You guys have been working with ARM for a long time. You're working with Intel, you're working with NVIDIA, you've got some announcements this week. Gunnar, how do you keep Linux from becoming Franken OS with all these capabilities? >> This is a great question. First is, the most important thing is to be working closely with, I mean, the whole point of Linux and the reason why Linux works is because you have all these people working together to make the same thing, right? And so fighting that is a bad idea. Working together with everyone, leaning into that collaboration, that's an important part of making it work over time. The other one is having, just like in any good relationship, having healthy boundaries. And so making sure that we're clear about the things that we need to keep stable and the places where we're allowed to innovate and striking the right balance between those two things, that allows us to continue to ship one coherent operating system while still keeping literally thousands of platforms happy. >> So you're not trying to suck in all the full function, you're trying to accommodate that function that the ecosystem is going to develop? >> Yeah, that's right. So the idea is that what we strive for is consistency across all of the infrastructures and then allowing for kind of optimizations and we still let ourselves take advantage of whatever indigenous feature might appear on, such an ARM chip or thus in a such cloud platform. But really, we're trying to deliver a uniform platform experience to the application developers, right? Because they can't be having, like there can't be kind of one version of RHEL over here and another version of RHEL over here, the ecosystem wouldn't work. The whole point of Linux and the whole point of Red Hat Enterprise Linux is to be the same so that everything else can be different. >> And what incentives do you use to keep customers current? >> To keep customers current? Well so the best thing to do I found is to meet customers where they are. So a lot of people think we release RHEL 9 at the same time we have Red Hat Enterprise Linux 8, we have Red Hat Enterprise Linux 7, all these are running at the same time, and then we also have multiple minor release streams inside those. So at any given time, we're running, let's say, a dozen different versions of RHEL are being maintained and kept up-to-date, and we do this precisely to make sure that we're not force marching people into the new version and they have a Red Hat Enterprise Linux subscription, they should just be able to sit there and enjoy the minor version that they like. And we try and keep that going for as long as possible. >> Even if it's 10 years out of date? >> So, 10 years, interesting you chose that number because that's the end of life. >> That's the end of the life cycle. >> Right. And so 10 years is about, that's the natural life of a given major release, but again inside that you have several 10-year life cycles kind of cascading on each other, right? So nine is the start of the next 10-year cycle while we're still living inside the 10-year cycle of seven and eight. So lots of options for customers. >> How are you thinking about the edge? how do you define, let's not go to the definition, but at high level. (Gunnar laughing) Like I've been in a conference last week. It was Dell Tech World, I'll just say it. They were sort of the edge to them was the retail store. >> Yeah. >> Lowe's, okay, cool, I guess that's edgy, I guess, But I think space is the edge. (Gunnar chuckling) >> Right, right, right. >> Or a vehicle. How do you think about the edge? All the above or but the exciting stuff to me is that far edge, but I wonder if you can comment. >> Yeah, so there's all kinds of taxonomies out there for the edge. For me, I'm a simple country product manager at heart and so, I try to keep it simple, right? And the way I think about the edge is, here's a use case in which somebody needs a small operating system that deploys on probably a small piece of hardware, usually varying sizes, but it could be pretty small. That thing needs to be updated without any human touching it, right? And it needs to be reliably maintained without any human touching it. Usually in the edge cases, actually touching the hardware is a very expensive proposition. So we're trying to be as hands off as possible. >> No truck rolls. >> No truck rolls ever, right, exactly. (Dave chuckling) And then, now that I've got that stable base, I'm going to go take an application. I'll probably put it in a container for simplicity's sake and same thing, I want to be able to deploy that application. If something goes wrong, I need to build a roll back to a known good state and then I need to set of management tools that allow me to touch things, make sure that everything is healthy, make sure that the updates roll out correctly, maybe do some AB testing, things like that. So I think about that as, that's the, when we talk about the edge case for RHEL, that's the horizontal use case and then we can do specializations inside particular verticals or particular industries, but at bottom that's the use case we're talking about when we talk about the edge. >> And an assumption of connectivity at some point? >> Yeah. >> Right, you didn't have to always be on. >> Intermittent, latent, eventual connectivity. >> Eventual connectivity. (chuckles) That's right in some tech terms. >> Red Hat was originally a one trick pony. I mean, RHEL was it and now you've got all of these other extensions and different markets that you expanded into. What's your role in coordinating what all those different functions are doing? >> Yes, you look at all the innovations we've made, whether it's in storage, whether it's in OpenShift and elsewhere, RHEL remains the beating heart, right? It's the place where everything starts. And so a lot of what my team does is, yes, we're trying to make all the partners happy, we're also trying to make our internal partners happy, right? So the OpenShift folks need stuff out of RHEL, just like any other software vendor. And so I really think about RHEL is yes, we're a platform, yes, we're a product in our own right, but we're also a service organization for all the other parts of the portfolio. And the reason for that is we need to make sure all this stuff works together, right? Part of the whole reasoning behind the Red Hat Portfolio at large is that each of these pieces build on each other and compliment each other, right? I think that's an important part of the Red Hat mission, the RHEL mission. >> There's an article in the journal yesterday about how the tech industry was sort of pounding the drum on H-1B visas, there's a limit. I think it's been the same limit since 2005, 65,000 a year. We are facing, customers are facing, you guys, I'm sure as well, we are, real skills shortage, there's a lack of talent. How are you seeing companies deal with that? What are you advising them? What are you guys doing yourselves? >> Yeah, it's interesting, especially as everybody went through some flavor of digital transformation during the pandemic and now everybody's going through some, and kind of connected to that, everybody's making a move to the public cloud. They're making operating system choices when they're making those platform choices, right? And I think what's interesting is that, what they're coming to is, "Well, I have a Linux skills shortage and for a thousand reasons the market has not provided enough Linux admins." I mean, these are very lucrative positions, right? With command a lot of money, you would expect their supply would eventually catch up, but for whatever reason, it's not catching up. So I can't solve this by throwing bodies at it so I need to figure out a more efficient way of running my Linux operation. People are making a couple choices. The first is they're ensuring that they have consistency in their operating system choices, whether it's on premise or in the cloud, or even out on the edge, if I have to juggle three, four different operating systems, as I'm going through these three or four different infrastructures, that doesn't make any sense, 'cause the one thing is most precious to me is my Linux talent, right? And so I need to make sure that they're consistent, optimized and efficient. The other thing they're doing is tooling and automation and especially through tools like Ansible, right? Being able to take advantage of as much automation as possible and much consistency as possible so that they can make the most of the Linux talent that they do have. And so with Red Hat Enterprise Linux 9, in particular, you see us make a big investment in things like more automation tools for things like SAP and SQL server deployments, you'll see us make investments in things like basic stuff like the web console, right? We should now be able to go and point and click and go basic Linux administration tasks that lowers the barrier to entry and makes it easier to find people to actually administer the systems that you have. >> As you move out onto these new platforms, particularly on the edge, many of them will be much smaller, limited function. How do you make the decisions about what features you're going to keep or what you're going to keep in RHEL when you're running on a thermostat? >> Okay, so let me be clear, I don't want RHEL to run on a thermostat. (everybody laughing) >> I gave you advantage over it. >> I can't handle the margins on something like that, but at the end. >> You're running on, you're running on the GM. >> Yeah, no that's, right? And so the, so the choice at the, the most important thing we can do is give customers the tools that they need to make the choice that's appropriate for their deployment. I have learned over several years in this business that if I start choosing what content a customer decide wants on their operating system I will always guess it wrong, right? So my job is to make sure that I have a library of reliable, secure software options for them, that they can use as ingredients into their solution. And I give them tools that allow them to kind of curate the operating system that they need. So that's the tool like Image Builder, which we just announced, the image builder service lets a customer go in and point and click and kind of compose the edge operating system they need, hit a button and now they have an atomic image that they can go deploy out on the edge reliably, right? >> Gunnar can you clarify the cadence of releases? >> Oh yeah. >> You guys, the change that you made there. >> Yeah. >> Why that change occurred and what what's the standard today? >> Yeah, so back when we released RHEl 8, so we were just talking about hardware and you know, it's ARM and X86, all these different kinds of hardware, the hardware market is internally. I tell everybody the hardware market just got real weird, right? It's just got, the schedules are crazy. We got so many more entrance. Everything is kind of out of sync from where it used to be, it used to be there was a metronome, right? You mentioned Moore's law earlier. It was like a 18 month metronome. Everybody could kind of set their watch to. >> Right. >> So that's gone, and so now we have so much hardware that we need to reconcile. The only way for us to provide the kind of stability and consistency that customers were looking for was to set a set our own clock. So we said three years for every major release, six months for every minor release and that we will ship a new minor release every six months and a new major release every three years, whether we need it or not. And that has value all by itself. It means that customers can now plan ahead of time and know, okay, in 36 months, the next major release is going to come on. And now that's something I can plan my workload around, that something I can plan a data center migration around, things like that. So the consistency of this and it was a terrifying promise to make three years ago. I am now delighted to announce that we actually made good on it three years later, right? And plan two again, three years from now. >> Is it follow up, is it primarily the processor, optionality and diversity, or as I was talking to an architect, system architect the other day in his premise was that we're moving from a processor centric world to a connect centric world, not just the processor, but the memories, the IO, the controllers, the nics and it's just keeping that system in balance. Does that affect you or is it primarily the processor? >> Oh, it absolutely affects us, yeah. >> How so? >> Yeah, so the operating system is the thing that everyone relies on to hide all that stuff from everybody else, right? And so if we cannot offer that abstraction from all of these hardware choices that people need to make, then we're not doing our job. And so that means we have to encompass all the hardware configurations and all the hardware use cases that we can in order to make an application successful. So if people want to go disaggregate all of their components, we have to let 'em do that. If they want to have a kind of more traditional kind of boxed up OEM experience, they should be able to do that too. So yeah, this is what I mean is because it is RHEL responsibility and our duty to make sure that people are insulated from all this chaos underneath, that is a good chunk of the job, yeah. >> The hardware and the OS used to be inseparable right before (indistinct) Hence the importance of hardware. >> Yeah, that's right. >> I'm curious how your job changes, so you just, every 36 months you roll on a new release, which you did today, you announced a new release. You go back into the workplace two days, how is life different? >> Not at all, so the only constant is change, right? And to be honest, a major release, that's a big event for our release teams. That's a big event for our engineering teams. It's a big event for our product management teams, but all these folks have moved on and like we're now we're already planning. RHEL 9.1 and 9.2 and 8.7 and the rest of the releases. And so it's kind of like brief celebration and then right back to work. >> Okay, don't change so much. >> What can we look forward to? What's the future look like of RHEL, RHEL 10? >> Oh yeah, more bigger, stronger, faster, more optimized for those and such and you get, >> Longer lower, wider. >> Yeah, that's right, yeah, that's right, yeah. >> I am curious about CentOS Stream because there was some controversy around the end of life for CentOS and the move to CentOS Stream. >> Yeah. >> A lot of people including me are not really clear on what stream is and how it differs from CentOS, can you clarify that? >> Absolutely, so when Red Hat Enterprise Linux was first created, this was back in the days of Red Hat Linux, right? And because we couldn't balance the needs of the hobbyist market from the needs of the enterprise market, we split into Red Hat Enterprise Linux and Fedora, okay? So then for 15 years, yeah, about 15 years we had Fedora which is where we took all of our risks. That was kind of our early program where we started integrating new components, new open source projects and all the rest of it. And then eventually we would take that innovation and then feed it into the next version of Red Hat Enterprise Linux. The trick with that is that the Red Hat Enterprise Linux work that we did was largely internal to Red Hat and wasn't accessible to partners. And we've just spent a lot of time talking about how much we need to be collaborating with partners. They really had, a lot of them had to wait until like the beta came out before they actually knew what was going to be in the box, okay, well that was okay for a while but now that the market is the way that it is, things are moving so quickly. We need a better way to allow partners to work together with us further upstream from the actual product development. So that's why we created CentOS Stream. So CentOS Stream is the place where we kind of host the party and people can watch the next version of Red Hat Enterprise get developed in real time, partners can come in and help, customers can come in and help. And we've been really proud of the fact that Red Hat Enterprise Linux 9 is the first release that came completely out of CentOS Stream. Another way of putting that is that Red Hat Enterprise Linux 9 is the first version of RHEL that was actually built, 80, 90% of it was built completely in the open. >> Okay, so that's the new playground. >> Yeah, that's right. >> You took a lot of negative pushback when you made the announcement, is that basically because the CentOS users didn't understand what you were doing? >> No, I think the, the CentOS Linux, when we brought CentOS Linux on, this was one of the things that we wanted to do, is we wanted to create this space where we could start collaborating with people. Here's the lesson we learned. It is very difficult to collaborate when you are downstream of the product you're trying to improve because you've already shipped the product. And so once you're for collaborating downstream, any changes you make have to go all the way up the water slide and before they can head all the way back down. So this was the real pivot that we made was moving that partnership and that collaboration activity from the downstream of Red Hat Enterprise Linux to putting it right in the critical path of Red Hat Enterprise Linux development. >> Great, well, thank you for that Gunnar. Thanks for coming on theCUBE, it's great to, >> Yeah, my pleasure. >> See you and have a great day tomorrow. Thanks, and we look forward to seeing you tomorrow. We start at 9:00 AM. East Coast time. I think the keynotes, we will be here right after that to break that down, Paul Gillin and myself. This is day one for theCUBE's coverage of Red Hat Summit 2022 from Boston. We'll see you tomorrow, thanks for watching. (upbeat music)

>> Announcer: From around the globe it's theCUBE with digital coverage of IBM Think 2021. Brought to you by IBM. >> Welcome back to theCUBE's coverage of IBM Think 2021. I'm John Furrier, you host of theCUBE. We're here with Howard Boville who's the Head of Hybrid Cloud Platform for IBM. He's been in the industry for many, many decades as a practitioner. Heading up organizations now at IBM, heading up the hybrid cloud. Howard, great to have you on theCUBE. >> Pleased to be here, John. Thank you for your time. >> Can you tell us a little bit about the digital transformation trends that you've seen over the past year as they have clearly shook the industry? Certainly, COVID. No one would have predicted provisioning VPN access or remote access for all the employees. I'm sure that wasn't on anyone's radar, but many more other disruptions and opportunities for accelerating these new, what are now obvious benefits. Can you take your time to explain what you've seen? >> Yeah, sure. So been a huge amount of acceleration of digital transformation. So VPN projects, as you mentioned, the people working from home, projects that, in the past, were taking many, many years to work through, then got done literally in weeks. And they're very complex when you get under the skin of them. And companies, therefore saw confidence in that and started to look at broader digital transformations. And you can kind of think about them in terms of their successes and their failures, or the lessons learned from them. So when it's done right, what I've observed from companies that have done it right, they've done it from a business process perspective. They've looked at their business processes that they want to transform as opposed to just the underpinning technology. But the companies that have been around for a while have also been understood that legacy's a problem. So God created the Earth is seven, or the world in seven days, but that's because He didn't have any legacy to deal with. So as companies have taken the confidence for the smaller projects to work through, they've found in these larger ones, where they've got legacy environments to work through, digital transformation's still very important but it's not as straightforward as they thought it might be. >> You know, one of the things that's coming out of the hybrid cloud discussion is a couple of things. One is everyone now agrees that this is the standard and multicloud's soon around the corner. Hybrid cloud is an operating model and it's a new kind of operating system with the ability to use Kubernetes and containers and microservices and other service meshes to integrate legacy. This is huge. What's the biggest pain points that you're seeing from an adoption standpoint that are blockers from clients? What's gettin' in the way of the obvious, now, path with hybrid cloud? >> Well actually, the first and foremost, the position that IBM's created by kind of calling it a hybrid cloud where companies will be on-premise and off-premise because of legacy, gives CIOs around the world a huge sigh of relief. And having sat in their seats, I've often thought I must be the dumbest person in the room because I don't understand this full on public cloud model because I can't see the benefits to my shareholders that that would deliver. I could see it to the pure player cloud service providers, but not to myself. So talking to CIOs, I think thank heavens for that. We're no longer as seen as Arondight when we're explaining that we'll be on-premise and off-premise, and it'll be heterogeneous environments we're operating within. But the simple way to think about the blockers and actually, you've done a nice job, yourself, John, in terms of explaining this, is cloud is just simply another resource pool that you use to run your applications or your datasets on. And in the past, you had a nicely curated environments when it was in your own environments but there are benefits that you can get by using more immersing technologies like cloud, particularly around developed productivity. But in chapter one of cloud with a pure player cloud providers, it was kind of a carbuncle that you kind of put onto the side of your organization which then became very difficult to this kind of Frankenstein's monster of piece parts to put together from an IT operations and a cyber security perspective. >> Okay, so you talk about this Franken-cloud model, before. I've heard that come from you. What is this about? You just referenced it, there. What is the Franken-cloud? >> Yeah, it's the simple way to think about it is in the old world when you ran all of your applications, your datasets, your developers, in your own data centers, you would create a curated model that would allow you to run it very strongly from an architectural perspective. Lots of different legacy environments, but the actual architecture you'd put around it would be clean and the IT operational environments would be clean, and the actually cyber security controls. You put on a third party capability whether that's a cloud service provider or a software as a service provider and you add a world of complexity where you have no controls over those environments, and you're certainly not driving the architectural standards. So you're putting together these piece parts in the same way as Dr. Frankenstein put together the monster that he created. And ultimately, that will turn upon you. It will create technical operational issues, it will create economic issues, and it absolutely will create cyber security issues. So the important thing to think about on these digital transformations is the architecture in a hybrid context is one that will work for you with a multicloud environment, whether that's from a software as a service provider or from a cloud service provider. >> It's interesting you bring up these other turning on you kind of the Franken-cloud, I get that. But let's bring that up to the positive. A client customer might say hey, you know, I did a great job of moving into the public cloud, I brought some stuff on hybrid. Oh my god, look at them pushin' some new stuff. And then they push new code and then things break. They call this day 2 operations or as you guys are referring to, AI ops. These are opportunities. So how does a company get their arms around that because that's going to be the next progression. Okay, I'm operating on a distributed basis. All right, great. I got an Edge, data center, whatever, but now I'm pushin' code all the time. I don't want it to break. >> I mean, most of my comments, John, are based upon the experiences and mistakes that I've made in my career. So that element that you talked about there, that day 2 operations, not only are we going through an inflection point in terms of the technologies that are used, and the architectures at a technical level you have to put together, the silicon that you think about, you got to really think about the carbon, the people, and the IT operating model that you have. 'Cause a lot of the actual manual work you did, previously, we'll be doing it in an automated fashion, so an AI fashion. So any transformation program needs to look at the actual transformation of the skills of the people that you have working for you. And the shouldn't feel fearful that it's a place where they actually won't have a role, they just won't have a role with the current skillset they've got. But there are adjacent skillset that you an have that they can actually get trained into or get on assignments where they get the experience to operate in that fashion. >> Hey, I'd love to get the comment on the Edge with the S system on a chip, SOC as it's called, as more and more capabilities are going to be at the Edge. But I want to stay on this quick cloud thing on Franken-cloud, because you know, one of the things that I see with the positives of cloud is that okay, get me more agile, but then I get worried that if I'm going too fast, I might break something or get fired. I got all this compliance, I don't want to get sued or you know, there's all kinds of regulations now, and compliance around distributed clouds, globally. So what's your take on that? What specific challenges do these companies face when they're either in regulated industries or don't want to go too fast? They got to watch that data and make sure it's not going to be misused. >> Yeah, so the philosophy that we have at IBM is different to chapter one in the pure player cloud providers which is we believe if you build the actual compliance controls in from the outset and have them as a standard of consumption for all customers, they can actual accelerate their adoption of cloud. So they can actually get to the benefits of cloud productivity, innovation far more quickly. And that's been evidenced by chapter one where all large institutions in multiple-year programs spend tens of millions of dollars and are building the compliance controls, themselves. You don't do that with IBM. You get that out of the box for the entire industry. We keep that fresh and current, and vibrant going forward so those non-functional requirements are no longer a consideration for you, and you can then focus your energy, your developers in terms of the actual point of innovation on the functional capabilities that you can provide. >> I want to get your reaction to something and a comment, if you don't mind. I mean, there's been a big trend of data clouds built on other people's clouds, and you got the needs of specialty in industries or critical needs. Do you see the need or do you see a path for specialty clouds or vertical clouds, specifically, as these, the AI in data can be relative to these verticals but you want, at the same time, horizontal scalability for a data plane or data access. What's your take on specialty clouds or vertical clouds? >> Yeah, I mean, that's at the heart of the thesis and the idea that we have here, at IBM, which is there is a need for specialty clouds in particular industries and their workloads. And really, as kind of people look back in the very near future, they'll say that's an evident thing because again, in the old world when it was in your own data center, you would have build types for specific types of applications and the processes that it supported and the risk posture of that, and then your associated datasets. So the capabilities that we build within our global availability zones is for the large enterprises and that's an area that's obviously at IBM's heritage. And then it's not just the software level, it's the hardware it runs on. So IBM provides the hardware from a mainframe power X86 so through all those kind of form factors, and then at an operating system level, obviously if you're Linux in terms of the capabilities that we have. So we can meet all of that stack but build them specifically for the applications and the datasets for the industries that we serve, and the AI capabilities necessary. >> That's great stuff. I want to get your take and shift gears to cyber security. I mean, every time you look, there's a headline of a breach. SolarWinds had more implications than anyone could imagine. Do you hire more firefighters to put out the fire? Do you make fire resistant materials? I mean, there's an optimization balance. What do you think is the best way we prevent cyber breaches goin' forward? What's your take on this? I'm sure you've had a perfect-- >> So in the world of cyber security, it's all of the above and then many more because you've got to put checks and balances in terms of every capability having kind of come from an environment where my old bank was named after the country that it was in, and therefore, nations states were to correct the light in terms of trying to breach the area. So all of those controls are necessary as you put them in. But the other element to think about on digital supply chains is again, if you actually have your supply chain on a cloud that has the compliance controls built in, they benefit in a Herodot, as well. Whereas, if you don't, you've got to actually ensure that they are actually attesting to the controls. The cloud that we built here, at IBM, give you continuous monitoring to ensure that those software as a service providers are actually adhering to the controls you want in real time. That is a massive game-changer in terms of the then logging information we can provide to customers to assure that their digital supply chain does not become compromised. >> Real quick while I got you here, as cyber standards become around hybrid, the early responses were specialized on AWS, Azure, or Google, and they pick one, I have a backup cloud, and then build your teams around that, your developer teams. Does that shift with hybrid? How does CSOs change with hybrid? >> So the benefit in terms of the entry to IBM has in the cloud space which is probably in terms of the current variants, two years old, is that we're not dealing with legacy. So we're kind of learning from the mistakes that these older cloud providers that have got a wealth of legacy in their environments both for the actual hardware level, but also at the code-base level, some more so than others in terms of the issues they have with their code bearers. And therefore, with the AI ops and the actual cyber security tools that we put in place we're building upon the bad experiences they've had but also other intelligence that we get in terms of threat factors as they come through, John. >> Howard, and the last question to end this segment, you've led a lot of digital transformation initiatives through your career. What have you found has been the best practice as that applies now, as companies are coming out of COVID, they want to have a growth strategy, they want to make sure the foundation's in place that's solid, that they can build upon. What's your lessons learned? What's your best practice advice? >> So you've got to deal with the difficult problems first, that sometimes are fundamental to get to pace. So controls appears to be a fairly mundane topic but unless you can deal with the controls, you can't actually get the accelerated pace. And then when you do these transformations you have to bring your people along with you at the same time as you're transforming the technology. So you need the silicon to be allied with the carbon and then you get people that are actually change hungry as opposed to change resistant. >> Howard Boville, thanks for comin' on theCUBE. Head of Hybrid Cloud Platforms at IBM. Thanks for joining us, today. >> You're welcome, thank you, John. >> Okay, I'm John Furrier with theCUBE for IBM Think 2021 coverage. Thanks for watching. (soft music) ♪ Dah de dah ♪ ♪ Dah ♪

>>from around the globe. It's the >>cube with digital >>coverage of IBM >>Think 2020 >>one brought to you by IBM. Welcome back to the cubes coverage of IBM Think 2021 john for your host of the cube we're here. Howard Belleville is the head of hybrid cloud platform for IBM been in the industry for many, many decades as a practitioner heading up organizations now at IBM heading up the hybrid cloud. Howard, great to have you on the cube. >>Pleased to be here, john thank you for your time. Can >>you tell us a little about the digital transformation trends that you've seen in the past year as they have clearly shook the industry? Certainly Covid no one would have predicted provisioning VPN access or remote access for all the employees. I'm sure that wasn't on anyone's radar but many more other disruptions and opportunities for accelerating these new what are now obvious benefits. Can you take your time to explain what you're seeing? >>Yeah, sure. So there's been a huge amount of acceleration of digital transformation. So VPN projects, as you mentioned, the people working from home projects that in the past were taking many, many years to work through then got done literally in weeks, um and they're very complex when you get under the skin of them. Um and companies therefore saw confidence in that and start to look at broader digital transformations um and you can kind of think about in terms of the successes and their failures or the lessons learned from them. So when it's done right, what I've observed from companies that have been it right, they've done it from a business process perspective, they looked at their business processes that they want to transform as opposed to just the independent technology, um but the companies that have been around for a while, I've also been understood that legacy is a problem. So God created the Earth in seven of the world in seven days, but that's because he didn't have any legacy to deal with. So as companies have taken the confidence for the smaller projects to work through, they found in these larger ones where they've got legacy environments to work through. Digital transformation is still very important, but it's not as straightforward as they thought it might be. >>You know, one of the things that's coming out of the hybrid cloud um, discussion is a couple of things. One is everyone now agrees that this is the standard and multi cloud soon around the corner. Um, Hybrid clouds and operating model. Um, and it's a new kind of operating system with with the ability to use kubernetes and containers and microservices and other service message to to integrate legacy. This is huge. What's the biggest pain points that you're seeing from an adoption standpoint that our blockers from clients, What's getting in the way of the obvious now path with hybrid cloud? >>Well actually the first and foremost the position that IBM is created by kind of calling out hybrid cloud where companies will be on premise and off premise because the legacy IFC IOS around the world, the huge sigh of relief and having sat in their seats, I often thought I must be the dumbest person in the room because I don't understand this full on public cloud model because I can't see the benefits to my shareholders that that would deliver. I could see it to the pure play cloud service providers but not to to myself. So talking to C I O S I think thank heavens for that were no longer seen as a Luddite when we're explaining that we'll be on premise and off premise and it'll be heterogeneous environments were operating with within the simple way to think about the blockers and actually done a nice job yourself, john in terms of explaining this is cloud, is is simply another resource tool that you use to run your applications or your data sets on and in the past you had nicely curated environment within it was in your own environment, but there are benefits that you can get by using more innocent technologies like cloud, particularly around developed productivity. But in chapter one of cloud with a pure flare cloud providers, it was kind of a carbuncle that you kind of put onto the side of your organization, which then became very difficult. This kind of Frankenstein's monster of piece parts to put together from an IT operations and cyber security perspective. >>Okay, so you talk about this Franken cloud model before, I've heard heard that come from you. What is this about? You just referenced it there. What is the Franken cloud? >>Yeah, that's the simple way to think about it is um, in the old world, when you run all of your applications, your data sets, your developers in your own data sensors, you would create a curated model that would allow you to very strongly from an architectural perspective, lots of different legacy environments. But the actual architecture put around it would be clean and the operational environment will be clean and the actual Cyprus security controls, you put on a third party capability, whether that's a cloud service provider or a software as a service provider. And you had a world of complexity where you have no control over those environments and you're certainly not driving the architectural standards. So you're putting together these peace parts in the same way as dr Frankenstein put together the monster he created and ultimately, that will turn upon you. It will create technical operational issues that will create economic issues and it absolutely will create cybersecurity issues. So the important thing to think about on these digital transformations is the architecture in a hybrid context is one that will work for you with a multi cloud environment, whether that's from a software as a service provider or from the cloud service >>provider. It's interesting you bring up these other turning on you kind of the Franken cloud. I get that. But let's bring that up to the positive a client customer might say, hey, you know, I did a great job of moving into the public cloud. I brought some stuff on hybrid. Oh my God, look at the push some new stuff and then I pushed new new code and then things breaks. They call this day to operations. Or as you guys are referring to a I ops. These are opportunities. So how does the company get their arms around that? Because that's gonna be the next progression. Okay. I'm operating on distributed basis. Alright, great. I got an edge data center, whatever. But now I'm pushing code all the time. I don't want it to break. >>Yeah, I mean most of my comments are based upon the experiences and the mistakes that I've made in my career. So that element that you talked about there that day to operations, not only are we going through an inflection point in terms of the technologies that are used and the architecture is at a technical level, you have to put together the silicon that you think about. You're going to really think about the carbon, the people and the operating model that you have because a lot of the actual manual work you did previously will be done in a in an automated fashion. So an Ai fashion. So any transformation program needs to look at the actual transformation of the skills of the people you have working for you and they shouldn't feel fearful that it's a place where they actually won't have a role. They just won't have a role with the current skill sets they've got. But there are adjacent skill sets that you can have that they can actually be trained into or get an assignment where they get the experience to operate in that fashion. >>I'd love to get the comment on the edge with this system on a chip soc as it's called As more and more capabilities are going to be at the edge. But I want to stay on this quick cloud thing on Franken cloud because you know, one of the things that I see with the positives of cloud is is that okay? Can be more agile. But then I get worried that if I'm going too fast, I might break something. I get fired. I got all this compliance, don't get sued or you know, there's all kinds of regulations now and compliance around distributed clouds globally. So what's your take on that? What specific challenges do these companies face when they're either in regulated industries or don't want to go too fast? I gotta, I gotta watch that day to make sure it's not gonna be misused. >>Yeah. So the, so the philosophy that we have at IBM is different to Chapter one and the pure player cloud providers, which is, we believe if you build the actual compliance controls in from the outset and have them as a standard of consumption for all customers, they can actually accelerate their adoption of cloud so they can actually get to the benefits of cloud productivity innovation far more quickly. And that's been evidenced by Chapter one where all large institutions in multiple year programs spend tens of millions of dollars and are building the compliance controls themselves. You don't do that with IBM, you get that out of the box for the entire industry. We keep that fresh and current and vibrant going forward. So there's non functional requirements and no longer a consideration for you and you can then focus your energy, your developers in terms of the actual points of innovation, on the functional capabilities that you can provide. >>I want to get your reaction to something and you comment if you don't mind. I mean, there's been a big trend of data clouds built on other people's clouds and you've got the needs of special specialty and industries or vertical needs. Do you see the need or you see a path for specialty clouds or vertical clouds? Specifically as these? The A. I. And data can be relative to these verticals but you want at the same time horizontal scalability for data plane or data access. What's your take on specialty clouds? >>That's at the heart of the thesis. And the idea that we have here at IBM, which is there is a need for specialty clouds of particular industries and their their workloads. And really it's kind of people look back in the very near future. That's a that's an evident thing because again in the old world when it was in your own data center, you would have build types for specific types of applications and the processes that are supported and the risk posture of that and then the associated data sets. Um so the capabilities that we built within our global availability zones is for the large enterprises and that's an area that's obviously being heritage and then it's not just the software level, it's the hardware it runs on. So IBM provides the hardware from a mainframe power X 86. So for all those kind of form factors and an operating system level, obviously through Lennox in terms of the capabilities that we have so we can meet all of that stack but build them specifically for the applications and the data sets for the industries that we serve and the capabilities necessary. >>That's great stuff. I want to take your take shift gears to cybersecurity. I mean every time you look at the headline of a breach, solar winds had more implications than anyone could imagine. You do you hire more firefighters to put out the fire? Do you make fire resistant materials? I mean, there's optimization balance. What do you think is the best way to prevent cyber breaches going forward? What's your take on this? I'm sure you have >>a person, the world world of cyber security, it's all of the above and them anymore because you've got to put checks and balances in terms of every capability having kind of come from an environment where my old bank was named after the country it was in and therefore Nation state to take great delight in terms of trying to breach the area. So all of those controls are necessary um as you, as you put them in the other element to think about on digital supply chains is again, if you actually have your supply chain on a cloud that has the compliance controls built in, they benefit and inherit that as well. Whereas if you don't, you got to actually ensure that they are actually attesting to the controls. The cloud that we built here at IBM gives you continuous monitoring to sure that those software as a service providers are actually adhering to the controls you want in real time. That is a massive game changer. In terms of the, the logging information we can provide to customers to ensure that their digital supply chain does not become compromised >>real quick. While I've got you here as cyber standards become around hybrid. Uh, the early responses were specialized on AWS as your google and they pick one to have a backup cloud and build your teams around that your developer teams. Does that shift with hybrid? How does seesaws change with hybrid? >>Yeah. So, the benefit in terms of the entry to IBM has in the cloud space, which is probably in terms of the current variants, two years old is that we're not dealing with legacy. Um so we're kind of learning from the mistakes of these older cloud providers that have got a wealth of legacy and their environments, both at the actual hardware level, but also the code base level, some more so than others in terms of the issues they have with their code base um and therefore with the ai ops and the actual cyber security tools that we put employers were building upon, the bad experiences they've had, but also other intelligence that we get in terms of threat vectors as they come through, john >>howard. In the last question to end the segment, you've led a lot of digital transformation initiatives through your career. What have you found has been the best practice as that applies now as companies are coming out of covid, they wanna have a growth strategy. You want to make sure the foundations in place, that's solid that they can build upon. What's your, what's your lessons to learn, What's your best practice advice. >>So you've got to deal with the difficult problems first that sometimes a fundamental to get to pierce. So controls appears to be a fairly mundane topic. But unless you can deal with the controls, you can't actually get the accelerated pierce. And when you do these transformations, you have to bring your people along with you at the same time as your transfer transforming the technology. So you need the silicon to be allied with the carbon and then you get people are actually change Hungary as opposed to change resistant. >>Howard bravo. Thanks for coming on the cube head of hybrid cloud platforms at IBM. Thanks for joining us today. >>You're welcome. Thank you, john. >>Okay. I'm John Free with the Cube for IBM think 2021 coverage. Thanks for watching. Mhm Yeah.

(upbeat music) >> Hi, and welcome to another CUBEConversation where we go in-depth into the topics that are most important to the technology industry with the thought leaders who are actually getting the work done. I'm Peter Burris, and we've got a great conversation today, and it all starts with the idea of how do you get smart people outside of your organization, in-service organizations to help you achieve your outcomes? It's a challenge because as we become more dependent upon services, we discover that service companies are often trying to sell us bills of goods or visions that aren't solving our exact problem. There's a new breed of service company that's really fascinated by your problem, and wants to sell it. Starts with engineering, starts with value add, and then leads to other types of potential relationships and activities. So what do those service companies look like? Well, to have that conversation, we've got Greg Tinker, who is the CTO and founder of Serene IT. Greg, welcome back to theCUBE. >> Thank you very much Peter, glad to be here. >> So tell us a little bit about Serene IT. >> So Serene IT is a, well we call it a next generation bar. So what do I mean by that? We mean that we are an engineering-first firm, so our staff is big, we're across the U.S., we have multiple branches and we just went international into Canada, with Serene IT Canada. We have other international branches that we coming online next year. So with that being said though, the key to our growth, the key to our success is the fact that we're an engineering firm first. We have very few sales staff. Our sales staff are more of an account management style, more of a nurturer or a farmer, we would call it, versus a hunter that means someone going out, because the customers are coming to us with their problems because they need a smart engineering bench to help them. They're not looking for somebody else's to bring them askew, or resell them a product. That can be easily done by some of the large conglomerates that are already out there, not to mention, spend 30 seconds on Google, you can pretty much buy anything you want. >> Yeah, and you know Fred Brookes said a million years ago, when I was, even before I got into computer science, wrote "The Mythical Man Month", and made the observation that the solution to a hard problem typically, is not more people, >> Right. >> It's working smarter, and working more with the right people. So tell a little about how you're able to find the right people from the industry, and bring them together to turn them into the right team. >> It's a great question, Peter, so I've been very fortunate. I loved my career at Hewlett Packard. I left on good terms because I saw a problem in the industry that I wanted to go and tackle head-on. It's easy for people to sit back and talk about it, it's more difficult to actually go and try to solve the problem, and I'm trying to solve the problem. The problem is, there's a lot of orders out there that bring very low value today, they bring a lot of resale. And that's great for those clients that just know what they want. The vast majority of customers don't know what they want today because the technologies are so advanced, they need help to get from where they were, a legacy model, to a more modern software-defined ecosystem. >> And the business problems are so complex. >> Yes. >> It's that combination of complex business problems, 'cause your competitions and your customers are pushing you, and now advanced technologies that have to be marshaled to solve those problems. >> That's exactly right, so with that being said, I set out build an engineering firm and resale would be something later, but we sell through the engineering consulting firms to solve those business problems for our clients. And so our engineering bench is comprised of engineers from Cisco, from Dell, from HPE, from a lot of big conglomerates that everybody all knows. But when you work in this industry, in the labs of these big conglomerates, me coming from HPE, when you do that, you get a lot of friends across the pillars. >> Sure. >> You build networks. >> You build networks. And quite frankly, it's the Marvel lab guys that own today Q-Logic. We all know each other, and with that being said, some of these guys want to go out and try to solve these big problems with companies like myself, and so with that being said, that's how we're building Serene IT, is engineering-first, and we have a very large technical bench today. Just think about it, the company came online in 2017 with just two, so today, we are significantly bigger than that. We're approaching a 50-plus headcount, and we continue to expand with multiple branches, and our growth rate is almost double every six months. And it's something I'm having a great deal of fun doing. The key thing here though is solving business problems and helping customers. >> Well let's talk about that, because every IT organization faces the challenge that they've been so focused on the hardware assets for so long, or the application assets. Now they're trying to focus on the data assets, but they find themselves often in conflict with the business They're not doing a particularly good job of translating a business opportunity into a technology solution still. >> True. >> You've got these great engineers. How are you getting them to also speak business, so that you facilitate that domain expertise about the business so it can be turned into a technology-reliable solution? >> Like any good engineering firm, you have to have levels right? So we have a knock all the way to level four, and our level four engineers are our master technologists that are usually patent published or some varied nature thereof, with usually a multitude of master ASC certification structures to be able to state the fact that they are level four. We also have some college kids that are coming up that are wanting to learn with us, which is good. But I want to tell you on that same point though, is we only allow those elite, the level three, the level four guys, to be in front of our clients, because they've been in this industry a long time. Like myself, we can understand the business problems, as well as the technology problems, and help a client go from zero to hero. That's what we do well. >> So you're bringing in people who have been business people, but have strong engineering backgrounds >> Correct. >> In product domains, in service domains, in the industry, and you're bringing them together and saying, let's go back to being engineers, that can still talk business. >> That's exactly it, that's the key differentiator with us, is the fact that we're not talking just essays, a lot of ours, in our mindsets have essays they call engineers. We don't hire anyone that can't put fingers on a keyboard. If they can't make magic happen on a keyboard, they're of no value to us, they're of no value to our clients, which is what they need help with. So if we're not able to sit down and have a conversation and pull out a laptop and make some some magic happen with, name it, Ansible, Puppet, Shell, Saltstack, that's just in automation CodeLogics, C-code we've got all the cool stuff in that space. But if we can't sit down and write Python, Ruby on Rails and whatnot, and make something tangible to a client in very short order, we didn't do our job. >> So a lot of companies that I've experienced, a lot of customers I've talked to, have what I would call the "goldilocks" problem with their service providers. By that I mean, some of their service providers don't have the technical chops to just throw numbers at it, so they're too cold. Some of their service providers are too smart, or pushing too hard and they get suspicious of them. How do you be that just right, stay focused on the problem bringing the other team, the engineers or the IT folks that you're working with along with you, so you get that natural technology transfer so the business gets the capability that it can run and you can go do something else? >> So that's a good point, Peter. I mean, we're still working out some of those details, I'll tell you, to be honest with you on that stuff. >> Everybody is. >> Yeah. We're getting better at it, you know customers. If we get to aggressive, and tell the customers this is what's wrong with your problem, this is where you need to go, we call their baby ugly, it puts a lot of contention right on the onset, so it causes problems. So we have to be very cognitive of what they have, and where they want to go, and show them where we're going and why we're doing it, and not just focus on "You did it the wrong way". We don't want to focus on that. That's already done, that ship's already sailed, why bash it? I tell my engineers don't talk negative, there's no good going to come of it. Focus on what you have, and where you need to go with it, and how we're going to get there. Keep it a positive message, and you'll find it'd be more receptive, and it's working for our team. >> Well I'll tell you, one of the things I've heard about Serene IT is that you guys especially developed competencies in technologies that have worked in the past. >> You can say that. >> It seems as though one of the things you're able to do is you're able not to make something so new and so distinct that the client can't see how they can possibly operate it without you. You're taking a lot of open-source, a lot of established tried-and-true technologies and using your smarts to put them together in new and interesting ways so the customer says, "Oh that was smart, that was smart. "I can do that, oh yes, now I get it". Is that, am I mis-characterizing your guys? >> No, you're not, you're actually spot-on. We actually have one of the largest ZFS file systems on the planet right now with 142 million users hitting it and-- >> ZFS? >> Yeah, it's old school. >> With 142 million, okay. >> Yeah, it's old-school But if what's old is new again, we're just putting a new wrapper around it. It worked great in its day, but you put that old technology, the file system itself that's been around for a long time, one of the biggest file systems at 128 bit. You take that file system and you put that on today's Red Hat, Caldera, SUSE, name your favorite. You put that on a big machine, a Linux machine today, a large scale like an HPDL380 with NVME drives with a back-end data store, like a 3PAR or Primäre, or name whatever you want on the back end with a big fiber channel, you'd be surprised what we can do with that thing. So we're able to keep customers' costs down by showing them we can take a old-school technology and make it far bigger than you ever imagined, and give you more horsepower and at less cost, and customers are really receptive to that. Now is that perfect for every footprint? No, that was a unique situation. Not everybody's got 142 million users.(chuckles) >> Well, that's true. And so let me build on that, because the other thing that the CIOs I talk to and senior IT people and also business people, increasingly, is they want to make sure that the solution works now, but that it's not going to end-of-life options for them. >> Yeah. >> How do you do this using tried-and-true technologies combined into new and interesting ways, in a way that still nonetheless gives customers future growth options or future application options? >> I'm not a fan of vendor-locking, I'm not a fan of Franken-monsters. Our team of engineers, we have a mandate that they do not build anything like that, I won't approve it. Because I don't want to have a customer locked in to Serene IT. That was never the intent. We want them to choose us, we want them to come to our team and get our value, so we can show them how to grow their business, and do it in a nice, sustainable way, so we can show their staff how to support it. That takes us into our managed services component. Most of the big things we design and do, we're what we call an adaptive managed services, an AMS model. What do I mean by that statement? We're not a WITO. What's a WITO, you ask? It's a "Walk In, Take Over". That's the big boys, that's the DXEs of the world, that's the Assentras, that's what they do. And they do that well. We're not here to compete with that. But what we're here to do is say, to a company or business, whoever they might be, you probably don't need us to take over everything in your IT shop, and really, we're not going to be the best at that, nor are they in some cases, the other vendors. I'll tell you, you know your business the best. We know infrastructure the best, and we can show you where you can build your skillsets up and get better at it. We can automate a lot of it and show you how to manage the automation, and there'll be certain key points that maybe you guys don't want to own for various reasons, and we will manage just that key component, and we do that today with a lot of our big clients. >> Greg Tinker, CTO and founder of Serene IT, thanks very much for being on theCUBE. >> Thank you, Peter. >> And once again, I want to thank you for participating in this CUBEConversation. Until next time. (upbeat music)

>> Welcome to the latest edition of the MIT Sloan Expert Series. I'm your host, Rebecca Knight. Our topic today is racial bias in the sharing economy, how Uber and Lyft are failing black passengers, and what to do about it. Here to talk about that is Chris Knittel. He is a professor of Applied Economics here at MIT Sloan, and he's also the co-author of a study that shows how Uber and Lyft drivers discriminate based on a passenger's skin color. Thanks so much for joining us. >> Oh, it's great to be here. >> Before we begin, I want to remind our viewers that we will be taking your questions live on social media. Please use the hashtag MITSloanExpert to pose your questions on Twitter. Chris, let's get started. >> Chris: Sure. So there is a lot of research that shows how difficult it is to hail a cab, particularly for black people. Uber and Lyft were supposed to represent a more egalitarian travel option, but you didn't find that. >> That's right, so what we found in two experiments that we ran, and one in Seattle, and one in Boston, is that Uber and Lyft drivers were discriminating based on race. >> Rebecca: We've already seen, actually some evidence of racial discrimination in the sharing economy, not just with ride sharing apps. >> Sure, so there's evidence for Airbnb. And what's interesting about Airbnb actually, is that discrimination is two-sided. So not only do white renters of properties not want to rent to black rentees, but white renters do not stay at a home of a black home owner. >> Did your findings and the findings of that other research you just talked about, does it make you discouraged? >> Partly, I was an optimist. We went into this, at least I went into this hoping that we wouldn't find discrimination, but one thing that has helped, or at least shined a more positive light, is that there are ways that we can do better in this sector. >> You've talked about this study, which you undertook with colleagues from the University of Washington and Stanford, shows the power of the experiment. Can you talk a little bit about what you mean by that? >> Sure, what we did was actually run two randomized control trials. Just like you would test whether a blood pressure medication works, so you would have a control group that wouldn't get the medication, and a treatment group that would. We did the same thing where we sent out in Seattle both black and white RAs that hailed Uber and Lyft rides, and we randomized whether or not it was a black RA calling the ride or a white RA that particular time, and they all drove the same exact route at the same exact times of the day. >> So what did you find? Let's talk about first, what you found in Seattle. >> Sure, so in Seattle, we measured how long it took for a ride to be accepted, and also, how long it took, once it was accepted, for the driver to show up and pick up the passenger. And what we found is, if you're a black research assistant, that in hailing an Uber ride, it took 30 percent longer for a ride to be accepted, and also 30 percent longer for the driver to show up and pick you up. >> 30 percent seems substantial. >> Well, for the time it takes to accept the ride, we're talking seconds, but for the time it takes for a passenger to actually be picked up, it's over a minute longer. And I'll mention also for Lyft, we found a 30 percent increase in the amount of time it took to be accepted, but there was no statistically significant impact on how long it took for the driver to actually show up. >> So, the thing about the minute difference, that can be material, particularly if you're trying to catch a cab, an Uber or a Lyft for a job interview or to get to the airport. >> Yeah, this is introspection, but I always seem to be late, so even a minute can be very costly. >> I hear you, I hear you. So why do you think there was the difference between Lyft and Uber? >> What's interesting, and we learned this while we were doing the experiment, a Lyft driver sees the name of the passenger before they accept the ride, whereas an Uber driver only sees the name after they've accepted. So in order for an Uber driver to discriminate, they have to first accept the ride, and then see the name and then cancel, whereas a Lyft driver can just pass it up right away. So it turns out because of that, the Lyft platform is more easily capable of handling discrimination because it pushed it to another driver faster than the Uber platform. >> I want to come back to that, but I want to say also, that difference caused you to change the way you did the experiment in Boston. >> In Boston, a couple differences. One is that we sent out RAs with two cell phones actually. So each RA had an Uber and Lyft account under a stereotypically white sounding name, and then also an Uber and Lyft account under a stereotypically black sounding name. That was one difference, and then also, what we measured in Boston that we didn't measure in Seattle, is cancellations. So an Uber driver accepts the ride, and then cancels on the RA. >> Let's go back to the stereotypically black sounding name verses white sounding name. You're an economist, how did you determine what those names are? >> We relied on another published paper that actually looked at birth records from the 1970s in Boston, and the birth records tell you not only the name, but also the race of the baby. So they found names that actually 100 percent of the time were African American or 100 percent of the time were not African American. So we relied on those names. >> And the names were... >> So you could imagine Jamal for example, compared to Jerry. >> Alright, Ayisha and Alison. >> Chris: Sure. >> So what was your headline finding in Boston? >> In Boston, what we found is, if you were a black male calling an Uber ride, that you were canceled upon more than twice as often as if you were a white male. >> And what about Lyft? >> For Lyft, there is no cancellation effect, and that's not because there's no discrimination, it's just that they don't have to accept and then cancel the ride, they can just pass up the ride completely. It's actually a nice little experiment within the experiment, we shouldn't find an effect of names on cancellations for Lyft and in fact, we don't. >> And also, the driver network is much thicker in Boston than in Seattle. >> So in Boston, although we found this cancellation effect, we didn't find that it has a measurable impact on how long you wait. And this is somewhat speculation, but we speculate that that's because the driver network is so much more dense in Boston that, although you were canceled upon, there's so many only drivers nearby, that it doesn't lead to a longer wait time. >> How do you think what you found compares to hailing traditional cabs? We started our conversation talking about the vast body of research that shows how difficult it is for black people to hail cabs. >> Yeah, we are quick to point out that we are not at all saying that Uber and Lyft are worse than traditional, status quo system, and we want to definitely make that clear. In fact, in Seattle, we had our same research assistants stand at the busiest corners and hail cabs. What we found there is, if you were a black research assistant, the first cab passed you 80 percent of the time. But if you were a white research assistant, it only passed you 20 percent of the time. So just like the previous literature has found, we found discrimination with the status quo system as well. >> You've talked to the companies about you findings, what has the response been? >> That's been actually heartening. Both companies reached out to us very quickly, and we've had continued conversations with them, and we're actually trying to design followup studies to minimize the amount of discrimination that's occurring for both Uber and Lyft. >> But those are off the record and... >> Right, we're not talking specifics, but what I can say is that the companies understand this research and they definitely want to do better. >> In fact, the companies both have issued statements about this, the first one is from Lyft, "we are extremely proud of the positive impact..." Uber has also responded. So let's talk about solutions to this. What do you and your colleagues who undertook this research suggest? >> We've been brainstorming, we don't know for sure if we have the silver bullet, but a few things could change, for example, you could imagine Uber and Lyft getting rid of names completely. We realize that has a trade off in the sense that it's nice to know the name of the driver... >> Rebecca: Sure, you can strike up a conversation... >> It makes it more social, it makes it more personal, more peer to peer if you will. But it would eliminate the type of discrimination that we uncovered. Another potential change is to delay when you give the name to the driver, so that the driver has to commit more to the ride than he or she previously had to. And that may increase the costs of discrimination. >> So that would be changing the software? >> Right, so you could imagine now, like I said, with Lyft that you see the name right away. Maybe you wait until they're 30 seconds away from the passenger before you give them the name. >> What about the dawn of the age of autonomous vehicles? Might that have an impact? We already know that Uber is experimenting with driverless cars in Pittsburgh and Arizona. >> That would obviously solve it, so that would take the human element out of things, and it's important to point out that these are the drivers that are deciding to discriminate. So provided you didn't write the autonomous vehicle software to discriminate, you would know for sure that that car is not going to discriminate. >> What about a driver education campaign? Do you think that would make a difference? I'm reminded of an essay written by Doug Glanville, who is an ESPN commentator and former pro ball player. He writes, on talking about his experience being denied service by an Uber driver, "the driver had concluded I was a threat, "either because I was dangerous myself, "or because I would direct him to a bad neighborhood, "or give him a lower tip, either way, "given the circumstances, it was hard "to attribute his refusal to anything other than my race. "Shortly after we walked away, I saw the driver assisting "another passenger who was white." >> We all hope that information helps, and eliminates discrimination. It's certainly possible that Uber and Lyft could have a full information campaign, where they show the tip rates for different ethnicities, they show the bad ride probabilities for different ethnicities, and my hope is that once the drivers learn that there aren't differences across ethnicities, that the drivers would internalize that, and stop discriminating. >> Policy, Senator Al Franken has weighed in on this, urging Uber and Lyft to address your research. Do you think that there could be policies too? Does government have a role to play? >> Potentially, but what I'll say again is, that Uber and Lyft, I think have all the incentive in the world to fix this, and that they seem to be taking active steps to fixing this. So what could policy makers do? They can, obviously it's already outlawed. They could come down and potentially fine the companies if there's more evidence of discrimination. But I would at least allow the companies some time to internalize this research, and respond to it, and see how effective they can be. >> Many, many think tanks and government advocacy groups have weighed in too. The MIT Sloan Expert Series recently sat down with Eva Millona of the Massachusetts Immigrant and Refugee Coalition. She will talk about this research in the context of immigration, let's roll that clip. >> We're an advocacy organization, and we represent the interest of foreign born, and our mission is to promote and enhance immigrant and refugee integration. Anecdotally, yes, I would say that the research, and given the impressive sample of the research really leads to a sad belief that discrimination is still out there, and there is a lot that needs to be done across sectors to really address these issues. We are really privileged to live in such a fantastic commonwealth with the right leadership and all sectors together, really making our commonwealth a welcoming place. And I do want to highlight the fantastic role of our Attorney General for standing up for our values, but Massachusetts is one state, and it could be an example, but the concern is nation wide. Given a very divisive campaign, and also not just a campaign, but also, what is currently happening at the national level that the current administration is really rejecting this welcoming effort, and the values of our country as a country, who welcomes immigrants. All sectors need to be involved in an effort to really make our society a better one for everyone. And it's going to take political leadership to really set the right tone, send the right message, and really look into the integration, and the welcoming of the newcomers as an investment in our future of our nation. Uber and Lyft have an opportunity here to provide leadership and come up with promotion of policies that integrate the newcomers, or that are welcoming to the newcomers, provide education and training, and train their people. And as troubling as the result of this research are, we like to believe that this is the attitude of the drivers, but not really what the corporate represents, so we see an opportunity for the corporate to really step in and work and promote policies of integration, policies of improvement and betterment for the whole of society and provide an example. Let me say thank you to Professor Knittle for his leadership and MIT for always being a leader, and looking into these issues. But if we can go deeper into A, the size, B, the geography, but also looking into a wider range of all communities that are represented. Looking into the Latino community, looking into the Arab communities in other parts of the nation in a more rigorous, more deep and larger size of research will be very helpful in terms of promoting better policies and integration for everybody who chooses America to be their home. >> That was Eva Millona of the Massechusetts Immigrant and Refugee Advocacy Coalition. Chris, are you confident this problem can in fact be remedied? >> I think we can do better, for sure. And I would say we need more studies like what we just preformed to see how widespread it is. We only studied two cities, we also haven't looked at all at how the driver's race impacts the discrimination. >> Now we're going to turn to you, questions from our viewers. Questions have already been coming in this morning and overnight, lots of great ones. Please use the hashtag MITSloanExpert to pose your question. The first one comes from Justin Wang, who is an MIT Sloan MBA student. He asks, "what policies can sharing economy startups "implement to reduce racial bias?" >> Well, I would say the first thing is to be aware of this. I think Uber and Lyft and Airbnb potentially were caught off guard with the amount of discrimination that was taking place. So the research that we preformed, and the research on Airbnb gives new startups a head start on designing their platforms. >> Just knowing that this is an issue. >> Knowing it's an issue, and potentially designing their platforms to think of ways to limit the amount of discrimination. >> Another question, did you look at gender bias? Do you have any indication that drivers discriminate based on gender? >> We did look at gender bias. The experiments weren't set up to necessarily nail that, but one thing that we found, for example in Boston, is that there is some evidence that women drivers were taken on longer trips. Again, both the male and the female RAs are going from the same point A to the same point B. >> Rebecca: That was a controlled part of the setting. >> That was the controlled part of the experiment. And we found evidence that women passengers were taken on longer trips and in fact, one of our RAs commented that she remembers going through the same intersection three times before she finally said something to the driver. >> And you think... So you didn't necessarily study this as part of it, but do you have any speculation, conjecture about why this was happening? >> Well, there's two potential motives. One is a financial motive that, by taking the passenger on a longer drive. They potentially get a higher fare. But I've heard anecdotal evidence that a more social motive might also be at play. For example, I have a colleague here at Sloan, who's told me that she's been asked out on dates three times while taking Uber and Lyft rides. >> So drivers taking the opportunity to flirt a little bit. >> Chris: Sure. >> Another question, can you comment on the hashtag DeleteUber campaign? This of course, is about the backlash against Uber responding that it was intending to profit from President Trump's executive order, the banning immigrants and refugees from certain countries from entering the United States. Uber maintains that its intentions were misunderstood, but it didn't stop the hashtag DeleteUber campaign. >> Yeah, I haven't followed that super closely, but to me it seems like Uber's getting a bit of a bad rap. One potential reason why they allowed Uber drivers to continue working is that, maybe they wanted to bring protesters to the airports to protest. So from that perspective, actually having Uber and Lyft still in business would be beneficial. >> Another question, did your study take into account the race of the drivers themselves? >> We actually we not allowed to. So any time you do a randomized control trial in the field like this, you have to go through a campus committee that approves or disapproves the research, and they were worried that if we collected information on the driver, that potentially, Uber and Lyft could go back into their records and find the drivers that discriminate, and then have penalties assigned to those drivers. >> So it just wouldn't be allowed to... >> At least in this first phase, yeah. They didn't want us to collect those data. >> Last question, we have time for one more. Why aren't there more experiments in the field of applies economics like this one? That's a good question. >> That's a great question, and in fact, I think many of us are trying to push experiments as much as possible. My other line of research is actually in energy and climate change research, and we've been- >> Rebecca: You like the hot topic. (lauhging) >> We've been designing a bunch of experiments to look at how information impacts consumers' choices in terms of what cars to buy, how it impacts their use of electricity at home. And experiments, randomized control trials actually started in developmental economics, where MIT has actually pioneered their use. And again, it's the best way to actually test, the most rigorous way to test whether intervention actually has an effect because you have both the controlled group and the treatment group. >> So why aren't they done more often? >> Well, it's tough, often you need to find a third party, for example, we didn't need a third party in the sense that we could just send RAs out with Uber and Lyft. But if we wanted to do anything with the drivers, for example, an information campaign, or if we wanted to change the platform at all, we would've needed Uber and Lyft to partner with us, and that can sometimes be difficult to do. And also experiments, let's be honest, are pretty expensive. >> Expensive because, you obviously weren't partnered with Uber and Lyft for this one, but... >> Right, but we had research assistants take 1500 Uber and Lyft rides, so we had to pay for each of those rides, and we also had to give them an hourly rate for their time. >> Well, Chris Knittle, thank you so much. This has been great talking to you, and you've given us a lot to think about. >> It's been fun, thanks for having me. >> And thank you for joining us on this edition of the MIT Sloan Expert Series. We hope to see you again soon.