>> Announcer: Live, from Midtown Manhattan, it's theCUBE. Covering Big Data, New York City, 2017. Brought to you by SiliconANGLE Media and its ecosystem sponsors. >> Okay welcome back, everyone. Live in Manhattan, New York City, in Hell's Kitchen, this is theCUBE's special event, our annual CUBE-Wikibon Research Big Data event in Manhattan. Alongside Strata, Hadoop; formerly Hadoop World, now called Strata Data, as the world continues. This is our annual event; it's our fifth year here, sixth overall, wanted to kind of move from uptown. I'm John Furrier, the co-host of theCUBE, with Peter Burris, Head of Research at SiliconANGLE and GM of Wikibon Research. Our next guest is Tim Smith, who's the SVP of technical operations at AppNexus; technical operations for large scale is an understatement. But before we get going; Tim, just talk about what AppNexus as a company, what you guys do, what's the core business? >> Sure, AppNexus is the second largest digital advertising marketplace after google. We're an internet technology company that harnessed, we harness data and machine learning to power the companies that comprise the open internet. We began by building a powerful technology platform, in which we embedded core capabilities, tools and features. With me so far? >> Yeah, we got it. >> Okay, on top of that platform, we built a core suite of cloud-based enterprise products that enable the buying and selling of digital advertising, and a scale-transparent and low-cost marketplace where other companies can transact; either using our enterprise products, or those offered by other companies. If you want to hear a little about the daily peaks, peak feeds and speeds, it is Strata, we should probably talk about that. We do about 11.8 billion impressions transacted on a daily basis. Each of those is a real-time auction conducted in a fraction of a second, well under half a second. We see about 225 billion impressions per day, and we handle about 5 million queries per second at peak load. We produce about 150 terabytes of data each day, and we move about 400 gigabits into and out of the internet at peak, all those numbers are daily peaks. Makes sense? >> Yep. >> Okay, so by way of comparison, which might be useful for people, I believe the NYSE currently does roughly 2 million trades per day. So if we round that up to 3 million trades a day and assume the NYSE were to conduct that volume every single day of the year; 7 days a week, 365 days a year, that'd be about a billion trades a year. Similarly, I believe Visa did about 28-and-a-half billion transactions in their fiscal third quarter. I'll round that up to 30 billion, and average it out to about 333 million transactions per day and annualize it to about 4 billion transactions per year. Little bit of math, but as I mentioned, AppNexus does an excess of 10 billion transactions per day. And so it seems reasonable to say that AppNexus does roughly 10 times the transaction volume in one day, than the NYSE does in a year. And similarly, it seems reasonable to say that AppNexus daily does more than two times the transaction volume that Visa does in a year. Obviously, these are all just very rough numbers based on publicly available information about the NYSE and Visa, and both the NYSE and Visa do far, far more volume than AppNexus when measured in terms of dollars. So given our volumes, it's imperative that AppNexus does each transaction with the maximum efficiency and lowest reasonable possible cost, and that is one of the most challenging aspects of my job. >> So thanks for spending the time to give the overview. There's a lot of data; I mean 10 billion a day is massive volume. I mean the internet, and you see the scale, is insane. We're in a new era right now of web-scale. We've seen it in Facebook, and it's enormous. It's only going to get bigger, right? So on the online ad tech, you guys are essentially doing like a Google model, that's not everything but Google, which is still huge numbers. Then you include Microsoft and everybody else. Really heavy lifting, IT-like situation. What's the environment like? And just talk about, you know, what's it like for you guys. Because you got a lot of opp's, I mean terms of dev opp's. You can't break anything, because that 10 billion transaction or near, it's a significant impact. So you have to have everything buttoned-up super tight, yet you got to innovate and grow with the future growth. What's the IT environment like? >> It's interesting. We have about 8,000 servers spread across about seven data centers on three continents, and we run, as you mentioned, around the clock. There's no closing bell; downtime is not acceptable. So when you look at our environment, you're talking about four major categories of server complexes. We have real-time processing, which is the actual ad serving. We have a data pipeline, which is what we call our big data environment. We also have client-facing environment and an infrastructure environment. So we use a lot of different tools and applications, but I think the most relevant ones to this discussion are Hadoop and its friends HDFS, and Hive and Spark. And then we use the Vertica Analytics Platform. And together Hadoop and its friends, and Vertica comprise our entire data pipeline. They're both very disk-intensive. They're cluster based applications, and it's a lot of challenge to keep them up and running. >> So what are some of those challenges? Just explain a little bit, because you also have a lot of opportunity. I mean, it's money flowing through the air, basically; digital air, if you will. I mean, they got a lot of stuff happening. Take us through the challenges. >> You know, our biggest apps are all clustered. And all of our clusters are built with commodity servers, just like a lot of other environments. The big data app clusters traditionally have had internal disks, while almost all of our other servers are very light on disk. One of the biggest challenges is, since the server is the fundamental building block of a cluster, then regardless of whether you need more compute or more storage, you always have to add more servers to get it. That really limits flexibility and creates a lot of inefficiencies, and I really, really am obsessive about reducing and eliminating inefficiencies. So, with me so far? >> Yep. >> Great. The inefficiencies result from two major factors. First, not all workloads require the same ratio of compute to storage. Some workloads are more compute-intensive, and others are really less dependent on storage, while other workloads require a lot more storage. So we have to use standard server configurations and as a result, we wind up with underutilized compute and storage. This is undesirable, it's inefficient, yet given our scale, we have to use standardized configurations. So that's the first big challenge. The second is the compute to disk ratio. It's generally fixed when you buy the servers. Yes, we can certainly add more disks in the field, but that's a labor intensive, and it's complicated from a logistics and an asset management standpoint, and you're fundamentally limited by the number of disk slots in the server. So now you're right back into the trap of more storage requires more servers, regardless of whether you need more compute or not. And then you compound the inefficiencies. >> Couldn't you just move the resources from, unused resources, from one cluster to the other? >> I've been asked that a lot; and no, it's just not that simple. Each application cluster becomes a silo due to its configuration of storage and compute. This means you just can't move servers from clusters because the clusters are optimized for the workloads, and the fact that you can't move resources from one cluster to another, it's more inefficiencies. And then they're compounded over time since workloads change, and the ideal ratio of compute-to-storage changes. And the end result is unused resources trapped in silos and configurations that are no longer optimized for your workload. And there's only really one solution that we've been able to find. And to paraphrase an orator far, far more talented than I am, namely Ronald Reagan, we need to open this gate, tear down these silos. The silos just have to go away. They fundamentally limit flexibility and efficiency. >> What were some of the other issues caused by using servers with internal drives? >> You have more maintenance, you've got to deal with the logistics. But the biggest problem is service and storage have significantly different life cycles. Servers typically have a three year life cycle before they're obsolete. Storage typically is four to six years. You can sometimes stretch that a little further with the storage. Inside the servers that are replaced every 3 years, we end up replacing storage before the end of its effective lifetime; that's inefficient. Further, since the storage is inside the servers, we have to do massive data migrations when we replace servers. Migrations, they're time consuming, they're logistically difficult, and they're high risk. >> So how did DriveScale help you guys? Because you guys certainly have a challenging environment, you laid out the the story, and we appreciate that. How did DriveScale help you with the challenges? >> Well, what we really wanted to do was disaggregate storage from servers, and DriveScale enables us to do that. Disaggregating resources is a new term in the industry, but I think lot of people are focusing on it. I can explain it if you think that would make sense. >> What do you mean by disaggregating resources? Can you explain that, and how it works? >> Sure, so instead of buying servers with internal drives, we now buy diskless servers with JBODs. And DriveScale lets us easily compose servers with whatever amount of disk storage we need, from the server resource pool and the disk resource pool; and they're separate pools. This means we have the right balance of compute and storage for each workload, and we can easily adjust it over time. And all of this is done via software, so it's easy to do with a GUI or in our case, at our scale, scripting. And it's done on demand, and it's much more efficient. >> How does it help you with the underutilized resource challenge you mentioned earlier? >> Well, since we can add and remove resources from each cluster, we can manage exactly how much compute power and storage is deployed for each workload. Since this is all done via software, it can be done quickly and easily. We don't have to send a technician into a data center to physically swap drives, add drives, move drives. It's all done via software and it's very, very efficient. >> Can you move resources between silos? >> Well, yes and no. First off, our goal is no more silos. That said, we still have clusters, and once we completely migrate to DriveScale, all of our compute and storage resources will be consolidated into just a few common pools. And disk storage will no longer differentiate pools; thus, we have fewer pools. For more, we have fewer pools and can use the resources in each pool for more workloads. And when our needs change and they always do, we can reallocate resources as needed. >> What of the life cycle management challenge? How you guys address that? >> Well that's addressed with DriveScale. The compute and the storage are now disaggregated or separated into diskless servers and JBODs, so we can upgrade one without touching the other. We want to upgrade servers to take advantage of new processors or new memory architectures, we just replace the servers, re-combine the disks with the new servers, and we're back up and operating. It saves the cost of buying new disks when we don't need to, and it also simplifies logistics and reduces risk, as we no longer have to run the old plant and the new plant concurrently, and do a complicated data migration. >> What about this qualifying server and storage vendors? Do you still do that? Or how's that impact -- >> We actually don't have to do it. We're still using the same server vendor. We've used Dell for many, many years, we continue to use them. We are using them for storage and there was no real work, we just had to add DriveScale into the mix. >> What's it like working with DriveScale? >> They're really wonderful to work with. They have a really seasoned team. They were at Sun Microsystems and Cisco, they built some of the really foundational products that changed the internet, that the internet was built on. They're really talented, they really bright, and they're really focused on customer success. >> Great story, thanks for sharing that. My final question for you is, you guys have a very big, awesome environment, you've got a lot of scale there. It's great for a startup to get into an environment like this, because one, they could get access to the data, work with a good team like you have. What's it like working with a startup? >> You know it's always challenging at first; too many things to do. >> They got talented guys. Most of the startups, those early day startups, they got all their A players out there. >> They have their A players, and we've been very pleased working with them. We're dealing with the top talent, some of the top talent in the industry, that created the industry. They have a proven track record. We really don't have any concerns, we know they're committed to our success and they have a great team, and great investors. >> A final, final question. For your friends out there are watching, and other practitioners who are trying to run things at scale with a cloud. What's your advice to them? You've been operating at scale, and a lot of, billions of transactions, I mean huge; it's only going to get bigger. Put your IT friendly advice hat on. What's the mindset of operators out there, technical op's, as dev op's comes in seeing a lot of that. What do people need to be thinking about to run at scale? >> There's no magic silver bullet. There's no magic answers. The public cloud is very helpful in a lot of ways, but you really have to think hard about your economics, you have to think about your scale. You just have to be sure that you're going into each decision knowing that you've looked at the costs and the benefits, the performance, the risks, and you don't expect there to be simple answers. >> Yeah, there's no magic beans as they say. You've got to make it work for the business. >> No magic beans, I wish there were. >> Tim, thanks so much for the story. Appreciate the commentaries. Live coverage at Big Data NYC, it's theCUBE. Be back with more after this short break. (upbeat techno music)

(electronic music) >> Live from Toronto, Canada it's the Cube! Covering Blockchain Futurist Conference 2018. Brought to you by The Cube! >> Hello, welcome back. Cube exclusive coverage here in Toronto for the untraceable Blockchain Futurist Conference. Two days of wall-to-wall with the Cube. I'm John Furrier, my co-host Dave Valante, we're initiating this Blockchain coverage to all 2018 Cube events all around the world. You'll see us more and more talking to the most important people. Excited to have, here at The Cube, San Kim, CEO of Lucidity. on the front page of siliconangle.com, our journalism team, with news. Also doing the really interesting Blockchain advertising, if you can believe what that could be. We know about Brave and the attention token, a lot of activity going around on what is the benefit to the user around advertising. Certainly having having immutability and data might be interesting. Sam, welcome to The Cube >> Thank you. >> So, first of all, big news today on Silicon Angle. We covered you guys, you guys announced a strategic investor. >> Yes. >> What's the hard news? >> Yeah, well, thank you for covering us today. Today we announced our initial funding and our strategic investor is Pythia. Pythia represents the hard chain foundation, and so we're really excited about this opportunity, We believe our chain represents an incredible advancement of base protocol layers and so, we're looking, we'll be supporting them as we go forward, as we work closely with Pythia, our chain, and that community. >> Tell me about what you guys offer taken specific context, folks may or may not be familiar with what you do. What's the basic premise of your opportunity, technology and problems that you solve, and how do you use Blockchain for that? Yeah, so, we started, we were a digital advertising protocol. Effectively, we are a shared ledger for the digital advertising ecosystem, and if you know digital advertising, it operates at a tremendous scale. And so we have to build this Layer 2 technology that sits on top of the traditional, the base layer protocols, like Ethereum and Archain. In order to address the three challenges. The three challenges, one being scalability, the second is difficulty in sharing privacy, and the third is the high overhead cost of decentralizing a network. And so we've built this Layer 2 technology that uses a plasma sidechain, and we use something called a time series database, that solves those three problems. And, we're looking to support additional chains, in addition to Ethereum, and so obviously our chain is a natural extension for us. >> Yeah, and you guys obviously get, we cover you guys from a broad perspective, that's a big problem in advertising. >> But are you guys charting the user value proposition, or the digital marketer or agency proposition, or both? >> Yeah, so we're not trying to tokenize digital advertising. Our token is basically used internally as a proof of stake token. So, the advertiser, we're asking them to pay in fiat, and we convert that into a stable coin. And on our current instincts, it's the Dai token by MakerDAO. And so, what we are trying to solve is the transparency issue, that's rampant in the supply chain. So for example, when you run a digital ad today, you use anywhere from seven to 15 vendors, and those vendors, each of them have their own database, and they never communicate that data across to each other, and so there's discrepancies, and it also opens itself up to a lot of fraud. And so the industry is a 225 billion dollar industry, and the industry itself estimates that there's, like, 30% of that money is wasted. And a lot of that is because there's no reconciliation of that data, there's no transparency, and so we've created this protocol layer, for all 15 vendors to submit their data. And, in real time, we can understand, which impressions were valid, which ones were fraudulent, and, well, not just transparency, but now that we as industry participants don't have to argue with one another, we'll start to trust one another, and then we can move the industry forward. >> In the market it'll adjust the pricing as a result of that as well, right? >> Oh, absolutely, absolutely, and it's just about identifying where is the value created, right? So if you're a value creator in the supply chain, you could probably estimate that, the advertiser's going to eliminate the less valuable ones, and focus on the valuable and the adding ones. So basically, if you're fraudulent, like yeah, you might get hurt, but the real adders will benefit from it. >> Just to clarify a question, you talked about the overheads of decentralizing advertising. I infer from that that an advertising supply chain, by its inherent nature is decentralized? Or are you talking about more of a disruptive model? Can you explain? >> Yeah, so we're not re-creating a whole ecosystem, >> Right >> We're interoperable with the existing architecture. >> Which, is decentralized by its very nature, you're saying, or...? >> No, no, no, it's not decentralized >> Okay >> It's very centralized, like all the metrics are controlled by a few players. >> So it's no seven people in the supply chain, that form that central entity... >> Yes, it's all central entities, and we're asking them to submit their data, into this shared ledger, that works across all of the different industry structures. >> So it is disrupting that... >> Oh, it's highly disruptive in terms of that, but we're not trying to re-create the infrastructure like a lot of other blockchain architect companies. >> Oh, I see, so you're tapping into the existing, and you're providing good auditing, I imagine with this, right, so the benefit might be auditing. So give an example of how that would render itself. >> Yeah, so, one of the areas that we're focused on today, is just looking at the impressions, in a programmatic ad buying. And so, let's say, let's just focus, instead of talking about the 15 vendors, let's just talk about the four. The four is the advertiser, is the DSP, which is basically the buying platform, the SSP, which also represents the exchange, and then the publisher. Now there is, we were asked that all four submit their data into the smart contract, and we verify whether that impression was valid. If you think of a fraudulent example, like a bot, they will not be able to mimic the data across the whole supply chain. And so because we're looking at the data wholistically, rather than just the slices of it, we can identify those fraudulent behaviors. >> This is the benefit of horizontally scalable, integrated systems. Cloud can help you, Blockchain helps you. How's the uptake been? Give us an update on who's involved, what's been the successes, and how's your success going? >> So we've been really excited to work with the IAB, and the IAB stands for the Interactive Advertisement Bureau. They're the bodies that set standards in digital advertising and we're working very closely with them. We launched our pilot, the first official pilot with the IAB, and we have great advertisers that are working with us, we're working with a lot of the agencies, we're actually even working closely with the publishers, and the ad networks, and the exchangers. AppNexus is one of the major partners with us, and the reception's been really positive because I think everybody wants that transparency. >> Well, some of the status quo might not want that transparency, I mean, let's face it, right? >> The fraud is rampant, it really is. >> A 220 billion dollar industry, I betcha there's a lot of people in it that are like, oh boy, here comes lucidity! I mean, come on, what about that? >> I'm sure that exists, but we haven't really come across it because the advertiser, at the end of the day, has become really aware that there is this rampant fraud, there is this waste. And I don't want to attribute everything to fraud, I think some of it is just wasted, because of the quality of the data. And so, the advertiser is demanding and at the end of the day, we're here to serve the advertiser, right? We're here to deliver value to the advertiser, and I think the industry is mature enough now, to where we recognize that. And so we don't think of transparency as a threat to the business anymore. We think of it as a value enhancement to our customer, the advertiser. >> Yeah, and I would personally totally agree with that, because as I said, the market will correct itself. Higher quality advertising is going to deliver more revenue, ultimately, alone, because there's going to be better outcomes. Right, so if you can increase your hit rate, you'd be happy to lower the clicks, you know? >> Is there any benefit for publishers? >> Yeah, I mean, publishers today have to basically trust what their partners are paying them. There's no way for them to verify and validate it. And so, with our system, we enable publishers to look into, it's our sidechain, right? And so, they are able to look at the events, but we obscure the data, we hash the data that's there so that we make it anonymous. But then they're able to see, like, okay, these are the impressions I've manned, here are the ones that were considered valid and verified, and here's what I should get paid. So the publishers now get the transparency, that which they lack today. >> So much of that industry is a black box, you might have a big media buyer, who's got voodoo, you know, that sprinkles magic dust, sends you a big bill, and you're like whoa! Is this really worth it? >> Bots, fake traffic.. >> You can automate a lot of that... >> And you've been doing this for 20 years! This has been the status quo for 20 years! >> We need a change. So, talk about the company, how big, how much funding did you actually owe? Is it privately funded, what's the funding mechanism? How big are you guys, what's the story? >> So today we announced that we raised five million dollars, we did it in traditional means. We did not do an ICO. >> Venture capital? >> It's a mix of venture capital, and obviously Pythia is the fund for our chain, so, but it was an equity deal. And that's the brow we're going to continue with. We do have an internal token, but we are not looking at doing a public sale. >> So not a security token, preferred stock, classic funding. >> So wait, so you did a security token? >> No no, no, preffered stock, classic venture capital. Well, great! Yeah, that's awesome, congratulations. We'll keep in touch, it's great to have you come on. >> Thank you very much >> Thanks very much, appreciate the time. >> And thank you for covering us! >> Of course! We love innovative things, in advertising specifically because it's freaking broken, big time! We have no advertising on our site, because we want to get the best content possible. Of course, the Cube is supported by sponsors, we appreciate that. Thanks for coming on. Cube coverage here in Toronto for watching futurists, we'll be right back, stay with us, as we start to wind down day one. Be right back with more great interviews after this break. (light-hearted techno music)