>> My name is Franco Nori. Is a great pleasure to be here and I thank you for attending this meeting and I'll be talking about some of the work we are doing within the NTT-PHI group. I would like to thank the organizers for putting together this very interesting event. The topics studied by NTT-PHI are very exciting and I'm glad to be part of this great team. Let me first start with a brief overview of just a few interactions between our team and other groups within NTT-PHI. After this brief overview or these interactions then I'm going to start talking about machine learning and neural networks applied to computationally difficult problems in quantum physics. The first one I would like to raise is the following. Is it possible to have decoherence free interaction between qubits? And the proposed solution was a postdoc and a visitor and myself some years ago was to study decoherence free interaction between giant atoms made of superconducting qubits in the context of waveguide quantum electrodynamics. The theoretical prediction was confirmed by a very nice experiment performed by Will Oliver's group at MIT was probably so a few months ago in nature and it's called waveguide quantum electrodynamics with superconducting artificial giant atoms. And this is the first joint MIT Michigan nature paper during this NTT-PHI grand period. And we're very pleased with this. And I look forward to having additional collaborations like this one also with other NTT-PHI groups, Another collaboration inside NTT-PHI regards the quantum hall effects in a rapidly rotating polarity and condensates. And this work is mainly driven by two people, a Michael Fraser and Yoshihisa Yamamoto. They are the main driving forces of this project and this has been a great fun. We're also interacting inside the NTT-PHI environment with the groups of marandI Caltech, like McMahon Cornell, Oliver MIT, and as I mentioned before, Fraser Yamamoto NTT and others at NTT-PHI are also very welcome to interact with us. NTT-PHI is interested in various topics including how to use neural networks to solve computationally difficult and important problems. Let us now look at one example of using neural networks to study computationally difficult and hard problems. Everything we'll be talking today is mostly working progress to be extended and improve in the future. So the first example I would like to discuss is topological quantum phase transition retrieved through manifold learning, which is a variety of version of machine learning. This work is done in collaboration with Che, Gneiting and Liu all members of the group. preprint is available in the archive. Some groups are studying a quantum enhanced machine learning where machine learning is supposed to be used in actual quantum computers to use exponential speed-up and using quantum error correction we're not working on these kind of things we're doing something different. We're studying how to apply machine learning applied to quantum problems. For example how to identify quantum phases and phase transitions. We shall be talking about right now. How to achieve, how to perform quantum state tomography in a more efficient manner. That's another work of ours which I'll be showing later on. And how to assist the experimental data analysis which is a separate project which we recently published. But I will not discuss today because the experiments can produce massive amounts of data and machine learning can help to understand these huge tsunami of data provided by these experiments. Machine learning can be either supervised or unsupervised. Supervised is requires human labeled data. So we have here the blue dots have a label. The red dots have a different label. And the question is the new data corresponds to either the blue category or the red category. And many of these problems in machine learning they use the example of identifying cats and dogs but this is typical example. However, there are the cases which are also provides with there are no labels. So you're looking at the cluster structure and you need to define a metric, a distance between the different points to be able to correlate them together to create these clusters. And you can manifold learning is ideally suited to look at problems we just did our non-linearities and unsupervised. Once you're using the principle component analysis along this green axis here which are the principal axis here. You can actually identify a simple structure with linear projection when you increase the axis here, you get the red dots in one area, and the blue dots down here. But in general you could get red green, yellow, blue dots in a complicated manner and the correlations are better seen when you do an nonlinear embedding. And in unsupervised learning the colors represent similarities are not labels because there are no prior labels here. So we are interested on using machine learning to identify topological quantum phases. And this requires looking at the actual phases and their boundaries. And you start from a set of Hamiltonians or wave functions. And recall that this is difficult to do because there is no symmetry breaking, there is no local order parameters and in complicated cases you can not compute the topological properties analytically and numerically is very hard. So therefore machine learning is enriching the toolbox for studying topological quantum phase transitions. And before our work, there were quite a few groups looking at supervised machine learning. The shortcomings that you need to have prior knowledge of the system and the data must be labeled for each phase. This is needed in order to train the neural networks . More recently in the past few years, there has been increased push on looking at all supervised and Nonlinear embeddings. One of the shortcomings we have seen is that they all use the Euclidean distance which is a natural way to construct the similarity matrix. But we have proven that it is suboptimal. It is not the optimal way to look at distance. The Chebyshev distances provides better performance. So therefore the difficulty here is how to detect topological quantifies transition is a challenge because there is no local order parameters. Few years ago we thought well, three or so years ago machine learning may provide effective methods for identifying topological Features needed in the past few years. The past two years several groups are moving this direction. And we have shown that one type of machine learning called manifold learning can successfully retrieve topological quantum phase transitions in momentum and real spaces. We have also Shown that if you use the Chebyshev distance between data points are supposed to Euclidean distance, you sharpen the characteristic features of these topological quantum phases in momentum space and the afterwards we do so-called diffusion map, Isometric map can be applied to implement the dimensionality reduction and to learn about these phases and phase transition in an unsupervised manner. So this is a summary of this work on how to characterize and study topological phases. And the example we used is to look at the canonical famous models like the SSH model, the QWZ model, the quenched SSH model. We look at this momentous space and the real space, and we found that the metal works very well in all of these models. And moreover provides a implications and demonstrations for learning also in real space where the topological invariants could be either or known or hard to compute. So it provides insight on both momentum space and real space and its the capability of manifold learning is very good especially when you have the suitable metric in exploring topological quantum phase transition. So this is one area we would like to keep working on topological faces and how to detect them. Of course there are other problems where neural networks can be useful to solve computationally hard and important problems in quantum physics. And one of them is quantum state tomography which is important to evaluate the quality of state production experiments. The problem is quantum state tomography scales really bad. It is impossible to perform it for six and a half 20 qubits. If you have 2000 or more forget it, it's not going to work. So now we're seeing a very important process which is one here tomography which cannot be done because there is a computationally hard bottleneck. So machine learning is designed to efficiently handle big data. So the question we're asking a few years ago is chemistry learning help us to solve this bottleneck which is quantum state tomography. And this is a project called Eigenstate extraction with neural network tomography with a student Melkani , research scientists of the group Clemens Gneiting and I'll be brief in summarizing this now. The specific machine learning paradigm is the standard artificial neural networks. They have been recently shown in the past couple of years to be successful for tomography of pure States. Our approach will be to carry this over to mixed States. And this is done by successively reconstructing the eigenStates or the mixed states. So it is an iterative procedure where you can slowly slowly get into the desired target state. If you wish to see more details, this has been recently published in phys rev A and has been selected as a editor suggestion. I mean like some of the referees liked it. So tomography is very hard to do but it's important and machine learning can help us to do that using neural networks and these to achieve mixed state tomography using an iterative eigenstate reconstruction. So why it is so challenging? Because you're trying to reconstruct the quantum States from measurements. You have a single qubit, you have a few Pauli matrices there are very few measurements to make when you have N qubits then the N appears in the exponent. So the number of measurements grows exponentially and this exponential scaling makes the computation to be very difficult. It's prohibitively expensive for large system sizes. So this is the bottleneck is these exponential dependence on the number of qubits. So by the time you get to 20 or 24 it is impossible. It gets even worst. Experimental data is noisy and therefore you need to consider maximum-likelihood estimation in order to reconstruct the quantum state that kind of fits the measurements best. And again these are expensive. There was a seminal work sometime ago on ion-traps. The post-processing for eight qubits took them an entire week. There were different ideas proposed regarding compressed sensing to reduce measurements, linear regression, et cetera. But they all have problems and you quickly hit a wall. There's no way to avoid it. Indeed the initial estimate is that to do tomography for 14 qubits state, you will take centuries and you cannot support a graduate student for a century because you need to pay your retirement benefits and it is simply complicated. So therefore a team here sometime ago we're looking at the question of how to do a full reconstruction of 14-qubit States with in four hours. Actually it was three point three hours Though sometime ago and many experimental groups were telling us that was very popular paper to read and study because they wanted to do fast quantum state tomography. They could not support the student for one or two centuries. They wanted to get the results quickly. And then because we need to get these density matrices and then they need to do these measurements here. But we have N qubits the number of expectation values go like four to the N to the Pauli matrices becomes much bigger. A maximum likelihood makes it even more time consuming. And this is the paper by the group in Inns brook, where they go this one week post-processing and they will speed-up done by different groups and hours. Also how to do 14 qubit tomography in four hours, using linear regression. But the next question is can machine learning help with quantum state tomography? Can allow us to give us the tools to do the next step to improve it even further. And then the standard one is this one here. Therefore for neural networks there are some inputs here, X1, X2 X3. There are some weighting factors when you get an output function PHI we just call Nonlinear activation function that could be heavy side Sigmon piecewise, linear logistic hyperbolic. And this creates a decision boundary and input space where you get let's say the red one, the red dots on the left and the blue dots on the right. Some separation between them. And you could have either two layers or three layers or any number layers can do either shallow or deep. This cannot allow you to approximate any continuous function. You can train data via some cost function minimization. And then there are different varieties of neural nets. We're looking at some sequel restricted Boltzmann machine. Restricted means that the input layer speeds are not talking to each other. The output layers means are not talking to each other. And we got reasonably good results with the input layer, output layer, no hidden layer and the probability of finding a spin configuration called the Boltzmann factor. So we try to leverage Pure-state tomography for mixed-state tomography. By doing an iterative process where you start here. So there are the mixed States in the blue area the pure States boundary here. And then the initial state is here with the iterative process you get closer and closer to the actual mixed state. And then eventually once you get here, you do the final jump inside. So you're looking at a dominant eigenstate which is closest pure state and then computer some measurements and then do an iterative algorithm that to make you approach this desire state. And after you do that then you can essentially compare results with some data. We got some data for four to eight trapped-ion qubits approximate W States were produced and they were looking at let's say the dominant eigenstate is reliably recorded for any equal four, five six, seven, eight for the ion-state, for the eigenvalues we're still working because we're getting some results which are not as accurate as we would like to. So this is still work in progress, but for the States is working really well. So there is some cost scaling which is beneficial, goes like NR as opposed to N squared. And then the most relevant information on the quality of the state production is retrieved directly. This works for flexible rank. And so it is possible to extract the ion-state within network tomography. It is cost-effective and scalable and delivers the most relevant information about state generation. And it's an interesting and viable use case for machine learning in quantum physics. We're also now more recently working on how to do quantum state tomography using Conditional Generative Adversarial Networks. Usually the masters student are analyzed in PhD and then two former postdocs. So this CGANs refers to this Conditional Generative Adversarial Networks. In this framework you have two neural networks which are essentially having a dual, they're competing with each other. And one of them is called generator another one is called discriminator. And there they're learning multi-modal models from the data. And then we improved these by adding a cost of neural network layers that enable the conversion of outputs from any standard neural network into physical density matrix. So therefore to reconstruct the density matrix, the generator layer and the discriminator networks So the two networks, they must train each other on data using standard gradient-based methods. So we demonstrate that our quantum state tomography and the adversarial network can reconstruct the optical quantum state with very high fidelity which is orders of magnitude faster and from less data than a standard maximum likelihood metals. So we're excited about this. We also show that this quantum state tomography with these adversarial networks can reconstruct a quantum state in a single evolution of the generator network. If it has been pre-trained on similar quantum States. so requires some additional training. And all of these is still work in progress where some preliminary results written up but we're continuing. And I would like to thank all of you for attending this talk. And thanks again for the invitation.

>> from Boston, Massachusetts. It's the queue covering W T. G transformed 2019 by Winslow Technology Group. >> Hi. I'm Stew Minutemen. And this is the Cuban W. T. G. Transformed 2019 here home game in Boston, Massachusetts, our third year. The event happened a Welcome back to the program. Second time on the program in less than a year. Matt Kozlowski, Who's the vice president? Professional services, Winslow Technology Group. Thanks so much for joining. Thank you. Alright, uh, second tie I've had on the program, but first vest and cufflinks you like today. So, you know, showing your own individual style for, >> like, the Ted talk. Look, >> Absolutely. So we will keep this under 18 minutes. Okay? Probably be more like about 12 theirs and no slide. But you tell us a story of change and inspiration. Uh, you know, in all seriousness there what? I actually want to hear the story of change that we're seeing inside of Winslow attack. So, um, you know, question I asked, You know, some of your peers in the company is, you know, if I thought about Winslow attack, you know, just a couple of years ago, it's like, Oh, hey, great deal, partner. No, the pellet side, you know, picking up the servers and some of the other pieces. Yeah, Here, you bring it on Brook board on board. You know, professional services security. Uh, you know, tell us a little bit about you know what? What were you doing since last time we caught up? >> Sure. So if you think about years ago where we had not just winslow but like bars as a whole came from it was, like, way sell boxes and we sell things. And now we're transitioning where people are using cloud or the hybrid cloud models. And they're actually using software in infrastructure as services and way need, like professional services and consulting to help people on that journey. That's like the simplified version of it. >> Yeah, and just, you know, I want to play something back for you and see if it resonates with you. You know, if I go back, you know, let's say 5 to 10 years ago, it was, you know, we get the boxes and the bar gets it, and they've got to spend a lot of work to configure it and do all the pieces. And, you know, that kind of day. One roll out when we talked about OK, how many months from when the equipment got to the bar versus when we're up and running? When we rolled out converged infrastructure, hyper converged infrastructure and all this cloudy stuff, it actually shifted things backwards. Now, before it gets there, there's a lot of work that either the customer or the partner with the customer needs to do so. It shifted it because once it gets on site, well, there's less wiring and cabling. You configuration I need to do. But it just shifted where that engagement service happened. It did not eliminated that what you're saying? >> Yeah, so there's a lot in terms of like planning. I mean, even, like integration work that we do ahead of time. >> I would say things that have changed even over the last, like three or four years is like the complexity of everything is gone up like we're trying to simplify it. We're simplifying maybe the delivery of it and users. But behind the scenes, certainly it's It's more complicated, I would say, than than ever. >> Yeah, you know it. We're no longer just, you know, let's lock the door and Hafiz of Security and put the firewall in place. Right now, it's like, Oh, well, it's micro segmentation in all the places and my application spread out across. You know how many locations, how many services from and therefore write everything has become a little bit >> more and more >> complicated, eh? So how do we make sure we stay secure in 2019? >> So I think there's a couple areas they're so first is, like maintaining that same kind of sense of securing people, infrastructure and things along those lines that we've kind of been doing for a while now that your basic like firewalls and even vulnerability assessments and things like that. But I think over the last couple years and this as we move to like more of like distributed workforce, like people working from home, people working remotely, finding like the right people, there's gonna be more of a focus on like and point protection and, like protecting users at, like the end point >> or the mobile level on them than ever before. >> Um, >> a lot of talking the keynote this morning, amount cloud. Yeah, and you said, you know, where does that put things so, you know, give us from your standpoint. You know, obviously services were hugely important piece of it, you know, a CZ the box. And the location becomes a little bit less important, despite the fact that even when you have things like server list, we know that there's ultimately hardware sure runs underneath it somewhere. You know, what were those Winslow play today and in the future? >> Okay, so I'm gonna give you two kind of conflicting answers to that. So the 1st 1 is, if you look at reasons why people don't go to the cloud, it's there not comfortable in the security of it. I'll say in like the my like, real world, not in the academic or statistical version of it. One of the reasons people do go to the cloud is for security, right? Look a like a lot of health care organizations are goingto like cloud based electronic medical record systems. I feel like that in some ways has insulated or shifted >> some of the burden of the risk and keeping those systems secure to the provider that's hosting them. >> Which is probably better for us, his patients, right, And for the health >> care providers in general. In that case, >> yeah. You know, one of the things we know is that what you need to do as user is you can't just keep doing things the old way because your competition will move faster. Right? And we know from a security standpoint, my friends that aren't even security is like you need to be able to move fast. One of the great things about the cloud is you know, if I'm running on Azure eight of us Hey, that latticed latest patch in that security vulnerability did that get rolled out? Well, I'm not responsible. Yes, they absolutely right. I didn't have to wait for that roll out, you know? So So there's that piece of it. So you know, just how do I keep up obtained? I need to, as as user, do some updates, and therefore, I'm not saying everything goes in the public cloud, but how do I make sure that it's not? Oh, I update my software every two years, or it's I need to make sure that I'm closing those gaps and vulnerabilities of taking advantage of words. I >> think there's going to be like a shift in changing from like normal. CIS admits they're thinking about like patching Windows and patching Lennox and operating systems. But, like once we move information to the cloud and you think about it, more is like information security. So now data is in the cloud. I'm not patching the system's anymore because we'll just assume that, you know, eight of us Microsoft. They're doing a great job with that. But like once data say is in one drive like how my governing, like where that data's going, who's accessing it, who it's being shared with, how it's being backed up things along those lines. It's just a different mindset that people need to adopt, you know, in relation to securing information, not systems. All right, >> man, I'm trying to figure we gotta replace Patch Tuesday with some celebration or some battering event where we can try to tackle some of the some of these new challenges there, You know? What does that mean to some of the changing roles that you're seeing in the customers, though? I guess here here went to attack. You know, I was talking to Arctic wolf in a typical customer, you know, doesn't have their whole security team that runs 24 7 That's where your partner with that. So you know, we're just security fit in. The organization has said, If it was a large enterprise, you know, it's a four level discussion. You know you've got your sea. So where somebody like that, what does the typical kind of mid to small sized company security team look? >> Yeah, it looks like I'm gonna partner with someone. Or that's what it should look like because, like even if companies have like a managed provider, that's doing like patch management and things along those lines, there's something to be said for having like 1/3 party in another party party, like as your security partner, Because if the people that air like doing the patching, they're probably doing a great job at it. But, like you might not want them being the ones also doing like your vulnerability assessments. It's good to have, like different parties in there, So I feel like for smaller medium businesses, it's getting comfortable partnering on and using like professional services. Frankly, Tio to do that. All >> right, so it's really interest Matt next week. Actually, Amazon is holding a cloud security show here in Boston called Reinforced. So, uh, you know, Boston seems an interesting place, You know, the arse. A conference has always been out in San Francisco. Give us kind of the state of security here in the area. >> Okay, so I think I have a unique perspective on this because I'm not from the area. Like I'm from Connecticut. So I come up here. >> You really most people in the United States would be like Connecticut is a suburb of Austin. You know where you are? Yeah, that's that's the one you need to know. Where we are. You on the Yankees Red Sox line that goes down the middle of the state, right? Right around Hartford. >> Yeah, are are like, claim to fame is being in between both city. So yes, um, way do see, though, like Boston emerging as, like, a regional tech hub, if not like the tech hub of the East Coast. Frankly, so I feel like why not have it here? Like, why wouldn't we have it here? Compared to everywhere else? Like there's so many tech companies, and this just doesn't feel like a tech hub of the region's. >> Okay, Well, you know I'm all in favor of things where I could take the trainer drive to rather than have to fly around the president. Huge is part of you Give a session here on Talked about some branch somewhere Give give us so some of the key takeaways and thanks for the audience that they should be thinking about. >> So So in that session, I kind of invented a completely fictional account of a ransomware attack on a hospital. It was Bill on real world scenarios that I just kind of, like merged together. So I would say up front things that I would say that were important to talk about and that we're, you know, cyber security awareness training. I'm making sure people you know are understand. Like the risks involved with female security advance like modern and point protection. We kind of touched on that a little earlier. So, like older, signature based detection is just not not really effective anymore. Um, having a good tamper proof backup strategy is important, too. So let's say, like, systems get ransomware it. Everything's encrypted, like you need a way to restore that data without necessarily paying the ransom on DH like tamperproof backups >> are are the way to do that. Really? So >> all right, that I want to give you the final word. Uh, w t g transform 2019 gives a little inside some of the customers you're talking to. Some of the top of mine, diffuse or any. I don't work >> for me. A lot of the top mine issues around security seriously, but also like modernizing People's Data Center so that delivering on the hybrid cloud message of like installing hardware and software that not just provides, like data storage services on Prem but could do a lot of cloud tearing >> cloud archiving. Also >> because last, we really appreciate the updates. Thank you. Money for Sarah. We're all initiated. I want to thank our audience here. We've had a full day here. Got to talk to some of the users, some of the partners and, of course, our host for the event. Winslow Technology Group. Scott Winslow and the team. Great to see the growth. Always love to be able to dig in with the users and what's happening locally for myself, stupid. And want to thank the whole team here at the Cube for helping us to be ableto support these events and be sure to check out the cute dot net. You could do some searches there. You could find all the guests here and see previously what they've been talking about. See what future events were going out and dig their archive and is always if you have any questions, feel free to reach out myself, the rest of the team and always a pleasure to be able to share with you and thank you for watching.

>> Announcer: From Hell's Kitchen in New York City, it's theCUBE, on the ground at Serverlessconf. Brought to you by Silicon Angle Media. >> Hi, I'm Stu Miniman with theCUBE, here at Serverless Conference in Hell's Kitchen in New York City. Happy to welcome back to the program. keynote speaker at the event, and a guest that we've had on a couple times before, John Willis, who's the vice president of DevOps and digital practices at Eastray Technologies. John. >> In Hell's Kitchen. >> Stu: In Hell's Kitchen, and go Yankees. >> Yeah, man. I was at the game last night, the other night. Yeah. You'll see tonight. Yeah. Thank you. Glad to be here. >> Great to see you. So look, you've been talking to audiences about DevOps for as long as I can remember, as long as I've known you, definitely. Tell us, what's so important about serverless and how that fits into the world of the developer these days. >> Yeah, I mean, my interest, you know, I was invited to do a keynote, and my interest is to break down the tribal nature of new things. And I sound like a hypocrite because I'm the DevOps tribe, but I prefer to stop calling it DevOps, because there are super patterns that exist, and as I watch serverless, I spend a lot of time having these conversations around that yeah, we don't need that DevOps anymore, because we got serverless. It was the same reason like we didn't need any of the infrastructure stuff because we got cloud. And like, we keep throwing the baby out with the bathwater, and my presentation this morning was like, it's not about the technology, stupid. Like the principles of business value, how you understand value stream, how you inject the governance, the policy, the security, the values and the outcomes that you want. I know those sound like platitudes, like I get a sense that we're making the same mistake over again, and hey, sorry folks, Serverless is just another form of compute. Sorry to get you all wound up and then let you down. It's just compute, folks. And so all the core principles that we've really learned about high-performance organizations apply, they apply differently. Monitoring is differently. How do we deliver? But the principles stay the same. And that was my core message today. >> Yeah, no, very passionate, definitely came through in the keynote. I just have to ask you just on the tech for a second, I mean you were heavily involved in containers, you were part of a company that got acquired by Docker, you were a big proponent of unikernels, now it's serverless, how do you kind of paint that picture >> I think it's amazing tech, and more these days. So I left Docker and I'm going back to something I did 10 years ago, which is kind of consulting but transformation type consulting. It sounds platitudish, but like, I'm back in the mode of looking at things at bigger scale. How do you change an organization to think differently about things? So I've kind of taken a little bit of my tech hat off. I mean, I love containers and minimal delivery, right, I've been yacking about that for like the last two or three years, right? About how minimal delivery models work. And serverless is like, amazing too, like unikernels was an interesting model of function as a service. I think serverless will eat up a good portion, you know I've said this, and I don't know, I may have to modify it. You know, I would say four years ago, three years ago, and you guys been a big part of this discussion. The world went to most companies would say we're a cloud-first organization. I've been saying for the last couple of years, I think most organizations should now thinking that they're a container-first organization. So that doesn't say everything, it just means, and I think the world now should be kind of still container first, and I know that might sound horrible to serverless people, but then look at serverless functions as a place where it fits in the architecture, repeatability, and containers. And there's actually kind of a.. >> Is that just from a maturity standpoint, you know, containers a little bit more mature than serverless? >> I don't know that it's, I think there are like, there are models of architecture, right, and I don't know that, I mean I know there's a lot of successful startups in certain value streams and enterprises that are all serverless. I know a couple of friends that have built complete infrastructure on Amazon Lambda. It works. I just don't know that all value stream delivery of services will go complete serverless. I'm pretty certain that today, almost all applications can run on containers. So I'm not creating a division of war. I'm just saying that I think, and I could be dead wrong on this, but I think in this future like placeholder where we're container first, it's going to be, give me an exception of why it can't be containers left, like it has to be cloud, or it has to be bare metal, or it has to be (mumbles) and the right side is about mapping reusable functionality in functions. So I think you have like a container-first world assumes that smart architecture mandates repeatable functions in a function-like world. Does that make sense? >> Yeah, it does. So I think back on my career, there's so many times we said like, oh, we've got this new way to really simplify the environment and get rid of things you don't need to worry about. You know, I lived through the whole virtualization, oh wait, networking storage took us a decade to fix that. >> Yeah, yeah, yeah, yeah. >> Containers, oh we're going to just focus on the application. Oh wait, networking really important, you worked on a whole company focused specifically on that. >> DevOps for networking, yeah. >> Serverless, the question is, what's the rule of operations when it comes to serverless? >> Again, that's my thoughts on serverless and if it ain't right that's secondary to my real passion right now, which is when I hear the word NoOps for serverless, I cringe. Like this idea that you don't... I mean it's different. Do you need observability and telemetry in a serverless world? I ask you. Of course you do. Do you need to have repeatable patterns of delivery to make sure you don't have vulnerabilities in your code? Of course you do. That's Ops folks. And it's about supply chain and building repeatable, structured delivery with all the gates and the checks and the units, and none of that I believe goes away with serverless. Just like it didn't go away with cloud, just the way it didn't go with virtualization, right? So I think you know, we make a big mistake to think serverless means we don't need operations now. Does it mean that our providers, we have a different relationship with our providers? We don't own the server anymore. So we can't run detrace or those kind of things in that environment. But we still own the service. So who's the site reliability engineer for the service that's running on Lambda? Or functions of serverless, right? If it ain't, I mean if you don't got one, like you're going to have a bad service. >> Yeah, what are you hearing organizationally, what's happening in companies that you're talking to? You know, I was a at a show recently, I think it was Kelsey Hightower I think, it was like DevOps is a given at this point. So do you see that, you know, where's the line from what you've seen? >> Well the curse and the blessing of DevOps, the curse is we've never had a clear definition of it. I say we, you know, everybody, but. And the blessing is we've never had a clear definition. Like it's always emerged. And the problem is, I will tell you what my definition of DevOps is, it has really very little to do with technology. It has to do with human capital and how you create high-performing organizations and the principles and practices that lead to that. The DevOps handbook, if you will, is a lot about, that I co-authored with Gene and Patrick and Jez. Those things, that's my definition of DevOps, but the problem is, when you hear people have discussion about DevOps in lieu of a good definition, you can't really get upset when somebody thinks DevOps is like Jenkins and Sheffer Puppet and Ansable, and like oh no, you're wrong, right, like that's their view. So the problem that you run into then is, if your definition is that it's pure technology and it's tied to kind of cloud, and it's something like infrastructure is code, then in your world and your definition, serverless is going to make all that obsolete, or a good portion obsolete. But if your definition is more about how you create patterns and practices around humans who deliver services a certain way, then nothing about serverless makes any of that obsolete. >> All right, Jon, want to give you final word. What do you think people, that you know, just hearing about serverless first time, where do they start, what kind of things should they look at, or you know, if there's other things you think they should probably look at first? >> You know, I think you're asking the wrong guy for that really. I think there's far better people that you've interviewed take care of that. I mean I would go with Peters Brook, the founder of this conference. That was a book I read, he gave me a copy, it made sense to me, I was able to do some labs and then you know, as they say, the rest, Bob's your uncle, you know, there's a ton of stuff out there to figure out how to navigate. >> Anything, any commentary you'd make on the community for here, a couple of people just you know, it's new but very vibrant, reminds me a lot of the emerging tech where, you know, a lot of help from the community, it's pretty easy to get started. >> So yeah, so in the technology, yes. A lot of vendors, a lot of good stuff, great conversations, and I was actually pleasantly surprised there was less discussion about NoOps or you don't need operations, and I got kind of a little bit of a cheer when I mentioned that this morning. So it seems like there are some good lessons learned that I think the message loud and clear is that operations still exist, it just has to be thought about. The keynote yesterday, the gentleman in the keynote yesterday said, day one, closing keynote, said serverless things are different, in some case easier, but harder in other things, and that was through a cloud. Cloud was much easier from getting infrastructure but we ran into a whole lot of operational issues around how to match this cloud to scale. So serverless is easy to create a function, get it set up, cost-effective, but we're starting to learn all of the complex operational issues of MTTR, how do you restore stuff, what does SRE look like, I mean this is why we get paid the big bucks, dammit man. >> All right, John Willis, always a pleasure to catch up with you. I'm Stu Miniman, thank you so much for watching theCUBE.