Miles Ward (00:15):
Hi everybody. My name is Miles Ward, and this is Cloud N Clear. It’s a little conversation we have about what’s happening in public cloud. How Google is making paths forward, what’s happening in technology, and frankly, how much computing can we compute? And I have exactly the right guy for you to talk to about that. Josh, how about you introduce yourself, say hi to everybody.
Josh Hilliker (00:36):
Absolutely. Thank you Miles. So hey everybody, I’m Josh Hilliker, I work at Intel Corporation. Been here almost 27 years, and during that time have gone from IT, which I’m very proud of having my IT stripes for two different tours in IT. Worked in the data center, worked in manufacturing on a certain technology. Acquired companies, kicked off a couple of client products, one of them called Intel vPro Technology, that’s out in the market. Absolute, vPro rocks and I still love vPro. And then working the last decade in the data center, even deeper. And I used to run a DMZ and a data center when I was back in IT.
Josh Hilliker (01:15):
But over the last decade, it’s been really kind of double clicking in on my passion, which is around telemetry and around infrastructure modernization. And what’s great is, it just had a natural draw to the Cloud of, okay, well, that’s great on-prem, what can I do in the Cloud? And what’s possible in the Cloud? And that brought me into this career of going into be a Cloud solution architect. So certified on Google, certified on AWS, Docker and of course our Cloud use certifications. So I’m a Cloud solution architect, a senior Cloud solution architect, also a senior director and I manage our Americas Cloud CSAs across the Americas. As well as I just got my Intel principle engineer title for my work and passion on telemetry. So what’s great is a lot of it kind of gets you right into that mindset of what’s possible in the Cloud and what really matters.
Miles Ward (02:13):
Yeah, that’s incredible background. The work that you’re doing and that I think you’re helping Intel do, is just hyper critical to our customers. Very happy to hear that you’ve passed your architecture certification, I wrote a few of the questions on that test. So happy to be plugging in with somebody that’s of like mind. I’ve been a huge fan of Intel’s contributions in this area, as they are such a massive part of what public Cloud actually is in practice. Help me with the starting points, where should people be thinking about how Intel is working together with the Cloud providers, certainly with Google, to take companies forward faster. What next?
Josh Hilliker (02:56):
No, great question. What’s interesting is that I talk to people about the Cloud and the first response, partnership customers are like, “Yeah, you guys are Intel, you’re not in the Cloud.” I’m like, “What? Hold on, hold.” I’m like, “Hey, hold on a second.” I’m like, “Wait a sec.” So 83% of the instances are powered by Intel. I’m like, “So let’s, let’s rewind that back again.” I’m like, “So you’re actually operating mostly on Intel architecture in the Cloud.” So we’re very relevant of, okay, why are we here now? I mean, I’ll give you a couple of the other kind of exciting points for me. One is that the ecosystem. You just can’t say enough about the ecosystem. So software, I don’t care if it’s open sourced, it’s ISV, we’re all over it. And there’s some numbers, I’ll just throw out some number magic, right?
Josh Hilliker (03:44):
It’s like 700 industry partners in our commercial ISVs that we’re working with. And this is, usually we’re sitting there as decades of work. And then if we get into open source, we’ve got 650 open source projects we’re actively engaged on. So there’s kind of the Biden numbers you. But what’s cool is when you double click into it, it is absolutely about optimization. And this has been the fun part when I talk to customers and the industry is, people are like, well optimize what? What does that mean? And it’s like, “Oh, let me tell you.” Sometimes it’s socket optimization. Sometimes it’s code optimization. Sometimes it’s a library we get added. And so as you kind of click through and understand more about that, it’s like, “Man, we are very relevant.”
Josh Hilliker (04:28):
And, and so I like to tell you, “Okay, we got this ecosystem. It’s super awesome.” Yes, if you’re doing enterprise apps, SAP HANA, as an example, we’ve been over it for a couple decades working within optimizations. And what’s great is Josh was probably about six months ago, an opportunity to meet with the folks in Germany that did the optimizations and they’re going through the last decade plus of optimizations.
Josh Hilliker (04:53):
And I’m just, feed me. You’re literally feeding me, oh my gosh. If you just… So some people, if you hear custody say, well, I’m just going to do SAP HANA, it’s like, “Do you know what’s underneath all that? Do you know what you’re getting and don’t realize you’re getting? And that on the right platform, it rocks, right?” So let me keep going. Two last points on this is that we are absolutely going after Cloud optimization tools, both investment, and I got some interesting new news to share in a little bit about something that it already hit the wire. So I’m not going to get in trouble for this, but there’s… Yeah, I don’t want to get, I don’t want to be a CSA for hire yet. No kidding. But there’s an announcement there. The other thing that was intriguing is, I’ve been here a long time, as we talked about having Pat Gelsinger come back to Intel.
Miles Ward (05:43):
I’ve talked to a bunch of Intel folks. They’re losing their minds about the level of impact he’s having. I’m super excited about it.
Josh Hilliker (05:50):
This is like a new Unreal Engine just hit for gaming. This is epic. Oh my gosh. What? And so when he came back, he brought back this concept of this Torrid pace. Period. Everything’s to pace. So the meetings we’re in with them are… How can we do more in software? How can we do better? How can we do more in Cloud? We talked to him about what we’re doing in Cloud. And he’s like, “double it.” You’re like, “yeah, yeah.” So Torrid is like, we laugh at… We’ll get Torrid shirts and say, “Torrid” on them, right? And wear it when we meet with him. But it’s like, he literally has this Torrid pace. So that’s the why.
Miles Ward (06:30):
Our customers need that. Right? I think it’s hard. It’s easy to look at the growth rate of Cloud and the financial performance of… You want to do numbers, right? 53% faster revenue growth for companies that use Cloud versus companies that don’t right? So it’s unambiguous. Of course this is the better way to operate. But when you’re there, I think I hear back from customers all the time. They’re like, “I want to get better. I want my systems to be more efficient or high performance.” I want the experience of customers to improve.
Miles Ward (07:00):
I want to be able to better manage data, but they want to know, what’s the most immediate, most positive next step? And I think that they have this instinct that’s going to come from radical software changes on their side, or crazy re architecture of their internal systems. And we keep waving, it’s probably about just sort of getting your app tuned for the processors that it runs on, like pro tip, right? It’s a big ass slice of your cost. It’s a big driver of the sort of overall effect. Maybe you’ll start there pro tip, right?
Josh Hilliker (07:30):
Yeah. Total pro tip. It’s like, you put into Cloud and you let it go is not the answer. Right? You got to do care and feeding. Right? So it’s your absolutely right. It’s funny when it’s like, “well, man, we got this other offer to go to this other architecture.” And I’m thinking about, you’re like, “have you even played with what you have? Have you even looked at what you’re doing? Have you looked at the opposition? What is your Cloud density? What are you really doing?” And they’re like, “I don’t know.” And you’re like, “okay. Let’s just… Let’s be content with what you have for a millisecond. Let me show you what you got. And then let me show you a path.” Right? Because we’re seeing some wicked cool pass. I’ll share in a minute here about going in one to, in two, right? It’s like easy lift. Oh my gosh, performance. Right?
Miles Ward (08:15):
I run a whole team of technical account managers and customer success managers. We’re talking to customers all the time and right in the middle is, you go through and just look at the instances that they’re running, right? It’s a huge slice of their bill. What laptops do your developers work on? They go, “I get them the brand newest stuff, because they’ve got to be very efficient.” I’m like, “you are on computers from six years ago, my man. Let’s let’s upgrade.” Right? I think that stuff just becomes invisible in some ways, when you don’t have your sort of fingers and toes on the individual hardware pieces.
Miles Ward (08:48):
And so it takes a little reminder, it takes a little prompt. And I think very often there are places in in Cloud where so much of the management now moves to the Cloud provider, right? Sort of glorious. If I’m running on banner, I probably don’t need a storage DBA because that sort of Google’s job to do. But you still absolutely need people that are thinking through the infrastructure building blocks that you do have access to. Right? And the instances and compute is a core part of that.
Josh Hilliker (09:15):
We have this cool, cool role inside IT that I got more vetted on here the last two years, that’s called Cloud broker. And it’s someone basically mining your shop, right? They’re there making sure, what are you doing? How are you doing it? Did you know? And by the way, I’m going to protect you before you even hurt yourself. And this team is… They’re so cool. I went out and hired three of them out of the team to work for me. Because “I’m like, okay, I love you guys smacking me. I need you to work for me now. I need you to be on the team and tell customers, how do you manage the shop? How do you keep your eye on it?” So you’re totally right. It’s like, “do you want to use seven, eight year old computer hardware in your virtual data center called the Cloud? Or do you want to pay attention?” No, no, no ding, but let me open up the book and show you what page to go to let’s do this.
Miles Ward (10:11):
So this is the reason, like I sit on a board of a group called the FinOps Foundation. It’s part of the Linux Software Foundation. And so what we see a lot of time… Right? Which is great. Right? And we now staff a team of basically accountants because it’s one thing for me to go to an engineer and go, “Hey, you would save a buck or two, if you sort of poke the right buttons and set this up in a way that is efficient.” And they go, “right, but is it one or two? Because I have a bunch of other opportunities where I’m going to save or where I’m going to earn new revenues or the rest I got to prioritize.” So unless I have the ability to explicitly describe the positive effect they’re about to get, they won’t do that work.
Miles Ward (10:47):
So I cut loose the accountants and actually compute it for them and say, “it will be 21.6, 2% savings on your bill. That means this many hours of your staff is actually a productive use of time.” That… It feels like spoon feeding sometimes. But if that’s what gets people to a place where they’re operating in the best practices way in a way that is most efficient for them, I think it’s time well spent. And if that’s the contribution that Slack can make, I think we’re happy to do it. It has definitely affected customers. I know there’s been a bunch of joint projects that we’ve done together as a part of Intel collaboration program for optimization. Can you talk a little about like VTune and some of the stuff we’re doing on compute to make it so the folks go faster?
Josh Hilliker (11:31):
Ah, oh, thank you. Thank you. And real quick, it’s like, instead of unleashed the Kraken and it’s unleash the film.
Miles Ward (11:37):
Yeah.
Josh Hilliker (11:38):
Oh yeah. Right. It’s like unleash [crosstalk 00:11:39] And then they…
Miles Ward (11:43):
My man Robin dives in there with his jujitsu spreadsheets and just causes grave, grave havoc. It’s beautiful.
Josh Hilliker (11:50):
Absolutely. That’s so… Aw, that’s so great. All right. So what… Yeah. So how… The partnership with SADA, right? It’s so great. Right? And I’m not just saying that, but it is. I’ve been working with SADA throughout each of the different milestones and different projects we’ve been working on and my team has. Let me talk a little bit about kind of the partnership and then talk a little about the cool tools because you immediately brought VTune and immediately I want to go down a Vtune… You got me. I want to talk about VTune for a long time. So give me a moment. I’m going to come back to Vtune.
Miles Ward (12:17):
I know I was teasing. I was like “Ooh, delicious.”
Josh Hilliker (12:19):
I’m going to share with you. But in the beginning days with SADA, it was our kind of awareness that SADA’s one of the largest, if not the largest partner with Google for Google compute engine services. So it just makes sense. It’s like, “well kind of a duh.” Okay, yeah, we should be working with SADA. And that was the beauty early on is we started to provide kind of how are we doing Cloud benchmarking? And again, another area where customers are kind of…
Josh Hilliker (12:51):
They’re not listening to this stuff yet, or they haven’t really thought of this yet is I can benchmark in the Cloud. Well, I’ve been benchmarking on prem for years. I used Spec In of the past, I use Spec JBB I use… But I do it in the Cloud and it’s like, “well, yeah. Absolutely.” So that was one of our early engagements. Let’s show you how we do benchmarking in the Cloud. Let’s get some synergy there for different workloads. And that worked out really well. Because then you could, again, start to have the conversation with the customer on unleash, the FinOps and say, “look, the benchmark says you are going to get a better price or be performance for price. Wouldn’t you want to do that?”
Miles Ward (13:26):
It’s just moving from… I can put up case study after case study, after case study, right? We have this sort of like checklist. I have customers that have saved 20 or 30 or 40 or 50 or 60, and I have a customer that’s done 70. I need an 80 still, I’m working to get the like full rainbow of percentage discounts in the case studies that we’ve got. But all of those sound like great news, if you’re the CFO trying to figure out how to shave cost out of the thing, but they want to go, “right, but I know how much it costs me to get at that advantage, which isn’t 20 or is it 60, right? Help me understand what it actually works out to,” so that the specificity unlocks the action.
Miles Ward (14:02):
And I’m in violent agreement about how often I saw IT departments take responsibility over benchmarking and sort of proving out the purchase decision. I think that stuff made sense when you were buying for three years or five years or something. And when you have this… There’s this freedom that comes from like, “oh, I can turn them instance off in the next second. So if I make a bad decision, it’s okay, I’ll do a better one.” Which means you literally never be evaluate ever again. You just sort of forget about it. And then that’s why you never get this moment where it’s like, “oh, that’s probably a good idea for us to make a thoughtful choice here.” So we use PerfKit Benchmarker a lot as the way to fan out across a bunch of workloads and get a real coherent view of which instances are faster for the diverse workloads that most of our customers run.
Josh Hilliker (14:45):
Exactly. I mean, PKB is money in my eyes. Let me talk a little bit about tooling before I talk about tech. And by the way, we’ve been working on number of customers with you guys. I know we’ve got some case studies we could sprinkle throughout of savings and what have you. And those numbers we’ll talk about in a moment, but let’s talk about some of the tools for a moment. One of the… Ah, boy. So if you look at the optimization discussion, there’s pieces around application characterization. There’s pieces around what’s happening at the PID level, right? Let’s get, let’s get deep for a minute. So if I want to look at the PID level and see what’s happening, I want to characterize the application to see where I’m finding latencies or where I’m finding bottlenecks, we have a great range of tools.
Josh Hilliker (15:31):
So if you start at the base and this is part of my exciting news… Woo! Is that in Google, the question becomes is “can I see specific information to help me on that application characterization?” And the answer is yes. So we had an announcement here in the last little bit here that we are acquiring a company called Granulate and [crosstalk 00:15:56]. Yes, there’s a total press out there, but we’ve been working on it for the last couple years. Right? And they’ve got a tool called GPRO profiler and I’m not here to sell tools, but I’ll tell you what, when I see a telemetry tool, show me evidence of what’s happening in all of the… I ask instances on Google, that gets me super excited. And that’s exactly what G profile does. It uses ptrace to look at the PID level and it gives you a flame graph of where you see latencies and problems.
Josh Hilliker (16:27):
So you know what exactly to optimize, what to change. And then you could also say, “ah, you’re not using AVX 512. Ah, you didn’t… You’re on the wrong instance. Ah, so you have these, these gotchas. Now, when we go from that to… I’d say kind of the G profiler helps me do the application characterization. Now I want to go get Enth Degree, I go to Vtune. And VTune, how it works on Vtune, this is kind of a fun learning I’ll call it, is that we take that workload, we replicate it onto an on-prem box. We run VTune to get the deep PMU’s, which are performance monitoring units, which are the counters. And then from there we spit out the same kind of flame graph and it gives us exact location of where the latency and the problem is. So are you back inbound?
Josh Hilliker (17:14):
Are you front inbound? What line of code… Are you calling a library ahead of when you… Are you calling it too late? So it’s doing too much store and that’s causing latency? Where in the code? And that’s… When we get to that point, it, to me, is the most exciting because you can really pinpoint down and then as they take that workload back onto the Cloud and then scale it, they’re seeing that win everywhere. So that’s kind of the… Now we’ve got more tools and I want to talk a little bit about Intel technologies for a moment, but let me finish out tools and I’ll come back to tech. We’ve realized about… Again, this is all in the last two years of great, I’d say, introspective at Intel of going, wow. And to be blunt, we’re not as relevant in the Cloud as we need to be.
Josh Hilliker (17:55):
How do we fix that? And so we’ve been public about like, “Hey, Intel’s back baby. We’re in there. 83% come on now.” And so we’ve gone out and grabbed Cloud optimizer tool partnerships with folks like Densify and we’ve kind of given them the here’s what optimization means. Here’s what configuration means. Here’s what you could do when you’re doing a validation of recommendation engine and help thing. So I looked at that saying “huge move forward on what are we doing? How do you do that to look at cost and save cost?” So just super excited about that.
Josh Hilliker (18:33):
Now if you continue on, yes, we’ve got more, we’ve got a whole one API that’s Scott, if you double click in OneApi, it has all of these… A foundation and then a couple of buckets of code. So you’ve got in there as VTune, actually V VPP, VTune profiler platform profiler is in there and you got of course more libraries, and what have you. Just super excited. Now I give a big pat on the back to Pat, no pun intended, but his drive on OneApi, his drive on making sure we’re doing more there in software is what I see as really kind of the why it’s that Torrid pace of, “Okay, do more.”
Miles Ward (19:07):
Yeah. So I think that there’s a lot where, you have this inbuilt [invant 00:19:15] at advantage where if you are already such a huge fraction of the compute that gets consumed, applications that are being designed in Cloud are inherently being tested in Intel environments, right? Like it’s just, that’s, what’s available there. That’s what people are using.a And I think the move to more containerized applications is setting people up to start to frankly, to attend to performance more. The companies that we see that still run just bare virtual machines. Right? It seems like I remember proposing virtual machines as a crazy idea to companies. And then now it’s like, “that’s the old guard legacy thing. Why would you ever do such a thing,” right? Which is just nuts, right? That transition I think you are also earning as a result of the containerization platforms, Kubernetes, [Anthos 00:20:04], those kinds of building blocks, a whole other layer of performance telemetry and basically behaviors from the application that spawn from that kind of performance result.
Miles Ward (20:14):
So you can make changes in the configuration of software, which build version you’re using the other kinds of building blocks that tie into the appropriate instructions that are available on the Intel side and see the performance. Now see it in a sub… Do the testing of that to not have that one machine that has the one IP that if you change it, you’re going to take downtime. And so you don’t have this operational risk that goes along with earning the performance benefit, where I think in kind of VM land, that felt more prevalent or at least harder to work around. What other kinds of workloads make sense for this? What other places that folks should hear the alarm bell? I should probably pay attention if I’m running something like this.
Josh Hilliker (20:54):
Good, good question. I can’t wait to come back to your virtual machine comment because I live through that translation too, of…
Miles Ward (21:01):
That’s impossible. You can’t run production. What are you talking about? Yeah, yeah.
Josh Hilliker (21:07):
Yeah. And now it’s like, “nah, sorry, let’s talk about workload.” So what type of workloads kind of what’s in Google, right? And what kind of workloads and what should be like kind of those application triggers you should be going, “oh.” So first off let’s just talk about standard. Right? We got Ice Lake. Okay, so yeah. I’m getting a little bit… Sorry about that. I’m getting a little platform-ish. Ice Lake’s our latest third generation Intel Xeon scalable processors. I know, but it’s Ice Lake. Okay. So Ice Lake is in two now. Right? So it’s… Okay. What are you doing? Web apps, medium, large databases. Right? And that’s all that in two general purpose. But again, we finally have… Well not finally, that’s probably about… We have Ice Lake in there. So it’s like, okay. So if you are on [In one 00:21:57], you should be thinking [In two 00:21:58] and hopefully you’re paying attention that. You know you have [In one 00:22:02] to then flag yourself.
Miles Ward (22:04):
Yeah. That’s just a no brainer update of performance and efficiency. So many people go like, “I don’t know. I put that in the calculator and it looks like it costs more.” I was like, “right, but do you run more than one VM?” They go, “yeah, yes.” And go, “well, great. You’re going to run fewer. I promise.” Right. That’s how the Cloud works. You get to use less gear for more throughput if you’re on faster gear. And they’re like, “oh, well that is better. Okay. I like it, let’s do it.” It’s hard. Because you got so many issues that companies have to work through. Right?
Miles Ward (22:32):
They’re in the middle of hiring and figuring out a dev pipeline and which features should I put in my product again? And oh God, how do we market this thing? Right? So, in the grand stack rank of which opportunities are going to pay the biggest dividends for their attention, I think it’s easy to go, well, if I’m in Cloud, I’m already ahead of the curve. So forget it. If it’s Cloud, it’s good. Which, hey, I agree too, but, but there is a second step. I think we’re always trying to set up with customers. Yes, there is a first step where you can get to you sort of sane operational profile or a rational security standard or you’ve got a development pipeline that isn’t totally nuts, but there’s always that sort of second step of refinement and optimization and really getting what you came for. Right? Actually getting out the benefits of these platforms.
Josh Hilliker (23:20):
Just care and feed. Just please care and feed. Right? All right. So we can tee down apps. What I’d say is that you’ve got your compute optimized of course. Right? So this is where you get into gaming. Right. So if you’re do kind of gaming, you should be looking at what is going on C2. M is when we get real fun, right? So the M class, because this is exciting to me of in memory database. Whoa. Right? So if I’m doing value key pair, right? I’m doing SAP HANA. Right? I’m doing analytics. It’s like [crosstalk 00:23:52] here we go. Right?
Miles Ward (23:52):
Cassandra Yugabyte, Cockroach, all that stuff.
Josh Hilliker (23:56):
Yeah. Speed, response, latency. I mean everything is… If it’s in memory, guess what, it’s going to be a lot faster. So there’s that. Then we get into of course VMR engine. And I love when I talk to customers around VMware, they’re like, “yeah, we have VMware, we’re thinking in the Cloud.” I’m like, “dude, let’s go. Let’s set it up and let’s start migrating. Let’s make this happen. Okay?” And then of course Bare Metal. And I, being a data center guy being a hardware guy, I love Bare Metal of course. Right? But it’s what’s right for the app you’re using, what’s right for the workload you’re using, what do you need it to do? Are you going to layer stuff like SAP HANA on it? Okay. Cool. Do you want to use the service? What do you want? So it’s like, I look at… So you got the databases, the key enterprise workloads, the [Hamady 00:24:51] Bare Metal or I’m just going to literally go to VMR engine, right? I going to use that and…
Miles Ward (24:56):
So there’s a bunch of… You have the upside opportunities that a more modern infrastructure presents and you have the terror coming from the other direction of, I still have this boat anchor of a crunchy data center that I don’t like, and none of the ISVs have anywhere near the RND budget to make this stuff any better there. They’re… All that dollars is now being spent by the big Cloud providers. Right? It is pretty… I love… I’ve built a bunch of startups. I love venture capital back companies. I love the fire of that work. It is hard to compete against a hundred billion dollars a year in RND across the three Cloud providers. Right? That’s more than your venture round my friends. It’s big bucks. So that kind of… It creates this gravitational hole of new capabilities and features, which is one of the big reasons I think Cloud is so exciting.
Miles Ward (25:51):
And with Intel at the center of that, there’s a lot of kind of future features that are coming. Right? We talked a lot about compute, but there’s also all the stuff that’s going to happen in networking and storage from you guys that I think is really craft breaking gear too. And many, many folks… Those both get to kind of the IT areas that folks feel like they’re going to get past when they go to Cloud. Yay, I don’t have to fight with networking. That’s Google’s problem anymore. I’m like, “well you probably have like a little bit of optimization there too. My friend.”
Josh Hilliker (26:20):
There’s always room for optimization. That’s the role that I am fully in. Because I’ve met with, as doing my on-prem work with customers, we go in there and we look at their compute density on-prem. And then as I shifted and really kind of took that knowledge and went into Cloud, it’s like, “okay, we got to look at Cloud density.” There’s always a room. There’s always a play for, what else could I do? And what’s easy. Now some of the… I got to share this with you just a little bit more around open source because I think it’s great is that we’ve been partnering with Google that as we find things like modifications, like TensorFlow, we’ve been working to upstream it. So by default, when you get it it’s already optimized. And that’s so cool.
Josh Hilliker (27:05):
And by the way, that’s I think that should be a industry expectation is you are doing the RND you talked about, you are doing the development and I’m getting that natural benefit. But I think it’s important what I tell customers is, “look, you need to think about that, you’re getting that and knowing what your hardware is underneath, not…” Yeah, I work in Intel and yeah, we’re a hardware Silicon provider and we do software, but I think it’s important to go, “Hey, these optimizations that are native now that have been upstreamed work better on IA and they work on better on these instances.” That’s just… So I think that’s the kind of, the beauty of working with Google is that.
Miles Ward (27:46):
It is exactly the kind of work that an individual customer will always do worse than your ability to do it in aggregate across all the customers. And so when they have evidence that, “wow, this part of the provider ecosystem is focused on our actual needs is working upstream from us. So I don’t have to pay attention and learn the extra bid and poke the extra buttons is just better.” That’s, I mean, the easy button is a valuable button and I think there’s a lot where the… Being able to select Intel as your default outcome, because you have confidence in the level of upstream work that they’re doing to make all this stuff efficient is a giant, giant sort of reduction in complexity. We definitely want everybody to test these things and race them and hold everybody accountable and push the performance expectations forward. But to make things go faster is an investment, right? And it’s really powerful to watch just how much of an investment Intel is making and lovely to hear that it’s re redoubled under Pat Gelsinger. That’s an incredible improvement.
Josh Hilliker (28:53):
Yeah. Well, what I think is great, and I’ve seen this with our partnership right with SADA is that the Cloud optimization program has been great. I, well… Let me use a different word. It’s been money, I guess money for the customer, right? Is that as we partner on things we’ve been seeing, okay, “Hey, 30% better on Ice Lake with their workload” and the customers go, boom! Because you’re right. If I could do more, then I need less. And therefore I can actually expand my business faster. Which is all about time to market new services, what I do. So that’s why, as we’ve been collaborating on that program and working with customers and kind of doing that trifecta with them of you saw a powerhouse with Google, Intel powerhouse and kind of triangling with the customer, it’s been phenomenal. The CSAs, including myself, the CSAs on my team myself have been just giddy as we’re seeing customers are getting excited about it and starting to ask the right questions like, “well, what else should I be thinking about?”
Josh Hilliker (29:57):
Yeah, welcome. So one thing you said, though, now I’m going to rewind and then fast forward, kind of like this, is that future tech, what’s coming. I know you dropped that carrot out there, so I’m going to take it. And this really ties into my passion and why I went after PE on telemetry is that infrastructure to me matters. Right? And getting full use of out of your infrastructure is important to me. And that means, what am I watching for telemetry? What are the decisions I’m making? What is the automation? So we look at the modern autonomous Cloud where it operates by itself and it knows what to do, what if we could make that better? And that’s where the infrastructure processing unit IPU comes in. Yeah! And you probably heard about this, but I’ll just give you the kind of the quick version is that if I can offload all of kind of the core function of the server onto another IPU and then have all the CPUs available for workload, wouldn’t that be great?
Miles Ward (31:06):
Yeah. Especially for the kind of performance optimization that you’re talking about, because you don’t have this sort of internal overhead associated with the virtualization layer or the container layer or these other building blocks. I mean, Google’s stack for this stuff is very unique. Having worked at Amazon and having installed a bunch of shit in the Microsoft data centers, Google stuff is a little space alien and awesome. And there are huge benefits, shit that does not make sense. So you can do some benchmarks right now on Cloud Run, which is container hosted infrastructure managed on the Google side, and the containerized one runs faster than the virtualized one and you’re going like, “isn’t there another layer of abstraction in there? I don’t get it.
Miles Ward (31:46):
How is that possible?” Right? You’re like, “well it’s stuff like the IPU. It’s manipulations in the back end to further accelerate the processing of these workloads where Google gets to bring all of the tools that Intel makes available and the rest of their infrastructure system makes available to accelerate workloads.” That’s if we can make it so that the most efficient environment for developers is the highest performance environment for the workload. That’s, that’s just a complete game change in terms of the productivity of our customers. That’s just huge.
Josh Hilliker (32:19):
If I look at where we’re going with the IPU, right? With Google, right? And how we’re going to offload, make it better performance. Right? I also take a step into what they talked about. Well, Google engineers talked about at our Intel On event back, what, gosh, last year? Where Samantha Alt, one of my partners in the business unit and Google talked about their work they’re doing in telemetry at that extremely deep level of MSRs and the PMU’s and how they’re using it to help even the automation of it. So to kind of shore up on your comment on that kind of sick, awesome alien stuff that’s happening, right? You kind of unpack that and go, “oh, I see what you guys are doing.” So the level of sophistication of how they’re doing management of how they’re doing provision, how they are working on getting the most amount out of it, and then you now add IPU to it.
Josh Hilliker (33:10):
You’re like, “this is a phenomenal environment that’s going to be perform very well for us.” So now one last thing, I’ll say this, but I can’t… Or I’m going to plant the seed and I’m going to walk away from this, is that our next generation XEon is coming. It’s called Sapphire rapids. I’ll let Google announce when it’s coming because I can’t do that. I was actually told last night, do not talk about that. Don’t talk about when. And so… But SPR right? Sapphire rapids is another kind of a blah! Okay. So I’ve been through all the transitions, right? We go Haswell to Broadwell, to Sky Lake, to Cascade Lake, to Ice Lake. And it’s like, you start looking and you’re like… You start to use monumental shifts in performance, monumental shifts in technology and SPR is not going to disappoint. So I’ll, again, I’ll let Google announce when they’re going to get it. You’ll probably hear before I will, because you guys are such a big partner with Google.
Miles Ward (34:11):
There’s lot in that too. That’s not just about kind of the workloads we’ve been talking about. Right? Data processing databases, large application hosting ,the rest. I think there’s a big push from the Intel folks, from what we’re watching to grab the ML workloads and grab because we’re watching… It’s one of the… If you want to talk about people that don’t do the care and feeding, right? That aren’t doing the evaluation. They kind of hear, this is how you’re supposed to set it up. And this is where the instructions have been written. So I’m going to sort of run it that way. The diversity of actual workloads in ML pipelines is huge, right? Some stuff is just terrible on some devices. And some… the difference is enormous. So we’re… Right now benchmarking for those kinds of workloads I think is increasingly difficult.
Miles Ward (34:58):
It’s even more nuanced. There’s more places where, because companies are running really different kinds of patterns. This group are doing a bunch of linear or regression. These folks are doing large model inference. These folks are doing super, super tiny model predictions and categorizations like large memory footprint requirement, large memory throughput requirement. Right? All those sort of different balancing acts mean that’s one of the spots where you really need to test. And I think there are magic bits and bites in the middle of Sapphire Rapids that I think will have an effect there.
Josh Hilliker (35:30):
Yeah. I mean, are you doing genomics or not? Right? Are you doing NLP or not? What are you doing it for your chat bot? How are… What is it for? How much data? How are you running it? And then what library and feature you actually going to apply? So, yeah. No, it’s interesting that one of the CSAs in my domain created this. If you do this, you will see this. And of course we got to be careful on how we articulate it, because you can’t do a [inaudible 00:35:58] of math when you, when you try to [crosstalk 00:36:00].
Miles Ward (35:58):
Your mileage may vary a little.
Josh Hilliker (36:01):
But you see kind of this depth of, but it’s… You look at it and go, “okay, if I just do this simple mod of this specific library, I get this out of it.” And you’re right, it’s a very boutique nature of current ML, right? Of, what am I using? What model? How is it working? And there’s still a way to like be aware of what’s out there. Right? So you can see things where you flip the library, you get the extension that you don’t have. Right? AVX. Right? You look at the VN and I, right? It’s like each one of those has a… you got to look at what’s the kernel. And so it’s interesting, when I get into those AI conversations with the AI team and a customer, we start rapping on, “okay, what version are you using? What model? What version? Where’d you get it? What’s this?” It’s like, “okay, we’ll do this, this, this, this, and now look at your performance.”
Miles Ward (36:46):
Well, look, Jeff, I super appreciate you creating and helping us get access to more ingredients for the meals on that menu. We are really motivated about making a positive impact and the performance of the customers that we serve. And this is a key way of us doing that. Thank you for taking time, walking through some of the building blocks. Are there like specific spots folks can go to learn more, maybe some websites they should check out?
Josh Hilliker (37:12):
Yeah, absolutely. So couple things, intel.com/Cloud, absolute money. I mean, I know it sounds like nah, but it’s money. That’s one. There’s also forward slash Cloud performance. And this is where it’s like [laughs 00:37:29] because it’ll give you buy workload, buy Google and you can look at performance with the recipe and then they’ll give you… Let me give you the last one. Last one’s going to be a little bit hard cause I’m going to have to send it to you because it’s got [inaudible 00:37:43] on this. The Xeon tuning guide. Okay. Now, okay… Two things. When I first saw that Intel did this and it was about nine months ago, I was like, about time. Thank you. So we did two things. We gave an open source database tuning guide with all flags and commands. We gave a open source Java tuning guide.
Josh Hilliker (38:08):
Okay. Now it’s a long URL so I’m going to have to probably send to you. But this is… As someone that does infrastructure as code, someone that’s deeply studying for my Terraform Hashi Corp test right now, and a firm believer of Terraform. Right? Knowing that configuration where I can put into my IAC code is so important. Right? And so we’re giving you the recipe. NoDB, it’s 20 flags that you can do to immediately speed up your database. Whether you’re using Postgres on top using my SQL on top of it’s that NoDB base. Right? And it’s like, “well, duh,”. But it’s like we’re giving it away. Here you go. We’re not holding back anything on what’s going to make it faster to run. So those are kind of the three places Cloud, Cloud performance.
Josh Hilliker (38:55):
And then now the last piece is the DevOps toolkit. So if you are a dev officer, right? And you care about that, we do have a DevOps toolkit. That’s out there where we give certain things some under NDA, so, yeah, I know, but a lot of it’s not. So it’s not out, it’s on GitHub. We have that posted. It’s going to end that Cloud area. You go to the Cloud and you go down to DevOps and we have all the links to Granulate. And one… I got to give one last plug. I know we’re getting close on time, Miles, but one last plug. I talked about G profiler with Granulate, which is that application characterization. The beautiful thing is G agent, and G agent speeds up between the application and the kernel level. And it works on Google.
Josh Hilliker (39:44):
So it looks for instructions like Java workloads, no GS. And it says, it starts to kind of look at the pattern and it accelerates the pattern. And I’ll tell you what, it’s been money. 20, 30, 40. I haven’t seen the 80% yet. Okay? When I find an 80%, I’m calling you, but you keep seeing these accelerations of workload and you’re like, “this is unbelievable.” So I had the guys in the lab tested a couple years ago so I could believe it. And I saw it and I was like, “all right, I’m a firm believer. I get it. You guys crack the code on this.” And therefore we had the announcement that just came out. So if those are for if you want to look for, “okay, Hey, it’s time to do some care and feeding.” Pick up a phone, get Granulate as part of our, our SADA, Intel collaboration. Right? And let’s talk about it and show it.
Miles Ward (40:33):
That’s a big part of the TAM and CSM function. They are happy to walk you through these tools, help get everybody access and make it so that your workload screen, that’s what they’re here to do.
Josh Hilliker (40:43):
And if you don’t, just read a couple of the things, see how it works. Ah, dude! G profiler is on GitHub. So if you’re like, “nah, this is, this is hokey pokey,” download G profiler, run it. It’ll tell you that this is a good candidate and here’s how much you’re going to save. And you just go, “take my money.”
Miles Ward (41:01):
Josh. I really appreciate it. Thank you for taking all the time.
Josh Hilliker (41:03):
You bet. Thanks Miles, thanks for the opportunity.
Miles Ward (41:07):
And everybody in the audience. Thank you so much. Cheers. Talk soon.
Speaker 3 (41:11):
Thank you for listening to Cloud N Clear. Check the show notes for links to this week’s topics. And don’t forget to connect with us on Twitter at Cloud and clear and our website, SADA.com. Be sure to rate and review the show on your favorite podcast app.
