Another great read, thanks for keeping this up. I look forward to every new issue and hope you never stop.
The section on Cloudlflare, Prince begins by comparing their severless solution to a bare metal type lift and shift. These two things are not comparable. AWS has a severless option called Lambda that is comparable to Workers.
Then he goes on to claim hyperscale utilization in the mid teens. This is a silly claim and again I have believe he knows better. I can't think of an industry that operates the way he describes. A company may guarantee you capacity but if you're not using it, they'll find someone who will. Airlines, electrical generation/transmission, oil pipelines they all work this way. Here's a quote from the 2019 Amazon shareholder letter discussing AWS and private DC utilization rates :
"AWS is also inherently more efficient than the traditional in-house data center. That’s primarily due to two things—higher utilization, and the fact that our servers and facilities are more efficient than what most companies can achieve running their own data centers. Typical single-company data centers operate at roughly 18% server utilization. They need that excess capacity to handle large usage spikes. AWS benefits from multitenant usage patterns and operates at far higher server utilization rates. "
Thank you. I'm aware of the model, but I've never seen the actual hard numbers, so it seemed like Prince would be well positioned to have an informed guess.
I asked him on twitter to elaborate and he replied he was talking about CPU utilization. My guess would be that because usage patterns are spiky, they can't get utilization too high on average because then they'd run out of capacity during spikes, which probably line up pretty well by geography (and you also can't shift workloads on the other side of the world if you're customers are paying to be located in a certain 'region'.
And I think his whole point was to compare the serverless model to EC2, which is still a huge huge fraction of everything that AWS sells. His point is that it is different and has different trade offs.
But not having access to the inside numbers at the hyperscalers, I can't say for sure what their average CPU utilization is.
It's certainly something I'd love to find more details about, but I'm not sure they release these kinds of metrics.
There may be two difference utilisation metrics being used interchangeably here. Using EC2 as an example, I agree it's safe to assume AWS achieves a high (more than mid-teens) utilisation rate of hardware capacity that is currently assigned to and paid for by a customer at any time. What Prince may be referring to is that the customers themselves are likely not efficiently using that capacity they are paying for.
This doesn't apply to Lambda where you only pay for the actual resources you consume (which is the same as Workers).
My understanding of Cloudflare's advantage here is that they already had a ton of hardware deployed for their network which was under utilised. Running Workers on this existing hardware has a minimal additional overhead for them, until Workers becomes popular enough to require more hardware.
Yes, there's some of that. Prince was generous enough to give a lot of precisions on what he meant here over Twitter, I still need to post about it in the nl at some point, but you can follow the thread here (he replies across a couple forks in the thread, but I think if you rewind from this one you'll get most of it):
We can say for certain it's "much higher" than 20%.
"AWS is far less capital intensive than the mode it’s replacing – do-it-yourself datacenters – which have low utilization rates, almost always below 20%. Pooling of workloads across customers gives AWS much higher utilization rates"
Interesting! I would love if there was more published from Cloudflare on how they measure "load". Many ways to do it. Plus it's more than CPU for instance the GPU stress, load on network bandwidth, storage utilization etc.
I'm actually looking to buy shoes that are more minimalist than the ones I have now, and at least with zero drop. I certainly want to go with evolution rather than against it, so I'm definitely looking to strengthen my feet and run on the front part of my feet rather than my heel.
One of the benefits I've had working from home for 15+ years is I almost never wear shoes, so my feet are used to walking this way. Now I just gotta build up the running part.
Another great read, thanks for keeping this up. I look forward to every new issue and hope you never stop.
The section on Cloudlflare, Prince begins by comparing their severless solution to a bare metal type lift and shift. These two things are not comparable. AWS has a severless option called Lambda that is comparable to Workers.
Then he goes on to claim hyperscale utilization in the mid teens. This is a silly claim and again I have believe he knows better. I can't think of an industry that operates the way he describes. A company may guarantee you capacity but if you're not using it, they'll find someone who will. Airlines, electrical generation/transmission, oil pipelines they all work this way. Here's a quote from the 2019 Amazon shareholder letter discussing AWS and private DC utilization rates :
"AWS is also inherently more efficient than the traditional in-house data center. That’s primarily due to two things—higher utilization, and the fact that our servers and facilities are more efficient than what most companies can achieve running their own data centers. Typical single-company data centers operate at roughly 18% server utilization. They need that excess capacity to handle large usage spikes. AWS benefits from multitenant usage patterns and operates at far higher server utilization rates. "
Thank you. I'm aware of the model, but I've never seen the actual hard numbers, so it seemed like Prince would be well positioned to have an informed guess.
I asked him on twitter to elaborate and he replied he was talking about CPU utilization. My guess would be that because usage patterns are spiky, they can't get utilization too high on average because then they'd run out of capacity during spikes, which probably line up pretty well by geography (and you also can't shift workloads on the other side of the world if you're customers are paying to be located in a certain 'region'.
And I think his whole point was to compare the serverless model to EC2, which is still a huge huge fraction of everything that AWS sells. His point is that it is different and has different trade offs.
But not having access to the inside numbers at the hyperscalers, I can't say for sure what their average CPU utilization is.
It's certainly something I'd love to find more details about, but I'm not sure they release these kinds of metrics.
There may be two difference utilisation metrics being used interchangeably here. Using EC2 as an example, I agree it's safe to assume AWS achieves a high (more than mid-teens) utilisation rate of hardware capacity that is currently assigned to and paid for by a customer at any time. What Prince may be referring to is that the customers themselves are likely not efficiently using that capacity they are paying for.
This doesn't apply to Lambda where you only pay for the actual resources you consume (which is the same as Workers).
My understanding of Cloudflare's advantage here is that they already had a ton of hardware deployed for their network which was under utilised. Running Workers on this existing hardware has a minimal additional overhead for them, until Workers becomes popular enough to require more hardware.
Yes, there's some of that. Prince was generous enough to give a lot of precisions on what he meant here over Twitter, I still need to post about it in the nl at some point, but you can follow the thread here (he replies across a couple forks in the thread, but I think if you rewind from this one you'll get most of it):
https://twitter.com/eastdakota/status/1532775221471563776
We can say for certain it's "much higher" than 20%.
"AWS is far less capital intensive than the mode it’s replacing – do-it-yourself datacenters – which have low utilization rates, almost always below 20%. Pooling of workloads across customers gives AWS much higher utilization rates"
That would be my intuition too, but I wish I could see those numbers.
I think we got our answer concerning utilization: severless w isolates>severless w containers>hyperscalers mixing & matching VMs>on prem infra
I think we can still confidently say AWS does not have a utilization rate in the teens.
Agreed. Maybe one day we will get to
Its like Noah’s Ark up in here now, this steamboat is packed! ⛴⛴⛴
Gotta build an armada!
Congrats on fetching a response from Matthew Prince! Happy for you, maybe he becomes a premium subscriber now ;)
Ha! That'd be neat. I do like how he's accessible on Twitter, it's not the first time he responds to my questions. 💚 🥃
Interesting! I would love if there was more published from Cloudflare on how they measure "load". Many ways to do it. Plus it's more than CPU for instance the GPU stress, load on network bandwidth, storage utilization etc.
Thank you for bringing out this topic!
Prince posted a bit more detail in this thread:
https://twitter.com/LibertyRPF/status/1532764603305840642
https://ucanr.edu/datastoreFiles/608-809.pdf
Global decoupling of agricultural land and food production
It would also be wise to plot kilocalories produced because I read somewhere the nutrition in the food has reduced
An orange 75 yrs ago had lot more nutrients than orange today (not sure if it’s true )
I suspect you wouldn't see that in kcals, and if anything, this may be going up because we tend to eat more energy-dense food than decades ago.
But if there was a way to track micronutrients, that surely would be interesting to see! 🤔
Hey
Can you share the source link got the cloud flare bit on this newsletter
Why is Cloudflare’s infrastructure so efficient at what it does vs the hyperscalers?
Investor day from May 12. Koyfin has the transcript here:
https://app.koyfin.com/news/ts/eq-crgkkc/all/2574584?sourceType=transcript
Cheers 💚 🥃
Is there a link to part 2 of the interview with Constellation's CFO? Cheers!
I assume you mean part 1?
Part 1 is here:
https://www.libertyrpf.com/p/286-stripe-financials-constellation#:~:text=Tegus
And the original interview is here:
https://www.csisoftware.com/docs/default-source/investor-relations/shareholder-q-a/april-6-2022---tegus-interview-with-cfo.pdf?sfvrsn=3b75814d_3/%20April-6-2022---Tegus-interview-with-CFO%20.pdf
Cheers! 💚 🥃
Ah, missed it, sorry and thanks!
No worries! Thanks for reading!
Thanks for the advice!
I'm actually looking to buy shoes that are more minimalist than the ones I have now, and at least with zero drop. I certainly want to go with evolution rather than against it, so I'm definitely looking to strengthen my feet and run on the front part of my feet rather than my heel.
One of the benefits I've had working from home for 15+ years is I almost never wear shoes, so my feet are used to walking this way. Now I just gotta build up the running part.