519: Thoughts on Nvidia Q2, Crowdstrike Q2 (Post SNAFU), Intel vs AMD + TSMC, Doom AI Engine, Gavin Baker Interview, and Taiwan

"you’re outsourcing decisions to the crowd"

Nothing happens unless first we dream.

—Carl Sandburg

🎮🤖🔮 If you watch this video 👆without context, you may not think it’s a big deal…

But what you’re seeing is GameNGen, a neural model trained on Doom that can generate 20 frames per second of real-time gameplay on a single TPU.

GameNGen is trained in two phases: (1) an RL-agent learns to play the game and the training sessions are recorded, and (2) a diffusion model is trained to produce the next frame, conditioned on the sequence of past frames and actions. Conditioning augmentations enable stable auto-regressive generation over long trajectories.

There’s no game engine! Just an AI hallucinating/remembering the game! 🤯

The model responds to the player’s keyboard inputs by generating the next frames based on what it generally knows about the game and this Doom level specifically as stored in its weights (it was trained on those maps).

In theory, there’s no reason why this can’t work for other games (though the more complex the game is, the harder the training to get good coherent results, I imagine). Doom levels are fairly bounded and more predictable than some very dynamic, open-world games with a lot of entities with complex behaviors, etc.

It’s funny to think that Doom fit on four floppies (💾 💾 💾 💾) and ran on my dad’s 386 DX/25mhz back in the day (with 4 megs of RAM!). This version, which isn’t too different if you just look at the output pixels, likely requires millions of times more compute, storage, and RAM to achieve the same result.

(I did some napkin math, and a TPU v4 is approximately 275 million times more powerful in terms of FLOPS than a 386 DX 25MHz, but that’s probably underselling it)

🏴‍☠️🔒🔑💵💰📈📉 I wonder what the OPSEC is at big companies like Nvidia when it comes to keeping earning numbers a secret until official publication time

Considering the amount of money riding on such results, with stock price swings in the hundreds of billions sometimes, I can imagine they must have to go to some pretty extreme measures. But does it always work?

This isn't the guy with the orange futures in a briefcase in trading places, that’s for sure!

🗣️🎙️🤖 It has just come out that Amazon’s next version of Alexa will be largely powered by Anthropic’s Claude. It’s expected to come out in October.

That feels way overdue, but Alexa’s ecosystem has to support countless third-party plug-ins and “smart home” devices and appliances… It must be quite a headache to make foundational changes without compromising the reliability of all that stuff. You also have to deal with hallucinations and debugging impossible-to-reproduce issues because of the much fuzzier and less deterministic nature of LLMs.

I’ll be curious to see how it turns out! But I know that my kids would love to be able to ask Pokemon and Minecraft questions without having to borrow my phone.

---

🏦 💰 Business & Investing 💳 💴

Nvidia Free Cash Flow per share over time 💚 🥃 🧗

🤖 🔥 Nvidia Q2 Highlights 🔥 🩻🔍🕵️

I don’t know how to think about writing about Nvidia here anymore. When I was first writing about the company a few years ago, we didn’t have the whole planet scrutinizing their every move. They may be ironic, but ironic earnings viewing parties at NYC pubs are still a sign that they dominate the zeitgeist right now.

At the same time, I don’t want to stop writing about something just because it’s popular. That’s too hipster — it’s just as bad to like something just because others like it as it is to *dislike* something just because others like it. Either way, you’re not thinking for yourself, you’re outsourcing decisions to the crowd.

So screw it, let’s have a look at Nvidia’s Q2 and see what the highlights are.

Let’s not spend too much time on the financials because everybody’s seen them, but they’re still bonkers:

• Data center revenue is up 16% sequentially and up 154% year-on-year to $26.3bn. That’s 87.6% of total revenue in the quarter. Remember that most of Nvidia’s revenue was from gaming not long ago.

• EBITDA margin: 63.5%

• FCF margin: 45% (down from 57.6% last quarter for various reasons, but still incredible)

• Cash from operations: +128.25% YoY

• GAAP net income: +168%

• Authorized a new $50bn buyback on top of the remaining $7.5 authorization.

Let’s have a look at what the company had to say about its business and products:

Cloud service providers represented roughly 45% of our Data Center revenue, and more than 50% stemmed from the consumer Internet and enterprise companies.

As big as the cloud hyperscalers are, they are still less than 50%.

I can imagine a dynamic where the hyperscalers are probably early adopters of a lot of new tech, while the long tail of big enterprises is slower… Does that mean that as they start to catch up they’ll become a bigger % of the total vs cloud?

On the other hand, big cloud also hosts most of the biggest AI companies and their demand for GPUs will keep increasing rapidly as long as scaling laws for models hold.

Each successive foundational model uses at least an order of magnitude more GPUs (in the words of Colette Kress, Nvidia’s CFO: “Next-generation models will require 10 to 20x more compute to train with significantly more data. The trend is expected to continue.”), and those GPUs are themselves faster because they are of a new generation (going from A100s to H100s-H200s to GB200/Blackwell and so on…).

So I could also see the cloud maintaining or even increasing its share because of the massive training clusters and inference needs for the frontier models. Training is sexier, but let’s not forget inference. As Kress said: “Over the trailing 4 quarters, we estimate that inference drove more than 40% of our Data Center revenue.”

On China: 🇨🇳

Our Data Center revenue in China grew sequentially in Q2 and a significant contributor to our Data Center revenue. As a percentage of total Data Center revenue, it remains below levels seen prior to the imposition of export controls. We continue to expect the China market to be very competitive going forward.

Looking at the 10Q, I see China (including Hong Kong) going up by 33% year-on-year, but Singapore went up from $1bn to $5.6bn year-on-year…

If I had to guess, a lot of this volume is just a proxy for indirect China purchases, either going to the country by bypassing export controls or being set up outside of China and accessed remotely.

The 10Q makes this clear:

Revenue by geographic areas is based upon the billing location of the customer. The end customer and shipping location may be different from our customer’s billing location. For example, most shipments associated with Singapore revenue were to locations other than Singapore and shipments to Singapore were insignificant.

On Blackwell and demand vs supply:

Hopper demand is strong and Blackwell is widely sampling. We executed a change to the Blackwell GPU [mask] to improve production yields.

Blackwell production ramp is scheduled to begin in the fourth quarter and continue into fiscal year '26. In Q4, we expect to get several billion dollars in Blackwell revenue. Hopper shipments are expected to increase in the second half of fiscal 2025.

Hopper supply and availability have improved. Demand for Blackwell platforms is well above supply, and we expect this to continue into next year.

How brilliant was the Mellanox acquisition?

The last piece of the puzzle that they needed to vertically integrate full-stack bandwidth-hungry AI data-centers:

Networking revenue increased 16% sequentially. Our Ethernet for AI revenue, which includes our Spectrum-X end-to-end Ethernet platform, doubled sequentially with hundreds of customers adopting our Ethernet offerings. [...]

We plan to launch new Spectrum-X products every year to support demand for scaling compute clusters from tens of thousands of GPUs today to millions of DPUs in the near future.

Spectrum-X is well on track to [being] a multibillion-dollar product line within a year.

Adding the latest Llama 3.1 to their foundry:

During the quarter, we announced a new NVIDIA AI foundry service to supercharge generative AI for the world's enterprises with Meta's Llama 3.1 collection of models. This marks a watershed moment for enterprise AI. Companies for the first time can leverage the capabilities of an open source, frontier-level model to develop customized AI applications to encode their institutional knowledge into an AI flywheel to automate and accelerate their business.

I wonder if this will have an impact on how much focus Nvidia will put on its own foundational models. I might have missed it, but they don’t seem to have gotten much traction so far. Maybe as long as Meta is creating and releasing frontier models with open licenses, Nvidia can focus on being the best place possible to run and fine-tune them rather than try to compete with their base models (and maybe focus more on specialist/vertical models) 🤔

The world's largest electronics manufacturer, Foxconn, is using NVIDIA Omniverse to power digital twins of the physical plants that produce NVIDIA Blackwell systems.

Using GPUs and Omniverse to manufacture GPUs that run Omniverse. This is a cool Ouroboros moment. Or maybe it’s closer to this:

Here’s Jensen with some higher-level thoughts on AI:

AI is a bit of a universal function approximator and it learns the function. And so you could learn the function of almost anything, and anything that you have that's predictable, anything that has structure, anything that you have previous examples of.

It's a fundamental new form of computer science. It's affecting how every layer of computing is done from CPU to GPU, from human-engineered algorithms to machine-learned algorithms, and the type of applications you could now develop and produce is fundamentally remarkable.

On scaling:

the frontier models are growing in quite substantial scale. And we're still all seeing the benefits of scaling. And whenever you double the size of a model, you also have to more than double the size of the data set to go train it. And so the amount of flops necessary in order to create that model goes up quadratically.

And so it's not unexpected to see that the next-generation models could take 10x, 20x, 40x more compute than last generation. So we have to continue to drive the generational performance up quite significantly so we can drive down the energy consumed and drive down the cost necessary to do it.

Makes you wonder about the energy bottleneck that we will hit because energy infrastructure cannot keep up with this kind of exponential curve… ⚡️🔌

This bit I find telling about the commoditization of foundational models:

The second reason for Hopper demand right now is because of the race to the next plateau. The first person to the next plateau gets to introduce some revolutionary level of AI. The second person who gets there is incrementally better or about the same. And so the ability to systematically and consistently race to the next plateau and be the first one there is how you establish leadership.

Basically, the models are expected to be fairly similar, so you want to be first for bragging rights, or maybe to get some people to switch to your model when it’s ahead, and hopefully you can keep them as customers…

Jensen doesn’t seem to be expecting much differentiation between models, at least not with the current dynamics at play (this could change with algo breakthroughs or if someone races ahead because of operational excellence at deploying the hardware and doing the training runs).

I think that’s enough for today 👋

😬 Intel vs AMD + TSMC, Fall From Grace Edition ❤️‍🩹🐜

Ben Thompson (💚 🥃 🎩) has this great line on Intel, putting things into context:

Intel ended last year with 124,800 people; to put that in context, TSMC had 76,478 employees and AMD 26,000, which is to say that the two companies combined had fewer employees than Intel while making better x86 chips, an actually competitive GPU, and oh yeah, making chips for everyone else on earth, including Apple and Nvidia.

Ouch.

🦅🔐 Crowdstrike Q2 (first earnings post IT SNAFU) 😬🫣

Let’s have a look at another interesting one. Ever since the mother-of-all-IT-SNAFUs 💣🟦 🟦 🟦, I’ve been curious to see what Crowdstrike would have to say about it and what kind of impact it would have on the numbers.

Quick numbers for context on the Q: