460: Tencent's GPU Hoard, Microsoft's Maia, Amazon to Sell Cars, SpaceX IPO?, Huawei vs Nvidia, DeepMind’s Lyria, and The Abyss
"Money without taste buys bigger mistakes."
That is an abstraction.
There have always been and always will be only individuals.
—Johann Wolfgang von Goethe
🍷🛢️🥃 🤔 During my exploration of scotch whisky a few years ago, I became fascinated with the concept of the solera.
I think it originates from the wine world. The idea is to do fractional blending in vessels that are never fully emptied.
Glenfiddich, a Speyside distillery in Scotland, has a 15-year-old whisky that uses a vatting process similar to the solera. The whisky is labelled as their "15 year old single malt Scotch Whisky". For Scotch whisky, the stated age must refer to the youngest of whisky's components. Barrels are emptied into the solera vat and mixed. Then whisky is drawn from the vat to be bottled, with the vat never being more than half emptied. Since the process began in 1998, the vat has never been emptied
In theory, if a winery or distillery is 200 years old and the solera mixing vat has never been fully emptied, some of the original wine or whisky is still in there.
Molecules from that original batch, and from every subsequent year, are going into every bottle, even if in smaller and smaller quantities per cohort each year (it’s a bit like Zeno’s paradox).
It got me thinking about what else is a solera, even if accidentally 🤔
I’m pretty sure that most vehicles’ fuel tanks are never 100% emptied, so there must be some molecules from very old gasoline or diesel in there (fuel is pretty volatile, but so is cask-strength whisky…)
How about batteries? Are there “old electrons” hanging around in my iPhone? haha
I know that in some industrial biochemical processes, like to create certain enzymes and biologics, the fermentation tanks are operated on a continuous basis. New nutrients are added and the final product is removed at the same time, always leaving some behind. (I wonder how long these can go on without needing a full flush and reset, or before they get accidentally contaminated.. What are the longest-running bioreactors out there? ¯\_(ツ)_/¯ )
Hot water tanks? In theory, you’re supposed to empty them once in a while, but few people do, and even then I doubt they are 100% empty…
🛀💭💵🚀🥱💰 Money amplifies.
Money without taste buys bigger mistakes.
One way to think about it is this equation:
Interestingness x Agency x Money
Someone could have 50 billion dollars, but if they don’t do anything interesting with it, well, it’s their money, whatever, but it won’t be notable to me. 🥱
But someone with almost nothing except a *very high* degree of interestingness and agency can do great things.
The quadrant that is truly magical is when you have someone with a significant level of interestingness *and* agency *and* substantial resources (translate: A wide range of options for what to do, a large potential decision tree, a lot of rocket fuel to throw on projects).
This combination can make a dent in the universe, to use Steve Jobs’ phrasing.
🎮 🧸🤗 Here’s a neat social experiment:
Can you play Fortnite as a pacifist?
Can you make friends in a battle royale deathmatch arena?
Great real-world experimentation by Pop Culture Detective:
Think of how many hours of gameplay it took to get these results, and how frustrating it must have been along the way…
🇺🇦🎧 This short podcast interview with Jack Watling, a Senior Research Fellow on Land Warfare and Military Sciences at the Royal United Services Institute for Defence and Security Studies, provides a good overview of the strategic challenges facing Ukraine as Russia’s war turns into a prolonged stalemate, which can favor Putin.
Not helping Ukraine to decisively fight back an invader and protect its population and freedom would not only be a tragedy for Ukrainians, but will lead to bigger problems for others down the road — from Russia, but also from other tyrants and would-be aggressors around the world who are watching closely for signs of weaknesses and lack of resolve in the alliance of democracies.
🏦 💰 Liberty Capital 💳 💴
🇨🇳 Tencent has a giant stockpile of Nvidia GPUs, enough to keep developing its AI models for a while 📦📦📦
In Edition #445, I wrote about China’s ‘Strategic GPU Reserve’:
Baidu, ByteDance, Tencent and Alibaba have made orders worth $1bn to acquire about 100,000 A800 processors from the US chipmaker to be delivered this year, according to multiple people familiar with the matter. The Chinese groups had also purchased a further $4bn worth of the graphics processing units to be delivered in 2024, two people close to Nvidia said. [...]
Tencent Cloud in April released a new server cluster — computing power for others to rent — wielding the Nvidia H800 GPU, a version of its latest H100 model adapted for China.
Alibaba Cloud has also received thousands of H800 chips from Nvidia
Tencent president Martin Lau recently gave an update on his GPU stockpile on the Q3 earnings call:
Now in terms of the chip situation, right now, we actually have one of the largest inventory of AI chips in China among all the players.
And one of the key things that we have done was actually we were the first to put in order for H800, and that allow us to have a pretty good inventory of H800 chips.
So we have enough chips to continue our development of Hunyuan for at least a couple more generations. And the ban does not really affect the development of Hunyuan and our AI capability in the near future.
It doesn’t sound to me like the US Chip Ban is achieving its objectives… 😬
Going forward, we will have to figure out ways to make the usage of our AI chips more efficient. We'll try to see whether we can offload a lot of the inference capability to lower-performance chips so that we can retain the majority of our high-performance AI chips for training purpose.
And we also try to look for domestic stores for these training chips.
Since Nvidia pretty much instantly announced new chips that were modified to pass the new export ban criteria, and the H20 seems pretty powerful*, I’m not sure if Tencent will feel much of a squeeze for now.
* The H20 is based on the H200, not the H100, and because you can do inference on a single GPU rather than having to spread it across multiple ones, it’s actually 20% faster at inference than the H100 using LLAMA 70B as a benchmark, according to friend-of-the-show Dylan Patel. The question is how it does for training. I don’t know.
🥇 Heuristic: CEOs praising their competitors 🥈
It's always a good sign when a CEO praises an obviously great competitor and doesn't try to pretend they don't exist or falsely claim they are worse than they really are. It’s a sign of honesty and being well-calibrated.
It reminds me of the old scuttlebutt trick of asking a bunch of people in an industry “If you couldn’t work for your company, who would you work for? Which of your competitors do you find scariest? Which do you admire the most?”
Then you cross-reference all the answers and see if a name keeps coming back.
🚘 🛒 Amazon to allow auto dealers to sell cars, starting with Hyundai
Keep reading with a 7-day free trial
Subscribe to Liberty’s Highlights to keep reading this post and get 7 days of free access to the full post archives.