411: Microsoft's Athena AI Chip, Population Crisis, GPU Bottlenecks, Industrial Titans, Samsung vs Google, EU Chips Act, and Tetris

"potentially rival the significance of the discovery of penicillin"

That thing that made you weird as a kid could make you great as an adult — if you don’t lose it.

—Kevin Kelly

I think you and I saw this coming:

‘The winner of a major photography prize has rejected the award after revealing that the winning image was generated by AI. ‘

The World Photography Organisation announced the winners of the Sony World Photography Awards 2023.

Among many stunning images that took home various prizes was “PSEUDOMNESIA: The Electricia” by German artist Boris Eldagsen, which won in the Creative category. A Sony press release described the image as “a haunting black-and-white portrait of two women from different generations, reminiscent of the visual language of 1940s family portraits.”

The artist rejected the prize, revealed that the image was made with generative AI, and called for a discussion about the nature of photography:

“I applied as a cheeky monkey, to find out if the competitions are prepared for AI images to enter. They are not,” he wrote. “We, the photo world, need an open discussion. A discussion about what we want to consider photography and what not. Is the umbrella of photography large enough to invite AI images to enter—or would this be a mistake? With my refusal of the award I hope to speed up this debate.” 

This is a good illustration that while generative AI can unbundle the creativity from the execution, they are both still very important, and the photo is as good as it is because Eldagsen has been a photographer for 30 years and spent a lot of time and energy fine-tuning prompts, doing inpainting and outpainting, and “draw[ing] on [his] wealth of photographic knowledge” to achieve this result.

You can find more context and backstory on Eldagsen’s website.

🍄🎧📺 In case you missed it, here’s my The Last of Us podcast with Matt Reustle and Tinkered Thinking. It was lots of fun, I hope you enjoy it!

🛀💭 If a method could be developed to teach Bayesian reasoning to children aged 10 to 15, and have it reinforced annually in school, the impact on humanity could potentially rival the significance of the discovery of penicillin 🤔

😊🙏🤕🫙🔮 The Warren Buffett test to ‘determine whether you live a blessed life’:

Warren Buffett often suggests the following thought experiment: assume that every human being currently alive, all eight billion souls on Earth, is a marble placed in a massive jar. 

If given the opportunity, would you put your marble in with the others, shake the jar, take another marble at random, and live that life instead? 

If the answer is no, then you know you have a blessed life.

h/t Rob Henderson

💚 🥃 🐇 This is the free edition of Liberty’s Highlights with 16,700+ subscribers.

You can get 2 extra editions/week full of juicy stuff + access to the private Discord 🗣🗣🗣 community by becoming a paid supporter (it’s quick & easy).

Paid post since last week 🐇:

• 🔒 409: Interview with Nvidia CEO Jensen Huang, Andy Jassy's Letter, ARM IPO, Google's TPUv4, HBO Max, and Video Game Development

• 🔒 410: Disaggregating P/E, Elon Musk AI Startup, Intel Foundry + ARM, Apple Manufacturing, Mr Beast, Deepfakes, and Rick Rubin

If you click the links above, you can see the intros for free and there’s a link to get a risk-free 7-day free trial of the paid version. You can only gain from this!

---

🏦 💰 Liberty Capital 💳 💴

🇯🇵 Population Crisis: Japan is now the ‘high’ fertility wealthy country in Asia 🤰🏻👩🏻‍🍼

Few things have a greater long-term secular impact on society than demographics, not only on the economy, but also on how vibrant a society is in general.

But things are not exactly moving in the right direction. Let’s look at Asia:

Noah Smith writes:

Meanwhile, Japan, the country whose name has become synonymous with low fertility, still has low fertility - between 1.3 and 1.53, depending on which estimate you use. But its rate is now the highest in its region (except for possibly North Korea, which doesn't give reliable numbers).

In Edition #393, I wrote about Japan’s effort to try to improve fertility in the country:

Japanese prime minister Fumio Kishida hopes to change the demographic trend with what he has promised will be an “unprecedented” set of measures. [...]

“The need to address the issue of children and child-rearing policies is a challenge that cannot be postponed,” he said. “We must create a children-first economic society and reverse the birth rate.”

In Edition #320, I wrote about the situation in South Korea:

That’s bad. From 6 children per woman to under 1 in one lifetime.

Cynics may sound smarter, and nothing sounds smarter than the post-Malthusian misanthropic meme of “well, there are too many people anyway, humans are a disease on the planet and the sooner we’re gone the better, if we shrink we’ll be more sustainable, etc”

It may sound sophisticated at cocktail parties, but it’s a very bad take.

The planet is large enough for many more billions of people, and the universe — as the JWST is reminding us — is nothing but endless space and resources, with nobody around to appreciate it (at least in most of it as far as we know).

Expanding human consciousness and keeping our civilization vibrant enough to keep solving our problems is the way out, not a slow collapse into a zero-sum world…

Existence at all is so unlikely that it is a privilege, and we should be very careful about wishing non-existence on billions of future people based on shaky economic theories or temporary, solvable problems.

After all, if our ancestors had stopped trying because they were so sure there would never be enough food for 1bn+ people on the planet, where would we be today…?

In the same way that a big city has more innovation, creation, entrepreneurship, cross-pollination between smart people, etc, than a small village, a planet with 10 billion people is more vibrant than one with 1 billion.

We just have to decouple the number of people from various bad things (ie. It’s probably worse to have 1 billion people powered by coal and oil than 10 billion powered by nuclear and deep geothermal).

Back in Edition #284, I wrote about fertility in the USA 🇺🇸:

The fertility rate in the U.S. has been below replacement (2.1/woman) since the mid-1970s (thank you immigration for helping with that!)

Immigration may seem like a solution, but it only works at the country level. At the planet scale, it’s just re-arranging pieces on the board, not creating new ones.

Microsoft has been working on ‘Athena’ AI-accelerator chip since 2019 🤖🐜

The software giant has been developing the chip, internally code-named Athena, since as early as 2019, according to two people with direct knowledge of the project. The chips are already available to a small group of Microsoft and OpenAI employees, who are testing the technology, one of them said. Microsoft is hoping the chip will perform better than what it currently buys from other vendors, saving it time and money on its costly AI efforts (Source)

It’s reported that at least 300 people are working on this piece of custom silicon.

At first it was thought that Marvell was Microsoft’s partner, but “a source with direct knowledge” says that MS owns the IP. MSFT 0.00%↑

It seems unlikely to me that they could’ve built the whole thing in-house, so I suspect that there’s something we don’t yet know about it… 🤔

🤖🤖🤖🤖 AI model scaling challenges and GPUs supply 🤖🤖🤖🤖

Sarah Constantin has a very interesting post about AI scaling laws.

I recommend you to check out the whole thing, but I want to highlight this section:

The current largest AI model as of April 2023, in terms of compute, is GPT-4, using 2.4x10^25 FLOPs.

GPT-4 was trained on tens of thousands of Nvidia GPUs and reportedly GPT-5 is being trained on 25,000 Nvidia GPUs.  OpenAI says they trained GPT-4 for six months.

In a recent interview, Sam Altman said that they weren’t training GPT-5 yet, but whenever that happens, it’s likely to use a bigger cluster of GPUs than GPT-4.

Currently the largest supercomputer in the world is 5.3x10^17 FLOPs, which (at 20% utilization and continuous usage) would have taken 7 years to train GPT-4.

If AI models continue to scale up compute, how does that compare to GPU production worldwide?

Reportedly, there were 136,000 GPU servers shipped worldwide in 2022.

So, OpenAI is already consuming, apparently, nearly 20% of the world’s high-performance GPUs?

I’m not sure if that figure is right, but it’s shocking if true.

I’m also unsure about the numbers and there are many assumptions made, but it seems plausible that it could be in that ballpark, or soon be when larger models get trained.

I’m also curious to know how much compute Google, Meta, Nvidia, etc, are using for training. Some of it is probably on ASICs like TPUs, but the general idea is the same.

There are millions of consumer-grade GPUs being shipped, but the data-center GPUs that are needed to train large models and the very fast interconnection fabric that allows them to shuffle data back and forth are a much scarcer resource.

Epoch AI finds a steady exponential growth trend in GPU FLOP/s from 1848 models of GPU between 2006 and 2021. The doubling rate is about 2x every 2.31 years, or slightly slower than Moore’s Law. 

This implies that, if GPU performance trends continue until 2030 and world GPU production doesn’t increase by a lot, the maximum amount of “compute” in an AI model can’t grow more than 3 orders of magnitude by the end of the decade.

I suspect that we will get a lot more juice from algorithmic improvements going forward rather than from just throwing more hardware at it.

There is a natural cycle to these things. At first, victory is to get the thing to work at all, and then as it matures, you can iterate, polish, and squeeze more efficiency out of it.

h/t Extra-Deluxe supporter Mikey Rabinowitz (💚💚💚💚💚 🥃 👋)

🏭🏭🏭 Lessons for tech from the old industrial titans

I enjoyed this conversation about what today’s tech industry can learn from the older industrial conglomerates that have, over decades, developed very effective processes for creating bottom-line value (rather than just topline growth).

• 🎧 Rob Wertheimer, Scott Davis - Lessons from the Industrial Titans

I’ve only read one chapter from the book and I enjoyed it.

I so rarely see long-form anything about Roper that I had to check it out. I learned a bunch about Brian Jellison although not much that I didn’t know about the company itself… ROP 0.00%↑

Samsung considering switching defaults search to Bing on its devices 📲

Google’s employees were shocked when they learned in March that the South Korean consumer electronics giant Samsung was considering replacing Google with Microsoft’s Bing as the default search engine on its devices. [...]

Google’s reaction to the Samsung threat was “panic,” according to internal messages reviewed by NYT. An estimated $3 billion in annual revenue was at stake with the Samsung contract. An additional $20 billion is tied to a similar Apple contract that will be up for renewal this year.

This feels very likely to be a negotiation tactic to get better terms out of Google, but even if that’s what it is, just the *perception* that Bing is now a likelier choice can cost Google real bucks. GOOG 0.00%↑

🇪🇺 European Union trying to keep up with U.S. with its own 43 billion-euro Chips Act to boost its semiconductors industry 💶 💶 💶 💶 💶 💶 💶 🐜

The European Parliament and the bloc's 27 member states struck an informal agreement for the 43 billion-euro ($47 billion) Chips Act, which pools public and private funds and allows for state aid to kick-start massive investments for chipmaking facilities. [...]

The EU Chips Act will link research, design and testing as well as coordinate EU and national investment. It's aims to help the semiconductor industry develop so that the bloc's global market share of chip production can double to 20% by 2030.

It’s not clear if there’s a path to success or if most of this money will be spent without moving the needle much… I don’t have high hopes 😬

---

🧪🔬 Liberty Labs 🧬 🔭

🥕👨‍🌾 👁️ ‘A WWII Propaganda Campaign Popularized the Myth That Carrots Help You See in the Dark’ (to hide the existence of radar! 📡)

The science is pretty sound that carrots, by virtue of their heavy dose of Vitamin A (in the form of beta carotene), are good for your eye health. [...]

the truth has been stretched into a pervasive myth that carrots hold within a super-vegetable power: improving your night-time vision. But carrots cannot help you see better in the dark any more than eating blueberries will turn you blue.

“Somewhere on the journey the message that carrots are good for your eyes became disfigured into improving eyesight” [...]

Ok, here comes the cool part: Carrots may have been used as a subterfuge to hide the existence of radar tech! 📡

During the 1940 Blitzkrieg, the Luftwaffe often struck under the cover of darkness. In order to make it more difficult for the German planes to hit targets, the British government issued citywide blackouts. The Royal Air Force were able to repel the German fighters in part because of the development of a new, secret radar technology. The on-board Airborne Interception Radar (AI), first used by the RAF in 1939, had the ability to pinpoint enemy bombers before they reached the English Channel.

But to keep that under wraps, according to Stolarczyk’s research pulled from the files of the Imperial War Museum, the Mass Observation Archive, and the UK National Archives, the Ministry provided another reason for their success: carrots. (Source)

It is not clear how effective it was and whether it worked just as well on the British as it did on the Germans, but it’s an intriguing potential source of this belief! 👀

🏋️‍♂️👴🏻🧓🏻🥇 Dr. Peter Attia on Longevity

Peter's new book is out, as I recently wrote about, and to promote it he’s been doing a bunch of interviews.

This one with Patrick O’Shaughnessy (☘️) is excellent:

• 🎧 Peter Attia - The Portfolio to Live Longer

They discuss many great things — their discussion on the importance of exercise and VO₂ max made me go out for a run 🏃‍♂️ as I listened to the second half of the podcast — but my highlight is about the concept of “resumé vs eulogy virtues”:

Peter Attia: I don't know what drew me to it, but sure enough, I just grabbed this book, The Road to Character. And the book talks about the differences between eulogy virtues and resume virtues.

I realized, boy, […] every single thing I've done to date has been resume boosting, nothing has been eulogy boosting. My eulogy is going to be very hollow. It will just be the usual platitudes that people say. My kids, if they were old enough, or my wife would really have much to say about me that is exceptional.

They could talk about how smart I was, and how hard I worked and all of these things, but I know deep down that that's irrelevant.

I think it’s a good way to think about things and help find some harmony between various aspects of our lives.

I also loved Patrick’s line near the end:

“Very often, purpose is borrowed, this is the mimetic idea.”

Worth thinking about how much purpose you’ve borrowed from others…

🇮🇳 India considering a major push into nuclear power ⚛️

As Germany is shooting itself in the foot by shutting down its remaining 3 world-class nuclear power stations, leading to more coal and gas being burned, India is looking to go the other way:

India’s government is pushing for construction of more nuclear power plants as the country looks to increase its supply of cleaner energy…. calling for as many as 20 new nuclear power facilities to be brought online over the next decade, more than doubling the number of operating nuclear power plants in the country.

Haryana state in northern India will be home to a 1,400-MW facility already under construction near Gorakhpur village, about 90 miles northwest of New Delhi. That plant will feature two 700-MW pressurized heavy-water reactors (PHWR) of Indian design.

The U.S. is backing this:

The U.S. and India in 2019 signed a deal in which the U.S. pledged to support construction of at least six nuclear power plants in India, and the two countries in February revisited previous agreements from as long ago as 2008 that could facilitate U.S. backing of India’s nuclear power program. (Source)

Modi’s government has talked about a goal of having 500 GW of energy from non-fossil fuel sources by 2030, which would require a rapid ramp-up of nuclear and renewable energy generation capacity.

This would be a major change, as India only has 6.8 GW of nuclear power right now, about 1.7% of total capacity.

🇵🇱 Poland is going nuclear too, with help from the U.S. ⚛️

Speaking of nuclear, Poland is also pulling an anti-Germany and making moves that could actually give its industry a significant long-term energy cost and reliability advantage over its neighbor:

the U.S. Export-Import Bank and U.S. International Development Finance Corporation signed letters of interest to lend up to $3 billion and up to $1 billion, respectively, to the Orlen Synthos Green Energy project. It's aims is to develop around 20 small BWRX-300 modular reactors designed by GE Hitachi Nuclear Energy. [...]

Poland’s first BWRX-300 reactor should launch in 2029 and will be the world’s second, after a similar one opens in Darlington, Canada.

Poland has one of the dirtiest power grids in Europe, with 70% of electricity coming from coal, so this is a big change.

---

🎨 🎭 Liberty Studio 👩‍🎨 🎥

🍿 I saw the ‘Tetris’ film (2023, Apple TV+) 🧱

I’ll share my impressions without too many spoilers, though it’s kind of a hard-to-spoil film when I think about it…

I was a bit disappointed, but it’s possible that my expectations were too high. The trailer that I saw a few months ago made it look like a kind of Cold War period piece with business machinations and video game nerdery. It seemed a lot of fun!

And it *was* fun, but the quality was rather uneven, and at times, the world seemed too small in ways that highlighted the limits of the film’s ambitions (and probably budget).

The young Ted Lasso-type hero flies around the world, goes back and forth between Russia, the US, and Japan, but somehow it’s always the same 10 people. The business intrigue stays rather simplistic.

My favorite scene was the Gameboy reveal!

That one gave me chills, probably because I had just listened to the Nintendo episode of Acquired.

A lot of the characters are kind of cardboard or caricatures, a lot of the writing is rather broad or on-the-nose… Thankfully the topic was interesting enough that I still enjoyed it, but overall I’d give it a C or maybe a C+.