It got me thinking about how these extremely large language models are trained on large corpuses (the real plural of corpus is corpora, but hey, I’m a rule-breaker) of text, a lot of which is compile from the public internet.
You realize what this means?
As someone who has been writing online for almost 3 decades, I’m in the machine.
There’s a tiny tiny tiny part of the model — the 🧠 of GPT-3 — that comes from me.
🛀 When it come to the average number of tabs that you have open in your web browser, there’s probably an inverted smiling curve effect:
Too few, and you may not be curious enough…
Too many, and you’ve lost control, you need a better system before you have a mental breakdown…
But there’s a number in the middle — and I don’t know exactly what it is, I’m sure it varies by person — that is probably correlated with healthy mental habits.
Me?
Oh, I’ve lost control. For sure…
That’s just *one* of my multiple browser windows, and I’ve got tons of things bookmarked in notes “for later” *sigh*…
🎧 🎙 Podcast discovery is a tough problem.
There are so many podcasts in existence, there must be many that I could learn greatly from, or be very entertained by, but I don’t even know they exist.
My current method is to randomly stumble upon podcasts because someone who’s taste I trust recommends it to me, or someone I already know and like is a guest and I initially listen because of that person, but I like the host and start exploring the rest of the episodes.
But I wish there was a recommendation engine similar to Amazon’s book algo (“People who bought this also bought X, Y and Z”), but with extra context on demand.
I don’t want just a list of “recommend podcasts”, like many apps try to do.
Podcasts are a bigger commitment, so I need a bit more context, not just a list of 50 recommended podcasts. What am I supposed to do, go through them one by one? I wish I could narrow it down more effectively, maybe based on short blurbs + quantitative metrics (“listeners of ‘Omega Tau’ and ‘Peter Attia’ loved XYZ with a correlation of 86%”)… ¯\_(ツ)_/¯
One of the great illustrations of the sunk cost fallacy is the extreme frequent occurrence in which a one-pound burrito is delivered point-to-point by means of:
A 2,800-lb car, driven by
A person devoting 100% of their attention to delivering said burrito, while
Following a prescribed route consisting of a series of turns on a mostly-2D grid, while
Moving at a variable speed because of stop signs, stoplights, and other obstacles.
Of the ~2lbs of CO2 emitted in a three mile version of this little transaction, 99.6% are spent on moving a person and a vehicle, while the other 0.4% are devoted to moving the food. The cost breakdown is harder to estimate, but a big chunk of it is paying a person to maneuver a fairly large vehicle around other rather large vehicles, all to deliver something comparatively tiny. [...]
This situation is temporary.
Soon enough, autonomous drones will be the dominant way to deliver small payloads—lunch, a snack, coffee, medication, household products
Makes a lot of sense to me, but the first question that comes to mind is:
Do we have tricks up our sleeves to make drones *much* quieter than they currently are?
Because that may be one of the biggest barriers to adoption in many places if these things are like flying leaf-blowers...
AI/ML is seeping into every crevice…
…often without the end user even having to think about it.
This is Manuvir Das, who’s Head of Enterprise Computing at NVIDIA:
Today, when we talk about an enterprise company using AI, they need to understand what AI is. They need to learn how to do AI, how to do training, how to do inference. But going forward, I think you'll see more companies adopt AI in a transparent manner. And what I mean by that is I have some application I used today from some vendor, right?
Maybe it's a document editing platform, right, like Office 365 Or it's an ERP system.
And the vendor just says to me, hey, there's a new version of this thing, guess what? It's now infused with AI. And so it's better because of that. A great example is if you're editing in Word today, you start typing a sentence, and it will suggest a completion for the sentence. And that is using AI, right?
So in this case, the company that's adopting it does not have to learn AI technology at all, they're just deploying the next version of an existing product that has been infused with AI.
But let’s be careful not to go too far…
That’s on the software side.
On the hardware side, there’s this vision that the silicon used to accelerate AI workloads (mostly on the inference side) are going to go from Ferraris to Toyotas and also going to spread almost everywhere:
it is absolutely the case today that if you just look at how many servers are shipped, right, new servers that are shipped and going into enterprise data centers, servers with GPUs in them are in the single-digit percentages, right? That is the case today. And we expect that, that will change dramatically.
The reason why it is the way it is today is because there has been a general mental model that you need a special kind of server to do AI, a server that you pack it to the gills with GPUs. You put 4 GPUs, 8 GPUs in the server, it's a bespoke thing. It's not a server that costs $10,000. It's a server that costs $100,000 or $200,000. And it's like a Ferrari.
And when you buy one of those, you better be running AI on a 24/7 or you wasted your money, right? That's been the general model, and that's why the penetration has been what it is. [...]
The shift that we are working on now with our customers is take a general purpose server […] You take that $10,000 server, you put one GPU in it that costs a few thousand dollars. And now you've got a server that on the one hand, can run your traditional workloads. [...]
So it's a multipurpose server.
And when that begins to happen, we expect it will drive a significant shift in this percentage, right? Because it's the mainstream volume servers that will begin to ship with GPUs in them. And as Jen-Hsun said, we fully expect that in the fullness of time, this will become the standard. And so if you look enough years out, the majority of servers will have GPUs in them [...]
Today, the AI servers with GPUs in them are bespoke. They've put in a corner of the data center, they serve one purpose only, whereas the servers we're working with OEMs are now along with the software we've built, these are multipurpose, the new workhorse of your data center. [...]
Your data scientists should just be thought of as another set of users who can come and access the same pool and your IT team should be able to just expand the pool naturally to these use cases, right? And that's why we did the work with VMware.
Summer Redstone Anecdote 🔥🏨🔥 🚒
First time I saw that one:
One of the underrated determinants of success is mere willfulness, which Redstone had in spades—he survived a hotel fire in 1979 by hanging out of the window until his wrist had been almost burned through.
This is from the 2018 book “The King of Content” (which I haven’t read, so I can’t tell you if it’s good).
The part in the second half about re-wiring your taste for certain foods that you don’t like much *but that you know are good for you* was particularly useful.
I think I’m going to do that with sauerkraut — eat it as a side with some of my fave dishes for a while, see if I develop more of a taste for it. Been meaning to eat fermented foods more regularly, but so far no luck (I gotta experiment with kimchi more too).
Long-Term Storage of Digital Data in DNA
Using life's preferred storage medium to back up our precious data would allow vast amounts of information to be archived in tiny molecules.
The data would also last thousands of years, according to scientists.
A team in Atlanta, US, has now developed a chip that they say could improve on existing forms of DNA storage by a factor of 100.
"The density of features on our new chip is [approximately] 100x higher than current commercial devices," Nicholas Guise, senior research scientist at Georgia Tech Research Institute (GTRI), told BBC News.
"So once we add all the control electronics - which is what we're doing over the next year of the program - we expect something like a 100x improvement over existing technology for DNA data storage." [...]
Scientists have said that, if formatted in DNA, every movie ever made could fit inside a volume smaller than a sugar cube. [...]
The current record for DNA digital data storage is around 200MB, with single synthesis runs lasting about 24 hours. But the new technology could write 100 times more DNA data in the same amount of time. [...]
This type of data is currently stored on magnetic tapes which should be replaced around every 10 years. (Source)
The problem with magnetic tapes is that they deteriorate over time, so even if they’re cheaper than DNA at first, having to duplicate and replace them ever X years adds up over longer periods.
And the cost of long-term magnetic storage isn’t on the cost curve decrease that writing and reading DNA is, so it seems just a matter of time…
Vaccine to clear up senescent cells 🧫🔬
Senescent cells refer to those that have stopped dividing but do not die. They damage nearby healthy cells by releasing chemicals that cause inflammation.
The team identified a protein found in senescent cells in humans and mice and created a peptide vaccine based on an amino acid that constitutes the protein.
The vaccine enables the body to create antibodies that attach themselves to senescent cells, which are removed by white blood cells that adhere to the antibodies.
When the team administered the vaccine to mice with arterial stiffening, many accumulated senescent cells were removed and areas affected by the disease shrank. When administered to aged mice, their frailty progression was slower than that of unvaccinated mice, according to the team. (Source)
A mix of removing these so-called ‘zombie’ cells via our own immune system (they would normally self-destroy via apoptosis, but some cells don’t follow the script and just stick around, causing all kinds of problems) and adding new healthy cells via stem cell therapies could be an interesting approach to lowering the risk to many diseases of aging.
COVID19: Updating what we track 🦠 🏥 🩺
Makes a lot of sense to me as we transition to a more endemic phase over time.
Right now the no-brainer thing to do is to give 3rd doses to older cohorts ASAP, as only a small minority have had it, and they’re the most at risk even when vaccinated (vaccines aren’t magic, they just train your immune system, so if you have a weakened immune system…).
Bayesian Look at COVID-19 Cases in a Heavily Vaccinated World (Redux)
This is something I wrote back in edition #168, and I think it’s time to bump it up:
This is probably an important one to get in front of: At some point in the future, almost everybody hospitalized with COVID-19 will be someone vaccinated (that’ll happen at different time in different countries, of course).
There’s an example that is often given to explain Bayesian probabilities, and the short version is: if 1% of women have breast cancer, and you screen them with a test that is correct 99% of the time (I won’t get into sensitivity and specificity), getting a “you have breast cancer” result should still mean that there’s only a 50/50 chance that you actually have it. (longer explanation here)
Why? Because the disease is rare enough that there’s so many more people who don’t have it, so just 1% of this vast group getting a false positive means as many people as the people who actually have it.
We’ll get to a similar point with COVID cases, because if almost everybody is vaccinated [in some places] (and most unvaccinated have had it and now have antibodies), then even if vaccines are very effective at protecting you, some small % of vaccinated people will still get sick and end up hospitalized.
This small % of a huge vaccinated population can be as much or more than the large % of the smaller unvaccinated population over time, and so you’ll get headlines about how hospitals are full of vaccinated people, and to those who don’t understand what I just explained, it’ll seem scary.
In the U.S., because vaccination rates are relatively low and vaccines are so effective at preventing severe outcomes, it’s still not the case. But as as we can see in this data from NYC, the per-capita numbers are very very different between vaccinated and not:
Liberty’s Highlights is reader-supported. To support my work, consider becoming a free or paid subscriber. 🦑
The Arts & History
Mad Men S5E11, 'The Other Woman'
Recently saw this one. What a powerful episode, one of the best-written of the whole series.
A-plot, B-plot and C-plot are all strong, with a unified theme across.
Everybody’s hurting others because of unrelated reasons that the victim doesn’t know about, as is often the case in real life.
That moment in the photo above... And the chronology trick with Joan. Umpf.
Guilty of preparation-procrastination..
Don't buy the book just yet, I wrote up Redstone's story :-)
Make a sauerkraut smoothie. Not as bad as eating it. And Kimchi is definitely better than both. And there is a cheat code: https://www.avogel.co.uk/food/products/molkosan-digestion/