114: Cloudflare's Secret Sauce, China's Big Tech, Jack Dorsey's Doc, RISC-V, the Data Era, Solar Cost Curve, One-Person SaaS, Double Negative, and Contagious Misery

"it’s not Worcestershire"

“Nearly all men can stand adversity, but if you want to test a man's character, give him power.”

—Duc de la Rochefoucauld

🎓 If I could add one concept to the curriculum of young school kids everywhere, it would be to teach them about the positive value of negative things that are prevented from happening (ie. sins of omission vs commission and their reverse: preventing sins of omission and commission).

Like when the government wants tech companies to weaken their encryption or build backdoors in their products. It may help solve certain crimes, but how many crimes don’t happen because of the stronger security? These have to be valued too, even if they literally don’t exist because they’ve been prevented.

A good life is about actively making good things happen and actively preventing bad things from happening.

If you stop preventing lots of bad things from happening as a trade-off to actively make a few good things happen, you have a blind spot.

I understand that the unthinking guide for many people is “can I be blamed for this/can I get the credit for this?” more than “is this decision creating the most positive outcomes in the world?”, but that’s why we need to teach people about it when they’re young and do our best to compensate for our brain’s built-in heuristics.

😕 A locked account on Twitter wrote:

"living anyone else's life except for your own is a recipe for misery"

It’s one of those simple-but-not-easy insights that we do well to re-learn periodically, and to check on to make sure we’re not succumbing to those external pressures.

But I think it’s even worse than just that, because misery tends to be contagious, so if you fall int that trap, you'll probably also make the people around you miserable, and may even teach your kids to be as miserable as you are (and hopefully they don’t pass that on to their kids)...

🛀 I think very money-motivated people often have trouble believing (or imagining?) that there are some for whom money is a weak motivator.

I think really internalizing that it's a real thing can help those people have a more accurate — and thus more useful — model of the world.

Personally, I’m not very money-motivated. I’m more freedom/independence-motivated.

Sometimes, that pushes me in same direction that someone money-motivated would go, but at other times, in the opposite direction (ie. Not pursuing a bunch of career options that may be more lucrative, but would cut severely into the control I have over what I do all day and the optionality to follow my curiosity).

📩 I wrote an email to the person in charge of the daycare where my youngest goes.

I understood why they kept windows closed when it was still cold outside, but lately the weather has been nice, and the windows were still closed (at least at the times I went there, maybe not always).

I recommended that they open them when it’s nice out, for ventilation.

Sent some of the latest stuff from the CDC about how most transmission is through the air, explained the virus spreads almost like cigarette smoke in a room, and that with enough airflow you can reduce risk by quite a lot, which is why we find a lot fewer cases of transmission outside.

She seemed receptive. I hope they’ll do it.

Sometimes it doesn't take much to make something happen. Most people want to do the right thing, but things that may seem obvious to us may not be to others, so with the right information and explanation…

Maybe you should send a polite email to your kids' daycare too.

---

Investing & Business

Cloudflare’s Secret Sauce (it’s not Worcestershire)

I’ve long been impressed by Cloudflare’s development velocity and engineering productivity (it sometimes feels like they do more with about 2k employees than many companies do with 10-20x that number).

I was reading the transcript of Cloudflare’s analyst/investor day, and Matthew Prince did a really good job explaining their model in the first part. He spent some time on some of the structural reasons why they’re able to achieve this.

It’s a bit of a long excerpt, but if you’re interested in the company, I think you’ll find it illuminating:

Most organizations are set up to have one product and engineering team. And that product and engineering team is set up to think about to talk to customers, listen to what their needs are and then constantly iterate making innovation after innovation [...]

that's what 90% of our product and engineering team do [...] But about 10% of our product and engineering organization is dedicated to something different. And that's what we call ETI, or emerging technology and incubation. And what's unique about that team is that they are not measured on immediate product adoption. They're not measured on revenue. They're not measured on what's going to be happening in the next quarter.

[...] focused on what the long-term opportunity is. … tasked with really dreaming up what the future is [...]

You ship your org chart, your teams produce what you incentivize them to…

So this is really smart.

I really wish I could visit this Q’s Lab/Batcave (though I guess with remote work, it’s not really anywhere)…

our Workers product, that came out of ETI. If you're excited about Cloudflare for Teams, that came out of ETI. What's powerful is once those products do take hold and once they do get adoption, then we graduate them into our traditional product and engineering ward, and then we're able to rapidly innovate and extend them based on what customer feedback is [...]

This part is crucial too.

Coming up with all kinds of cool stuff isn’t enough.

F.ex. there was an era at Microsoft when they had a great R&D lab that did really cool tech demos all the time, but could almost never turn these things into products, while Apple was tiny in comparison, but they could actual ship things and then iterate on them over and over until the products were really solid.

This “handing it off” to the 90% part of the eng team that can really execute and make it great is also crucial. One doesn’t work without the other.

those free customers are one of our secret weapons. They are one of the ways that we actually develop and test products and make reliable products faster and more reliably than anyone else.

of the 3 million-plus free customers that use our network, will typically get tens of thousands that raise their hand and volunteer to be beta testers or even alpha testers on these new features

Again, so valuable.

Many problems can’t be caught with just internal QA. You just don’t have large enough deployments, and it’s really hard to have realistic “real world” testing on a budget (your users always have weirder use cases and hardware than you expect).

But if you can deploy your products to lots of real users, you’ll both catch problems early, get really useful feedback on what to fix, what to change, what is working and what should be focused on, but also, by the time you get out of beta, your paid customers get something a lot more polished, which helps your reputation (higher reputation, lower CAC).

If you're a product manager at a traditional enterprise company, and you come up with a new idea, the first thing everyone else in the organization tells you is, "Don't tell anyone, certainly don't tell customers because we don't want them delaying upgrading to their next feature on the next cycle.” [...]

Secrecy is great when you’re building some integrated hardware consumer product like Apple. You want to make a media splash and leave your competitors trying to catch up.

But when you’re doing infrastructure services, what you want is the opposite of secrecy; you want as much sunlight and as many eyes on the thing as possible from as early as possible to catch problems and prove to skeptical crusty old IT guys that it actually works.

those [free] customers deliver an enormous amount of data. And in a security instance, that's hugely valuable. And so every time we see an attack on any one of our customers, the entire Cloudflare network effectively is one of the world's largest sensor networks …  our largest customers are able to be protected before anyone even is aware that a new attack is in the wild.

That’s another very powerful part of the model.

If you’re doing security, it really helps to train your models with lots of real world data to better help separate how regular users behave vs bad actors.

the real reason that we built Cloudflare Workers was actually to overcome a problem that we had internally. Back in 2017, we were starting to see our development process slow down to deploy a new feature across thousands of machines scattered all around the world was a slow, clunky and pretty scary process [...]

we implemented what we internally and somewhat cheekily called the Bezos Rule. And what the Bezos Rule is the exact same rule that Amazon put in place when they were developing AWS, which is, any API or any development tool that we build for ourselves and for our own team, we also are then going to make available to our customers [...]

our first customers were actually our internal developers. We've built it for ourselves first. [...] our strategy is to build the tools that we want to use and then turn around and make them available to our customers. [...] we're really focused on solving developer problems. And our internal developers look a whole lot like external developers in the wild

They first had to solve their own problem, but rather than keep that internal, which would’ve been easier, because it’s a lot more work to make an API polished enough to release in the wild — you can’t just create one-off hacks and duct tape things so they work for your particular use case — it now has to be all generalized.

So they opened it up, and created a solution to the similar problems that lots of others were having (how to deploy something everywhere easily) plus some new use cases (how to have different logic run in different places, like for data compliance around the world, how to make sure certain data never leaves certain jurisdictions or that certain requests are processed differently, etc).

All of their servers now run the same software stack while being built on cheap commodity hardware, and the products that they build internally are on the same platform that they’re opening up to external devs to build on.

🐙 ‘Global footprints of Chinese tech giants’ 🐙

h/t Lillian Li (check out her long-form analysis of Chinese tech companies)

Ransomware Shuts Down Factories by Encrypting Servers that Control Equipment

Ransomware operators shut down two production facilities belonging to a European manufacturer after deploying a relatively new strain that encrypted servers that control a manufacturer's industrial processes, a researcher from Kaspersky Lab said on Wednesday.

The ransomware, known as Cring, came to public attention in a January blog post. It takes hold of networks by exploiting long-patched vulnerabilities in VPNs sold by Fortinet. Tracked as CVE-2018-13379, the directory transversal vulnerability allows unauthenticated attackers to obtain a session file that contains the username and plaintext password for the VPN.

With an initial toehold, a live Cring operator performs reconnaissance and uses a customized version of the Mimikatz tool in an attempt to extract domain administrator credentials stored in server memory. Eventually, the attackers use the Cobalt Strike framework to install Cring. To mask the attack in progress, the hackers disguise the installation files as security software from Kaspersky Lab or other providers.

Once installed, the ransomware locks up data using 256-bit AES encryption and encrypts the key using an RSA-8192 public key hardcoded into the ransomware. A note left behind demands two bitcoins in exchange for the AES key that will unlock the data. (Source)

IT security. It’s a big deal.

Pricing Carbon

Max Rosner:

In 2004 a carbon price was paid on much less than 1% of emissions.

By now 78 different jurisdictions have implemented a carbon price and in 2021 a price will be paid on 22% of the world’s carbon emissions.

Source of the graph.

Jack Dorsey’s Philanthropy Google Docs

Chris Fralic:

1 year ago @jack gave 1/4 of his Square equity towards charity and is transparently sharing on a Google Doc how he’s donating it ($373M so far). And way back in 2015 he gave 1/3 of his Twitter equity (1% of $TWTR) in back to employees.

Here’s the Google Doc. Here’s Jack’s tweets from last April announcing this.

---

Science & Technology

Welcome to the Data-Centric World

This slide is by Applied Materials. I guess this makes it clear why Texas Instruments has been focusing so much on industrial semi content as one of its main verticals in recent years (and automotive too).

I posted some slides I found interesting in this thread. You can see the whole presentation here.

h/t Nick Ellis (thanks for the support! 💚🥃 )

Survival Curves from 1851 to 2011

Max Rosner:

I think 'survival curves' are a great way to visualize how mortality has changed over the course of history.

It might take a moment to wrap your head around how they work, but then you can discover a lot of information in them.

More on life expectancy changes over time (“It’s not just about child mortality”).

How Radial Engines Work

More details here.

Probably a good occasion to link this site full of animations showing how various types of engines work.

If you want more, there’s also this site with 507 types of mechanical movements.

RISC-V's Promise (What’s in it for China? The U.S.?)

Chinatalk has a good piece looking at RISC-V, mostly through a hybrid tech-policy lens.

• Labs Over Fabs: RISC-V's Promise

RISC-V still has a long way to go to be considered a real competitor to ARM or x86. Open-source hardware faces a far more challenging path to developing its ecosystem than software. First, the technical bar to contribute to an open-source hardware project is much higher and more specialized than most software projects. There are far more people who have picked up a software programming language like C than know enough electrical engineering to make sense of an ISA. [...]

Today, China has the liveliest RISC-V ecosystem. Every major Chinese tech firm seems to boast a RISC-V strategy, with Alibaba producing what may be the world’s fastest RISC-V chip targeting artificial intelligence applications. [...]

Rather than trying to protect specific firms or to acquire a defined set of technological capabilities, the [U.S.] government can help by supporting a healthy semiconductor ecosystem, including a well-trained workforce; an amply-funded venture capital environment, especially for early-stage firms; and an educational system that fosters new and disruptive ideas.

Software Infrastructure to Support One-Person SaaS Company

Pretty amazing to see what one person can do nowadays with modern software tools and third-party services/APIs/platforms:

• The Architecture Behind A One-Person Tech Startup

There’s still an incredible amount of work that went into designing and implementing all this, but the point is that it’s possible at all. A few years ago, there’s almost no way one person could’ve been able to do — and afford — all that is required with creating this product and operating the company and its infrastructure as a one-man-band.

Solar Power Cost Curve… and Spaceships

Patrick McKenzie:

One interesting consequence of the solar cost curve the last few years is that the pitch has been "Finance installation of solar power for N years, pay less in energy costs, difference repays loan" but because of rapid decrease in installed price it's been better to defer a year.

f you were receptive to that pitch in 2010, and didn't get non-economic returns to being solar powered, you would have been far better off paying for conventional power until 2015 then switching. (Results in you paying less than half as much.)

It reminds me a bit of a sci-fi novel I read a long time ago (was it ‘Forever War’ by Joe Haldeman? Maybe something else…) where a spaceship leaves for a distant star, and when it gets there many years later, there’s already a whole colony created by people who left later on much faster spaceships and got there way earlier. ¯\_(ツ)_/¯

---

The Arts & History

Just a Cool Shot of the Space Shuttle

I don’t remember where I got it, it was my laptop’s wallpaper image for a while.

Princess Bride GIFs

I made a few more GIFs for the side-business… I was tired of the super-pixelated Princess Bride GIFs floating around, so here are some of mine, fresh off the oven.