I’m going to leave the minute-by-minute Twitter meltdown news in more capable hands like Platformer and Garbage Day (which I highly recommend subscribing to if you find this news cycle interesting). What I wanted to touch on here today is a specific aspect of it that I am particularly fascinated by: the seemingly small failures that might end up taking the whole system down.

Right now there are lots of engineers who are “on call” for a bunch of Twitter systems that they didn’t work on. And if one of those systems go down and they get a page about it, not only do they not know how to fix it, there’s also, in many cases, no one left at the company who knows how to fix it (or they are to busy doing other things). So you get situations like this, as reported by Gergely Orosz:

Gergely Orosz @GergelyOrosz

Scoop from within Twitter: small things are breaking, not enough engineers to fix them. Noticed that notification counts are not showing? The BE service powering it is down since Thursday. A bug was filed, but the team that would fix it is full on with verification work.

2:25 PM ∙ Nov 12, 2022

---

454Likes104Retweets

Also see that TikTok video about how a local dev instance is currently running in production. Chris Stokel-Walker has a more detailed summary of all of this in Here’s how a Twitter engineer says it will break in the coming weeks:

It starts with the small things: “Bugs in whatever part of whatever client they’re using; whatever service in the back end they’re trying to use. They’ll be small annoyances to start, but as the back-end fixes are being delayed, things will accumulate until people will eventually just give up.” […]

The juddering manual retweets and faltering follower counts are indications that this is already happening. Twitter engineers have designed fail-safes that the platform can fall back on so that the functionality doesn’t go totally offline but cut-down versions are provided instead. That’s what we’re seeing, says Krueger.

But perhaps the most interesting part about all this is not that Twitter is dying, but that it is dying so slowly, thanks to breaking up the underlying technology in many different pieces (“microservices”) that can work independently:

Adam Roach @adambroach

[1/4] One of the fascinating things is watching Twitter’s behavior change, hour after hour, as various microservices go offline. It’s a real testament to that approach that things mostly work as various features wink out of existence.

5:25 PM ∙ Nov 11, 2022

---

4,021Likes674Retweets

So if—after the dust settles—more organizations prioritize breaking up their monoliths to move to microservices, I think we will at least be able to point to that as a silver lining…

What I’m reading

This week’s “must-read” is Vicki Boykis’s The art of the long goodbye—a moving piece about layoffs, and how the companies we work at shape us:

Every workplace has its own unique culture, its rhythms, its acronyms, its main characters and side characters, that one piece of software that everyone hates, and that one engineer that everyone relies on. A job can be just a job, but, regardless of whether you love or hate yours, our jobs and their component environment becomes terroirs where we work, struggle, hate, love, and places that shape who we are as human beings. If you have been in a terroir long enough, you will come out of it, for better or worse, transformed. 

I am hoping Vicki starts to write more frequently again, so make sure you subscribe to Normcore Tech to give her some extra encouragement!

Cal Newport’s reflections On Michael Crichton’s Busy Ambition are really interesting, and talks about the fact that there is no one right way to get work done:

In Crichton and Grisham we see two different models of ambition. The first model, exemplified by Crichton, is what I call Type 1. It craves activity and feasts at the buffet of appealing opportunities that success creates. The other model, exemplified by Grisham, is what I call Type 2. It craves simplicity and autonomy, and sees success as a source of leverage to reduce stressful obligations. Medical school wasn’t sufficiently stimulating for Crichton. Life as a lawyer was too hectic for Grisham. They therefore reacted to their success in much different ways when it respectively arrived.

We don’t have to copy whatever the latest productivity hack is. Instead, we should identify how we react under different circumstances, and then deliberately design the environments in which we do our best work.

I don’t really link to Lenny's Newsletter because I just assume that if you're subscribed here, you're already subscribed there. But just in case my assumption is wrong—if you're in Product / Marketing, please subscribe to Lenny Rachitsky. Always fantastic insights—such as in his latest post, How to determine your activation metric for Product-Led Growth:

Follow this three-step process:

Brainstorm, and explore your product usage data, for some potential “aha” moments in the user journey.

Run a regression analysis to see if there’s an inflection in retention when someone hits any of those moments, to establish a correlative relationship between some potential activation milestones and product retention.

Run some experiments to see if increasing the percentage of users hitting that moment increases their retention rate, to see if any of those correlative relationships translate into true causality. A good activation metric is causal for your retention, not just correlative.

→ For more on activation metrics (sometimes called “Time To Value”) and other measurements for Product-Led Growth, see 11 essential product metrics for measuring product-led growth.

And finally this week, sticking to the Product-Led Growth theme, Emily Kramer of MKT1 has another great post called Building an efficient marketing machine: the fuel & the engine:

The fuel is all the stuff that you say (out loud, in writing, or visually) to your audience—whether that be short-form copy on a website, a long-form blog post, an image in an ad, or an explainer video. The engine is all of the channels and processes you use to get the fuel out to your audience, plus the tools you use and the metrics you use to track your marketing and growth efforts. When you combine the fuel and the engine, your business grows fast.

She goes into more detail in the post about what tactics are considered “fuel” vs. “engine”, and how to get the balance right in your organization.

Some stray links

• 🏡 Airbnb Says Its Focus on Brand Marketing Instead of Search Is Working. “Its marketing spending is now low enough that it doesn’t anticipate drastic reductions even if economic headwinds worsen next year, it said.”

• 🐦 Twitter alternative: how Mastodon is designed to be “antiviral”. “The new social software is subtly designed to reduce the huge, viral surges of attention we see on Twitter.”

• 🗼 I absolutely adore this. “Studio Drift’s drones ‘rebuild’ incomplete monuments with light. From the Sagrada Familia to the Colosseum, the Dutch artist duo redesigns incomplete architecture using illuminating drones. The goal is to help architects visualize their designs at full scale and in a sustainable way.” (Instagram link, click through for more photos)

  domusweb

  

  A post shared by Domus (@domusweb)

Alright friends, that’ll do it for today. I hope you have a peaceful weekend. And if you enjoyed this, could I ask you to share it with someone you like? That would be really cool. 💛

Share