Irrational Exuberance - 2023 in review. @ Irrational Exuberance

Hi folks,

This is the weekly digest for my blog, Irrational Exuberance. Reach out with thoughts on Twitter at @lethain, or reply to this email.

Posts from this week:

- 2023 in review.
- Notes on How Big Things Get Done

2023 in review.

Previously: 2022, 2021, 2020, 2019, 2018, 2017

This was an eventful year. My son went to preschool, I joined Carta, left Calm, and wrote my third book. It was also a logistically intensive year, with our toddler heading to preschool, more work travel, and a bunch of other little bits and pieces. Here is my year in review summary.

I love to read other folks year-in writeups – if you write one, please send it my way!

Goals

Evaluating my goals for the year:

• [Completed] Write at least four good blog posts each year.

  I wrote a lot this year, including adding five to my popular posts page, including: Writing an engineering strategy, Measuring an engineering organization, Setting organizational values, and Writers who operate.

• [Completed] Write another book about engineering or leadership.

  I did this: The Engineering Executive’s Primer goes to print in late January, and should be available for purchase in February. The complete digital version will be available via O’Reilly in January.

  I am currently brainstorming a bit on a fourth book, very likely my last on the topic of engineering leadership, although it’ll take a bit of time to decide whether and when to take that on. Right now mostly thining about the topic of engineering strategy.

• [Mixed] Do something substantial and new every year that provides new perspective or deeper practice.

  Like clockwork, I struggle to give myself a passing grade on this one. Joining Carta has greatly expanded my perspective on executive leadership. I also worked with a new publisher, O’Reilly, which provided a different view into the book creation process than self-publishing or working with Stripe Press. This was also the first year I gave a keynote, this one at QCon, which maybe qualifies?

• [In progress] 20+ folks who I’ve managed or meaningfully supported move into VPE or CTO roles at 50+ person or $100M+ valuation companies.

  This goal is due in 2029. Without spending much time thinking this through, there are at least five folks who qualify here, and I bet I could get to at least ten if I spent long enough digging into it

• [Completed] Work towards a clear goal for physical exercise. (Hitting the goal isn’t important.)

  Discussed a bit more below, but I reset my running habbit and worked back up to doing a few eight mile runs. I’m mostly doing four milers now that I’m working full-time again, but it was very validating to stretch milleage a bit!

Carta & Calm

I left Calm earlier this year. I planned to take a year off, but ended up joining Carta after a couple months. When I explain this to folks, particularly those who I’d already told that I wasn’t going to go back to work immediately, what I tell them is: I felt confident that I would regret declining the offer to join Carta.

That’s still how I feel ~nine months into the job. Personally, learning and impact are the two things I value most in my work, and Carta remains the highest indexing job I’ve ever had on both counts.

An Engineering Executive’s Primer

I started and finished An Engineering Executive’s Primer this year. Coming into the year, I expected to write another book this decade, but it wasn’t this book, instead it was Infrastructure Engineering, which I ended up not making much progress on. I wrote up notes on writing Primer, and altogether I’m proud of the book and how quickly it came together.

Other books

It feels good to finish #3, and I think I could put down the pen at this point and not feel like a fraud to consider myself a writer, but I’m not done quite yet. I still have at least one more topic I want to spend some words on, engineering strategy. (I have no idea if I’ll ever get back to the Infrastructure Engineering book, I’m finding it hard to marshall the focus onto a topic that I’m not working on directly day to day.)

My first two books, An Elegant Puzzle and Staff Engineer are both doing well. Have been translated a few more times and so on, but nothing too wild. As I mentioned last year, I’m working hard to focus on the new things I do, and not to spend much time thinking about stuff I’ve already done, hence not reporting on book sales and such anymore.

First keynote

I gave my first first keynote, Solving the Engineering Strategy crisis at QCon SF. You can see a video recording of that talk on Youtube. I’m mostly avoiding conference talks these days, but it was impossible to pass up the opportunity to give my first keynote, particularly a keynote that didn’t require traveling for the conference.

There aren’t any conference talks on my schedule for 2024 but if I do some it’ll probably be focused on the topic of engineering strategy.

Advent of Code

I made it through day twelve of Advent of Code this year before deciding I needed to bail out. Some years ago I read Tanya Reilly’s ode to Advent of Code, and attempted it that year before getting busy, and decided to try again this year as quite a few work, professional and friend groups participated. I really enjoy working on these, but they’re competing for writing project time, and that’s just hard to fit in with the work travel, family visits, and so on that happen around the holidays. Maybe next year.

Angel investments

I did four angel investments this year, and invested in one fund as a limited partner. This is, give or take, roughly the sort of angel investing year I expect to have most years going forward, but it’s not a goal or a priority. I just evaluate the interesting things that come my way and occasionally invest. (I am mostly interested in developer experience, productivity, and infrastruture startups, as it’s the space I understand best and where I think my input is most useful.)

Reading

After finishing up Primer, I’ve been doing a bunch of professional reading. Much of this has been related to collecting my thoughts on engineering strategy, but some of it has been mining for ideas (including structural and presentation ideas) both as a leader and as an author who writes books.

The professional book I’ve read in the last few months are:

• Enterprise Architecture as Strategy
• The Crux
• Technology Strategy Patterns
• The Software Engineer’s Guidebook
• Tidy First?
• The Value Flywheel Effect
• How Big Things Get Done
• The Elements of Content Strategy
• Just Enough Ressearch
• Wiringframing For Everyone
• Design That Scales

Year of personal admin

In addition to various work stuff, this was also a year of personal admin for me, where I tried to catch up on a few years of neglected tasks and ambitions around the house and my body.

Wearing glasses

I wore glasses until I was 13 or so, then I stopped wearing them, essentially on a whim. My vision was good enough for most purposes, including getting a drivers license, so I just didn’t think about it much for the following 20-plus years. Not thinking about it was nice.

When I started my new job, I started getting frequent migraines. Trying to diagnose what might be going wrong, it was clear that I was spending more time looking directly at a computer monitor that I had for several years, and I tried wearing a pair of old glasses that I had made about fifteen years ago as a last resort if I needed them to pass the vision portion of the California driving exam.

It turned out, this worked very well, and my eyes and head have felt much better since returning to wearing glasses. I don’t wear them all the time, but I do wear them whenever I sit down to do more than a few minutes of work on the computer. Age is, I suppose, more than just a number.

Running

Since graduating college, I’ve always been a frequent runner, although rarely been a serious runner. More concretely, other than a detour for a stress fracture, I’ve gone on 2-3 runs a week, averaging 3-4 miles, for quite a long time. In roughly 2013, I ramped up my runs for a while, building up to 6-8 mile runs twice a week for a few months, before ramping back down to my shorter runs. The shorter runs are nice because they take less time, and they also put a bit less strain on my knees which have at times been a bit unreliable. Plus, I generally prefer to stress my knees playing basketball instead of running.

This year, I wanted to build up my running distance and pace, with the goal of reestablishing a higher fitness baseline. Starting from my 3-4 mile runs, I rebuilt up to 8 mile runs, with my fastest average pace at 8 miles being 8 minutes and 42 seconds. I intended to spend more time working on my pace doing short runs, but I got distracted by the new job.

Relatedly, I’ve long been on the fence about buying an Apple Watch, but decided to buy one to help track my runs, and it’s been a surprisingly delightful piece of hardware. (I specifically bought the Apple Watch Ultra.) If I hadn’t bought it, I would absolutely not know how much I’d run, or the pace I ran at. It’s safe to say that I wouldn’t have even tried to do a pace goal, which would have considerably reduced the impact of my running workouts (e.g. only doing slow, long runs, rather than a mix of slow/long and fast/short) and results.

These are, on an absolute scale, not particularly big achievements. I know many runners who are much faster, go much longer, and are even much faster while going much longer, but it still felt good for me! I have no ambitions to be a competitive racer, I just like to push myself a bit occasionally, and particularly to continue pushing myself as I get older to remember that physical decline is in many ways a sum of choices rather than an inevitability.

Invisalign

In January, I started on Invisalign to improve parts of my bite, along with crowding in my lower front teeth. The plan was that I’d only wear them for four months, but twelve months later I’m still wearing them as the original set of trays didn’t fully work out. I’m scheduled to finish in February now, and am looking forward to no longer timing coffee consumption quite so carefully.

That’s my annual year in review for 2023! If you’re writing one, please send it my way! Love to hear what folks are working on and thinking about over the course of years.

Notes on How Big Things Get Done

How Big Things Get Done by Bent Flyvbjerg and Dan Gardner is a fascinating look at why some megaprojects fail so resoundingly and why others succeed under budget and under schedule. It’s an exploration of planning methods, the role of expertise, the value of benchmarking similar projects, and much more. Not directly software engineering related, but very relevant to the work. Also, just well written.

“Think slow, act fast”

It’s fine for planning to be slow (p17), as long as delivery is fast. Each moment during delivery (the actual execution of a task) is a moment something can go wrong, so condensing that timeline is essential to reduce risk. That is of course, actually condensing the timeline, not just lying about it as discussed in the “Honest Numbers” section below.

Planning phase is preferable to delivery phase because (p18) “the costs of iteration are relatively low” during planning. The example of Pixar is used, where they storyboard films up to eight times before moving into delivery phase. This is a large investment, but it’s a much cheaper investment than making a bad film.

It’s also much easier to avoid “lock in”, which is premature commitment (p42) if you plan extensively before moving to delivery. Once you begin delivery, modifying the plan is quite challenging. To make this point, the authors make an extensive comparison between the building of the Sydney Opera House and the Bilbao Guggenheim Museum. The former changed plans frequently with massive delays, the later delivered ahead of time and under budget. (In part due to significant use of modeling for the Bilbao, discussed in “Pixar Planning” section below.)

Also a good discussion of good planning starting from the end and reasoning backwards. Have a clear sense of why you’re doing something before you try to solve it. There’s a mention of Amazon’s Press Releases (p52)–write a future internal press release as a mechanism to pitch your project–as one mechanism to support reasoning backwards.

“Pixar planning”

The book argues that good planning is “Pixar planning” (p60), where you’re able to iterate quickly and cheaply. The average Pixar film is storyboarded 8 times (p70) to cheaply explore improvements.

This means that good planning requires modeling techniques, including modeling software(p68) as one technique, to support rapid, cheap exploration. The example of Frank Gehry extensively modeling out his buildings in simulation software is used to explore how he was able to deliver the Bilbao Guggenheim Museum so effectively. (A few years ago I played around with creating the systems library for modeling systems thinking problems, which was one of my experiments towards this end.)

Finally, the book also observes that learning happens not only within projects but also across projects (p159). Solar and wind projects are significantly less risky than nuclear projects in part because solar and wind projects deploy hundreds or thousands modular units, rather than one very large unit. Even if some wind turbines are poorly designed or installed, they can learn from than for the next ones. Learning to build nuclear power plants is much harder, since so few of the projects occur.

Dataset of projects

One fascinating idea, mentioned a number of times but not deeply explored, is that the authors have a “database of big projects” (p4) where they track the scope and outcome of various projects. This was initially 259 projects (p111), growing up to 16,000 projects over time.

This is a remarkable resource because it makes it possible to benchmark projects against similar projects, referred to as “reference-class forecasting” (p109), or at least benchmark against something, “reference point” (p111). I’ve been thinking a lot about benchmarking recently, and this is definitely something that furthere my interest. (This book also mentions Superforecasting a handful of times, so I’ve ordered a copy of that to take notes from as well.)

Things can be inexperienced

This book says something I’ve understood for a while but never articulated clearly, which is that things can be inexperienced, such like people can (p86). They use the example of a potato peeler that cuts your fingers when you use it, which you replace with iterations of better potatoe peelers than are less likely to cut your fingers. The final edition is an experienced thing, whose design incorporates significant learning into it.

You could probably write an entire book on just that idea alone. Perhaps combined with the observation that we often lose sight of why things work. Perhaps that book is The Design of Everyday Things.

“Honest Numbers”

I also appreciated the discussion of “honest numbers” (p3), which is really a discussion about dishonest numbers and how they justify many projects. A recurring theme in the book is that many leaders deliberately misinform stakeholders about potential costs in order to build commitment, reach a point of no return, and then acknowledge the full costs.

This is eloquently captured in a quote from Willie Brown (p35):

In the world of civic projects, the first budget is really just a down payment. If people knew the real cost from the start, nothing would ever be approved."

This idea, termed “strategic misrepresentation” (p26), reminds me of a poor joke I sometimes tell, which is that “Vice Presidents never miss their targets, they just move the targets to what they accomplish.” Holding the powerful to account is difficult, even if they are acting in good faith, and when they’re acting in bad faith, then it’s remarkably challenging.

This is an important issue, because often the parties who make the commitment aren’t the ones who are stuck paying it off (p38):

Drapeau got his Olypmics. And although it took more than thirty years for Montreal to pay off the mountain of debt, the onus was on the taxpayers of Montreal and Quebec. Drapeau wasn’t even voted out of office.

Incentives are hard, and harder still when there’s not possibility for accountability, as is often the case for politics.

Altogether, a quick and interesting book. Well worth a read.

That's all for now! Hope to hear your thoughts on Twitter at @lethain!

This email was sent to you why did I get this?    unsubscribe from this list    update subscription preferences Will Larson · 77 Geary St · co Calm 3rd Floor · San Francisco, CA 94108-5723 · USA