Engineering strategy notes. @ Irrational Exuberance

Hi folks,

This is the weekly digest for my blog, Irrational Exuberance. Reach out with thoughts on Twitter at @lethain, or reply to this email.

Posts from this week:

- Engineering strategy notes.
- Notes on Technology Strategy Patterns
- Notes on The Software Engineer's Guidebook
- Notes on Tidy First?
- Notes on The Value Flywheel Effect

Engineering strategy notes.

Recently, I am thinking quite a bit about engineering strategy, and as part of that have started re-reading previous resources on the topic, and looking for new things to read while I refine my point of view on what makes for good engineering strategy.

The best introduction to my current theory of engineering strategy is Solving the Engineering Strategy Crisis, which has both written and video versions. You can also reading my other strategy writing via the strategy tag.

Let me know what I’m missing from this page!

HOWTOs

• Writing an engineering strategy is a chapter from The Engineering Executive’s Primer on setting engineering strategy as an executive
• Write five, then synthesize is a chapter from Staff Engineer on driving engineering strategy without executive authority (primarily through documentation)
• Wardley Maps by Simon Wardley explains how to use Wardley Maps to understand and improve strategy

Books

In addition to my own Staff Engineer and The Engineering Executive’s Primer, both of which have chapters on engineering strategy, some interesting books on this topic are:

• Technology Strategy Patterns by Eben Hewitt – a method-focused book on creating and communicating engineering strategy
• The Value Flywheel Effect by Anderson, McCann, and O’Reilly – an introduction to Wardley maps via an exploration of Liberty Mutual’s rationale for serverless
• The Phoenix Project by Kim, Behr, and Spafford – a modern retelling of Goldratt’s The Goal, which shows how to model and resolve problems using constraint optimization. Previously, I would not have considered this a strategy book, but as my opinion on what strategy is evolves (mapping plus guiding policies), I think it demonstrates a useful mapping strategy

Here are books that tackle strategy more generally (e.g. not engineering strategy):

• Good Strategy, Bad Strategy by Richard Rumelt – the most helpful strategy book that I have ever read, because it actually provides a usable definition of strategy. (Rumelt also has a 2023 book, The Crux, which is on my reading list.)
• Thinking in Systems: A Primer by Donella Meadows – a book on systems thinking, which for a long time was my sole tool for mapping things around me. This is not a software engineering book, but provides a lens into a useful mapping mechanism that you can apply to software and software development (Why limiting work-in-progress works is one example of me using systems thinking to model a software system)

Case studies

Every discussion of engineering strategy includes a weary remark about how few strategies are publically document. Acknowledging that concern, some case studies that I’ve found:

• Solving the Engineering Strategy Crisis – is both a blog and a video that includes a short capture of Uber, Stripe and Calm’s engineering strategies
• Magnitudes of exploration documented a public version of Stripe’s Engineering strategy
• The Value Flywheel Effect (linked under “Books” header above) is a good case study of Liberty Mutual’s engineering strategy, and additionally includes case studies for A Cloud Guru, Workgrid, and BBC
• Run less software by Rich Archbold – a fantastic writeup of a cornerstone of Intercom’s engineering strategy
• How Big Technical Changes Happen at Slack by Adams and Rodgers – this is not quite Slack’s engineering strategy, but it has many components of their engineering strategy within it
• The difficult teenage years: Setting tech strategy after a launch by Anna Shipman – a look at FT’s engineering strategy, particularly one that wasn’t really defined until somewhat late in the lifecycle (which is an extremely common occurence, even if we don’t admit it)

A few more resources that are either case study-ish or engineering-ish, so they don’t quite fit in the above list, but are nonetheless relevant reads:

• BoringTechnology.club by Dan McKinley – a guiding principle that many engineering strategies include
• GitLab Strategy – ok, it’s actually the GitLab company strategy. but given they’re a technology company that builds technology for technologists, it’s an interesting read despite being at a slightly higher altitude than an engineering strategy

Blog posts

• Engineering Strategy is a Fractal by Alex Morgadas – an interesting take on the idea of strategy concurrently existing at multiple layers of the organization
• Defining a Tech Strategy by Sarah Taraporewalla – an interesting template and approach to tech strategy. I’m not sure if the author had read Technology Strategy Patterns, but many of the ideas from that book show up here as well, but with different examples and nuance

Engineering principles or values

Many folks view principles or values as the same thing as strategy. I don’t agree with that myself, but often principles and values represent a component of strategy for a given company, and consequently are interesting to look at in this context:

• Monzo’s engineering principles
• uk.gov’s service standard for digital services

Collections of stuff

There are some good resources out that collecting engineering strategy links:

• aleixmorgadas/awesome-engineering-strategy is a surprisingly good collection of Engineering strategy content
• I collected engineering strategy resources written by others into Staff Engineer’s appendix

If there’s something important or interesting that’s missing here – let me know!

Notes on Technology Strategy Patterns

Technology Strategy Patterns by Eben Hewitt is a methods-based approach to engineering strategy, with a particular focus on the methods wielded by McKinsey consultants, software engineering mainstays like Thoughtworks, and philosophy. A valuable read for anyone looking to build their own theory of engineering strategy.

In June, 2019, I bought a copy of Technology Strategy Patterns by Eben Hewiit. A the time, I was trying to argue against a large, proposed migration to Java at Stripe, collecting thoughts that became Reclaim unreasonable software. Skimming through Patterns, I didn’t quite find what I was looking for, and I largely forgot about it for the next few years.

Fast-forwarding to 2023, I’ve been spending more time thinking about engineering strategy, including reading The Value Flywheel Effect for its view on engineering strategy, and I remembered Patterns. Well, I remembered that Patterns existed, and that I hadn’t really read it the last time I picked up a copy, so this time I decided to dig into it a bit deeper.

If I had to quote one section of thise book to capture its core essence, it would be Hewitt talking about how to write slides to explain your strategy:

Execute all the applicable creation patterns of this book…
Collect your output from doing so…
Do a MergeSort of the output…
Now you have a terrific strategy

This reminds me Rumelt’s Good Strategy, Bad Strategy, where a sufficiently good diagnosis leads to a very boring set of guiding policies. If you sufficiently understand any problem, then the solutions for addressng it tend to become self-evident. It also reminds me of a belief Wardley mapping, which is that the process of mapping reveals the structure of your circumstances and how strategy might improve it.

Patterns, like many software books leans heavily on the concept of patterns established in Christopher Alexander’s A Pattern Language, with a particular focus on patterns used by McKinsey consultants, Thoughtworks adjacent technology ideas like running a technology radar, and Hewitt’s personal experience.

If you believe that methods for exploring your situation lead to strategy–which most writers on strategy seem to have shared conviction around–then these methods are the foundation of strategy, and they are the part of the book that resonates most with me.

There are certainly other aspects that resonated as well:

• Explicitly decoupling development of strategy from presentation of strategy, and having a structured approach for how to do different kinds of strategy presentations (strong McKinsey vibe here, in the best possible way, of having clear patterns for communication)
• Broad set of tools for exploring your situation: MECE, logic trees, PESTEL, SWOT, and so many more. These are not the mechanisms I have generally used, which is why I find them so interesting
• Explicit acknowledgement of different contexts (world, industry, corporate and department) and how you have to think about all of these in different ways to think about different altitudes of strategy
• A great summary of an architect’s primary goals:
  1. Contain entropy
  2. Specify the non-functional requirements
  3. Determine trade-offs

Altogether, another very interesting read for anyone thinking about engineering strategy. There are a lot of ideas here, and they’re the best sort of ideas: those that are clearly Hewitt’s that he’s developed over a prolonged career.

Notes on The Software Engineer's Guidebook

The Software Engineer’s Guidebook by Gergely Orosz is a broad reference book for software engineers that will be particularly valuable for new software engineers and those who’ve worked most of their career in a small number of companies. It doesn’t go deep everywhere, but leaves a breadcrumb on most topics you’ll encounter as a software engineer, along with enough detail to guide deeper exploration in other, narrower books.

Gergely Orosz is the author of The Pragmatic Engineer, and almost certainly the current title holder of “widest-reaching writer on software engineering” with ~500,000 subscribers. He recently released The Software Engineer’s Guidebook, which is nominally his third book after The Tech Resume Inside Out and Building Mobile Apps at Scale, but you could also argue that Guidebook is his first fully formed book. The first two were experiments in the book medium, and this is the first book that Gergely entered with a fully formed plan.

Before getting into my full thoughts, let me pause momentarily to acknowledge that Gergely and I are friendly, have colaborated in the past, and I imagine we’ll colaborate in the future. That said, if you’re going to my blog for a vicious takedown of anything, I think you’ll leave disappointed–that’s not what we do here.

Where it shines

The thing that Guidebook does exceptionally well is briefly cover a broad set of topics. There are so many ideas here that a new graduate or engineer who’s only worked at one company would have never encountered. Even five years into my career, there are topics in this book that I simply didn’t understand, particularly where you only really get visibility as a manager. Narrowing that gap for newer engineers is valuable.

Covering so many topics, most of the coverage is shallow, but I think that’s appropriate here. This is a book of breadcrumbs that lead to numerous deeper resources on the particular topics.

Some might argue that this mile-wide, inch-deep approach is a failure because it doesn’t go deeper, but I think that’s misguided. All books struggle with audiences who have already grown past their subject-matter, and it’s not particularly interesting to evaluate book through the lens of audiences who no longer need them.

Positioning

As you enter the world of writing books and publishing, one inescapable constraint is that you need broad topics to support high sales numbers. Guidebook is essentially the perfect book from a positioning perspective, as it:

1. targets early career engineers
2. aims to be an encyclopedic reference to continue using over time

Combining those two ideas, you have a book that can support wide sales, and is likely to remain in folks’ libraries as they advance in their career.

Self-publishing

One interesting aspect of this book is that it’s self-published. This is partially a historical accident, as Gergely had significantly less reach when he initially pitched this book, but also a fully deliberated decision: as it stands today, Gergely could have certainly gotten his book publshed, with essentially no constraints, by almost any publisher.

Gergely makes a living from his writing, and the financials of self-publishing are simply undeniable if you bring your own distribution, which Gergely does. I’m not sure if technology publishers can really solve that gap for established authors with significant distribution.

Publishers have a lot to offer to new authors without access to editors or feedback on structuring an effective book. Publishers have some things to offer for authors without their own distribution mechanism, although I think they have a bit less to offer than first-time authors imagine. For established authors with strong distribution mechanisms who are willing to deal with the operational overhead of publishing a book… there just isn’t that much on offer to offset the financial costs.

Notes on Tidy First?

Tidy First? by Kent Beck captures the spirit of Ousterhout’s A Philosophy of Software Design while also recognizing the inherent tensions of developing software within a team and business. You can also read it in about two hours. Recommended!

A Philosophy of Software Design by John Ousterhout is one of my favorite books on software design. When I heard that Kent Beck had a new book out, Tidy First?, that was deliberately engaging with similar content but a markedly different pedagogy, I knew I had to read it. (It also helped when I realized it’s around one hundred pages and you can read it in a couple hours.)

[Tidy First?* starts with an explanation of Guard Clauses, arguing that instead of:

if (condition):
if (not other condition):
...some code...

You are better off writing:

if (not condition) return
if (other condition):
...some code...

Building from there, Beck builds out an approach to software where you think deliberately about separating and sequencing changes that “tidy” (modify structure) as opposed to changes that modify application logic.

Here’s my best attempt to summarize the book into bullet points:

• There isn’t a single way to do things, there are things that make sense in context, and you know your context
• There are many distinct ways to tidy code, which make code easier to work with: guard clauses, removing dead code, normalizing symmetries, and so on
• Tidying and logic changes are different types of work, and should be done in distinct pull requests
• This speeds up pull request review,and on high-cohesion teams tidying commits shouldn’t require code review at all
• Tidying should be done in small amounts, not large amounts
• Tidying is usually best to do before changing application logic, to the extent that it reduces the cost of making the logical change
• It’s also OK to tidy after your change, later when you have time, or even never (for code that doesn’t change much)
• Coupling is really bad for maintainable code

I really enjoyed this book, and reading it I flipped between three predominant thoughts:

1. I’ve never seen a team that works this way–do any teams work this way?
2. Most of the ways I’ve seen teams work fall into the “never tidy” camp, which is sort of an implicit belief that software can’t get much better except by replacing it entirely. Which is a bit depressing, if you really think about it
3. Wouldn’t it be inspiring to live in a world where your team believes that software can actually improve without replacing it entirely?

Any book that can force you to reconsider a somewhat strongly held belief, particularly after spending a fair amount of time in industry, is doing something right, and Tidy First? is certainly in that category.

Curious to see how many teams end up adopting this approach, and what makes teams likely or unlikely to adopt it. Is this an approach that mostly works for small teams in code that is already modular? In small monoliths? In large monorepos? We’ll see what the industry takes from it over the next few years.

Notes on The Value Flywheel Effect

The Value Flywheel Effect is a worthwhile read. It’s imperfect, but a fascinating look into real-world application of Wardley mapping, and a rare view of a company’s engineering strategy.

I’m currently diving into the topic of engineering strategy, and a sub-topic that I’ve not previously spent much time on is Wardley maps. As I dug into it a bit more, The Value Flywheel Effect by Anderson, McCann, and O’Reilly was recommended as a primer, so I bought a copy and spend some time working through it.

First and foremost, I think this is a very interesting book and worth reading if you’re trying to build a holistic point of view on engineering strategy. I think it’s best consumed as the authors’ explanation of Liberty Mutual’s engineering strategy, which is quite valuable–too few companies are willing to describe their strategies in any detail.

There’s also quite a bit else in this book:

• a comprehensive introduction to Wardley mapping
• the authors’ concept of “The Value Flywheel”, which is a flywheel with four phases
  • Phase 1: Clarity of Purpose
  • Phase 2: Challenge & Landscape
  • Phase 3: Next Best Action
  • Phase 4: Long-Term Value
• a great deal of serverless evangelism
• four case studies of adopting serverless and some elements of The Value Flywheel

This is, from my perspective, where the book runs into a bit of trouble: it tries to do a bit too much, and some of the goals don’t fit together cleanly. For example, you might have taken the book’s content and written three related but distinct books:

1. An introduction to Wardley mapping – I found this valuable, and would have found it even more valuable if I hadn’t spend some time over the past few weeks using other resources to learn about Wardley mapping
2. A explanation and case studies of The Value Flywheel (of both Liberty Mutual itself and the others) – these are obviously sanitized, but I always find companies self-descriptions of their engineering strategies fascinating, even if you have to read between the lines a bit
3. A pitch for serverless – more on this later

The interweaving of all three felt like a stretch at time.

That said, I think the right way to read industry material is to extract the interesting, and there’s a lot of interesting to extract here: a deep, ongoing practice of Wardley mapping, an exploration of companies adopting serverless successfully.

The case studies in particular are interesting, as I still rarely see serverless used in scaled applications. There does appear to be a sweet spot for companies who are migrating some older patterns to serverless–e.g. technologists working hard to explain the value of technology within companies that depend on technology but rarely think of themselves as “technology companies. There’s potentially a bit less obvious value for companies who are already running continuous deployment with containers on a well-maintained Kubernetes cluster or some other sort of Platform-as-a-service.

That's all for now! Hope to hear your thoughts on Twitter at @lethain!

This email was sent to you why did I get this?    unsubscribe from this list    update subscription preferences Will Larson · 77 Geary St · co Calm 3rd Floor · San Francisco, CA 94108-5723 · USA