Irrational Exuberance - Navigators @ Irrational Exuberance

Hi folks,

This is the weekly digest for my blog, Irrational Exuberance. Reach out with thoughts on Twitter at @lethain, or reply to this email.

Posts from this week:

- Navigators
- Notes on The Crux

Navigators

In Staff Engineer’s chapter on Managing Technical Quality, one of the very last suggestions is creating a centralized process to curate technical changes:

Curate technology change using architecture reviews, investment strategies, and a structured process for adopting new tools. Most misalignment comes from missing context, and these are the organizational leverage points to inject context into decision-making. Many organizations start here, but it’s the last box of tools that I recommend opening. How can you provide consistent architecture reviews without an articulated vision? Why tell folks your strategy after they’ve designed something rather than in their onboarding process?

As the quote mentions, this was the last suggestion, but not because it’s the ultimate option. Instead, it’s because I find that centralized groups–typically called the Architecture Team or something along those lines–tend to perform poorly in practice. Sometimes they’re inefficient, sometimes they make bad choices, sometimes they just don’t do much at all. (I documented some of the most common challenges these architecture teams face in Scaling Consistency.)

After I joined Carta earlier this year, we spent a while exploring areas where we could make it easier for engineers to do highly productive work. One topic that came up was technical decision making within engineering:

• Many engineers disagreed with decisions being made elsewhere in the organization, particularly feeling that decisions were made inconsistently
• It was open to interpretation who was accountable for technical decisions in any given part of the business (people roughly agreed, but if you asked four people who was responsible for making decisions regarding Something Important, you’d generally get at least two names)
• When two teams disagreed on a technical decision, it was unclear how to resolve the issue short of escalating it to the CTO

The standard playbook here is to first document best practices, hope that’s sufficient, and then reluctantly establish an Architecture Team to manage escalations. At a company like Carta, with ~400 engineers, it’s further true that aligning on best practices is an escalation-laden exercise. That’s because best practices are context dependent, and there is no universal context in an organization of that size. Best practices are a balance between global and local optimization, and some of the decided upon best practices will certainly be worse locally for some teams.

That meant that we either needed to establish working groups for deciding each of the areas of best practice, or decide on an Architecture Team to serve as the “standing working group” for the full set of practices we wanted to codify. Carta has a large bench of very talented engineers, so we certainly could have established an Architecture Team.

There was, however, one little problem: me. I rather dislike Architecture Teams.

It’s not that Architecture teams can’t be good, I’ve worked with a number of very good ones, but even the best are slowed by consensus, and suffer the downsides of consensus-driven decision making (particularly, optimizing for acceptable decisions rather than best decisions). I’ve also found that consensus-driven groups are slow to improve, even with direct coaching, because it’s generally difficult to figure out who to coach, and accountability can be a bit amorphous.

So, instead we went with a pattern that I’ve experimented with a few times: Navigators. The core structure of Navigators are along these lines:

• Each major area of the business has one Navigator (we started with about ten). That Navigator is an active contributor to the software written and operated within that area
• There is exactly one Navigator for a given area; Navigators do not overlap
• Each Navigator is wholly accountable for the technical decisions made in their area. More than merely being accountable, they can make decisions. This includes interpreting organizational strategy to apply it correctly within their context
• In practice, most real issues are an intersection of technical, prioritization and people constraints. Navigators are responsible for aligning with the people leadership in their area when making technical decisions
• Each Navigator is wholly accountable to the CTO in their decision making and interpretation of organizational strategy
• Navigators are the escalation point for technical consideration within their area, and a pair of Navigators is the escalation point for cross-area issues
• If a pair of Navigators are unable to resolve a cross-area concern, or a Navigator and people-manager unable to resolve a technical-social concern, then the CTO is the escalation point

We’ve been running in this mode for about six months, and I must admit that I think it’s working quite well. It’s very powerful to have a clearly accountable party that everyone agrees is accountable. There have been some messy conversations, but we know who is responsible for what, and we’ve been able to sit down and have those conversations. Afterwards, we’ve been able to improve, and do better for the next set of conversations. (I’ve rarely seen Architecture Teams improve quickly, but I have seen Navigators improve quickly.)

The Navigator pattern is so obvious, and in my experience so much better than the alternatives, that it’s worth asking ourselves why we don’t see it very often. My short answer is that I think the industry continually underestimates what senior engineers can do if we give them the necessary context and remit to do it. It’s considered a radical act to trust your engineers to make truly important decisions, but it doesn’t need to be that way. As long as we’re willing to hold engineers accountable for carrying important roles, then trusting them is no more radical than trusting anyone else in your organization: there’s some risk, but it’s a risk well worth taking.

Notes on The Crux

The Crux by Richard Rumelt is a fantastic follow on to his Good Strategy, Bad Strategy, providing many of the same core ideas but in a more readable format, and a clearer target to take down: the incoherent outputs of process and goal-driven strategy.

Recently, I’ve been looking for more strategy books to read, and folks pointed out that I’d missed a new book from Richard Rumelt, The Crux. No book has influenced my thinking about strategy more than Rumelt’s previous work, Good Strategy, Bad Strategy, so I felt obligated to pop this one to the top of my reading queue.

The short summary here is that I think The Crux is quite good, and that it does a better job of concisely landing the problem areas to avoid than Good Strategy, Bad Strategy. Conversely, I think that the later’s focus on structured exploration of problems is a bit more useful for someone who is trying to deploy strategy in their own environment. If you had to pick just one, I’d still recommend Good Strategy, Bad Strategy first, but I certainly recommend reading both.

With that said, a brief exploration of some of the core ideas in the book. Starting with Rumelt’s definition of the “crux” (p4):

I began to use the term crux to denote the outcome of a three-part strategic skill. The first part is judgment about which issues are truly important and which are secondary. The second part is judgment about the difficulties of dealing with these issues. And the third part is the ability to focus, to avoid spreading resources too thinly, not trying to do everything at once.

This definition captures much of what resonated with his earlier focus on diagnosis, and sharpens it a bit further.

Rumelts summarizes his goals for the book as (p11):

I explore four themes in the pages that follow. First, the best way to deal with strategic issues is by squarely facing the challenge… Second, understand the sources of power and leverage that are relevant to your situation… Third, avoid the bright, shiny distractions that abound… Fourth, there are multiple pitfalls when executives work in a group, or workshop, to formulate strategy…

We emphasizes several recurring reasons why strategy-work tends to go wrong and produce low value results, the most prominent and recurring theme is:

Don’t start with goals–start by understanding the challenge and finding its crux.

There’s another theme that comes up in a variety of ways, including a dedicated chapter:

Don’t Confuse Current Financial Results with Strategy

Both of these resonate quite a bit. The first feels right but somewhat asbtractly: I’ve seen so many clueless strategy statements that don’t solve any problem I have. The second hits very close to home: the longer that I’m an executive, the more I wonder if I’m balancing properly between the financial, business, and engineering perspectives.

The financial perspective is very reassuring, because the numbers always add up, which isn’t true on the business or engineering sides of the house. Conversely, the financial lens provides guardrails but it doesn’t actually tell us what to do, and can never convicingly break an impasse preventing us from reaching a compelling future.

Challenge-based vs constraint-based strategy

As I read through the book, I thought more and more about two related but subtly different ideas: challenge-based strategy as described by Rumelt, and constraint-based strategy which is closer to my opinion of what defines engineering strategy.

Exploring the difference a bit:

• Challenge-based strategy is reasoning forward from your current problems to identify actions that will address those challenges
• Constraint-based strategy is defining guiding policies that help everyone within your organization to understand how to craft appropriate actions of their own that are in alignment with the other decisions being made concurrently by other individuals in the same organization

This highlights a key difference between what Rumelt’s most interested in–how do businesses make strategic decisions–and what I’m interested in–how do engineering organizations position themselves to be strategic within a wider corporate context. More to explore here, but an interesting learning for me.

Deduction versus design

Rumelt quotes Gary Hamel, describing an awkward reality of discussing strategy (p17):

The dirty little secret of the strategy industry is that it doesn’t have any theory of strategy creation.

This is, I think, the biggest issue I have found reading work on strategy, which he connects to a larger pedagolical issue within modern education that eskews the study of the exact things that folks in industry need the most (p36):

My own life experience supports Simon’s comments about the replacement of design with deduction in professional schools. For the academics who currently populate top professional schools, design is a bit like shop class, akin to automobile repair or welding, and residing at a far remove from respectable activities like the mathematical modeling of stochastic processes and the statistical analysis of selection bias.

Viewing most industry work as beneath academic interest very much is something I’ve encountered in management and leadership, somewhat to my confusion. Although I find this less directly applicable to software development, where many of the issues at hand can successfully be rendered into mathematics, but not all of them. There’s a reason we have highly sophisticated typing theory paired with largely ad-hoc bags of practices labeled as “agile” or whatnot in the industry, and it’s in part because the former attracts academic attention and the later does not.

Strategy as creation

Rumelt talks about strategy as a sort of design, and of design as an act of creation (p32):

The result is a design rather than a choice. It is a creation embodying purpose. I call it a “creation” because it is nonobvious to most others, the prodcut of insight and judgment ratehr than an algorithm.

This connects back to the Gary Hamel quote regarding the void that exists where ideally we’d have a clear set of steps for creating strategy. I need to ponder this one a bit more as well, but something that resonates with me is the idea that creation is not well understood, and also that creation “in committee” is exceptionally difficult.

I’ve always found that strategy is best driven by a single mind, with all relevant context loaded into that mind, as opposed to being spread across a diffuse group, which I think is true for all truly creative activities. Art evolves through discussion (literal and figurative) among many artists, but the creation of each individual piece of art is created by a single individual operating from a place of focus.

Like I started with, I think this is a worthwhile read, and the biggest idea that stood out for me is the tension between executives needing to prioritize working the Crux with the entirety of managing the team and business around them. There’s a real art to that balance, and one that many executives struggle with.

More generally, this book has very much helped me refine my thesis for writing a book on engineering strategy, which I’m not totally confident I’ll actually write, but I’m coming to have a decent bit of an outline after my recent binge of strategy reading and the tangential work in the writing of my third book.

That's all for now! Hope to hear your thoughts on Twitter at @lethain!

This email was sent to you why did I get this?    unsubscribe from this list    update subscription preferences Will Larson · 77 Geary St · co Calm 3rd Floor · San Francisco, CA 94108-5723 · USA