The Engineering executive’s role in hiring. @ Irrational Exuberance

Hi folks,

This is the weekly digest for my blog, Irrational Exuberance. Reach out with thoughts on Twitter at @lethain, or reply to this email.

Posts from this week:

- The Engineering executive’s role in hiring.

The Engineering executive’s role in hiring.

Everyone in an engineering organization contributes to the hiring process. As an engineer, you may have taken pride in being an effective interviewer. As an engineering manager, you may have prioritized becoming a strong closer, convincing candidates to join your team. As a more senior manager, you will have likely shifted focus to training others and spending time with candidates for particularly senior roles.

As an engineering executive, your role in the hiring process will shift once again. You’ll continue to make some key leadership hires yourself, but you’ll spend more and more time designing and debugging your overall interview process.

In this post, we’ll cover:

• Establishing your overall hiring process, including job descriptions, rubrics, hiring loops, and so on
• How many executives focus so much on perfect hiring processes that their processes fail
• Your role in monitoring and debugging the hiring process
• Helping close key candidates across your organization
• Options for leveling candidates appropriately
• How managing headcount is a key part of managing hiring
• How to train your hiring managers to avoid challenges like pursuing non-existent unicorn candidates
• Calibrating hiring inside and outside of your network, along with considering company-internal candidates
• Evaluating building an engineering brand for your company
• Deciding whether to introduce a hiring committee process
• Remembering that the system serves you, not the opposite

After reading through, you’ll have a clear plan for structuring your overall hiring process, as well as your specific role within that new process. You’ve spent much of your career serving a hiring process, and now you need to create a system that serves you.

This is an unedited chapter from O’Reilly’s The Engineering Executive’s Primer.

Establish a hiring process

Unless you’re joining an extremely early-stage company, the engineering organization will already have some sort of hiring process in place. Unless there’s widespread agreement that the current process isn’t working, you should participate in the existing process to get a feel for how it works.

It’s almost always the case that you can adapt the existing process to accomplish your goals rather than starting from scratch, and incorporating what already exists will both simplify retraining the team on the new process and build good-will with the folks who built the previous process.

Regardless of where you start, your final process should include every one of these components:

• Applicant Tracking Systems (ATS): a good ATS is the core mechanism for coordinating your interviewing process. Although many early companies try, running an effective hiring process without an ATS is time intensive with limited return: don’t try it. There are enough reasonable options out there that I won’t recommend any one in particular.

• Interview loop documentation: every role should have a documented interview loop that covers the interviews, the trained interviewers for each interview, and links to each interviews’ definition and rubric..

• Leveling framework: articulate how you level candidates based on their interview performance and prior experience. In particular, describe when you level candidates in your process.

• Interview definition and rubric: define an explicit problem or set of questions to ask for each interview. Then add an explicit rubric for evaluating that candidate’s answers. My experience is that it’s preferable to be very consistent on which questions to ask. For example, using the same programming problem for all candidates’ programming interviews.
  
  A frequent pushback is that candidates will cheat by learning the problem from previous candidates, which is certainly possible. However, I’ve found the risk of cheating is still lower than the risk of poor signal due to solving inconsistent problems. (Furthermore, it’s usually pretty clear which candidates are cheating. Make sure you have additional sections for them to complete if they go fast, and note if their ability to solve those sections degrades in a surprising way.)

• Hiring role definitions: every interviewer, hiring manager, and recruiter will engage with your hiring process using assumptions built on the prior processes they’ve worked in. This will often lead to disagreement between hiring managers and recruiters about who’s responsible for the closing process, who has input on the offer’s compensation details, and so on. The best way to avoid this is being very explicit about who is responsible for what.

• Job description template: you should create a baseline template for job descriptions, with a consistent structure and background on your organization, benefits, and mission.

• Job description library: hiring managers should use the job description template to write a job description for each role they hire. These job descriptions should be aggregated in a shared folder where they can be reused rather than reinvented. This also simplifies keeping descriptions updated as you refine shared components.

• Hiring manager and interviewer training: finally, the last component of an effective hiring process is a clear mechanism for training your interviewers. The most effective process I have seen is having new interviewers shadow several trained interviewers, combined with one reverse-shadow interview where an experienced interviewer shadows (and gives feedback to) the new interviewer.

  There are certainly other approaches to consider, including training materials or classes, but I’ve found that many interviewers simply don’t listen in those training, whereas shadowing and reverse-shadowing is much harder to fake your participation.

If you’re joining a relatively scaled engineering organization, it’s likely that most of these will already exist, and that you can quickly formalize the few undocumented portions. On the other hand, if you’re joining a smaller organization, it’s quite possible that you’ll start from a place where none of these materials exist. In the latter case, I’d aim to introduce one or two components at a time over the course of a year: going too fast will overwhelm the team, but isolating changes will lead to retraining fatigue from the team as their hiring process changes repeatedly.

Pursue effective rather than perfect

The two biggest errors that executives make in designing their hiring processes are not designing a process at all—hopefully addressed by the preceding section—and designing overly heavy processes that make it ineffective to hire. The latter is particularly challenging to notice, because you’ll often believe you are optimizing the process when you’re actually slowing it down.

The three clearest indications that you’ve over-optimized your hiring process are:

• Recruiters hire fewer than five engineers per recruiter per quarter (excluding the scenario where they are constrained by headcount)
• You frequently need new interview loops or new interview questions
• It routinely takes more than two weeks from a candidate’s first interview to you making an offer to that candidate

Each of these indicate a process that’s consuming a lot of energy without generating much impact. Often the cause is an indecisive executive who adds steps to find clearer signals, which generally obscures reality rather than clarifies it. I’ve also seen this caused by well-meaning, structured thinkers who are trying to replace biased human judgment with more structured approaches. Neither of these are inherently bad ideas, and it’s through inspecting the above indicators that you can check whether you’re really improving your process or if it just feels like progress.

As the responsible executive, I recommend you require a high bar for each extension to your hiring process. Even if individually they make a great deal of sense, in aggregate they will quickly make the process too cumbersome to operate. Each specialized interview loop, each additional interview to design and train the team on, each approval step, each movement from an accountable individual to a committee–all of these will improve quality, but often in a way that leads to worse outcomes as the process grows heavy. If the current process works, even if it’s not ideal, push the team to work with it rather than extend it. You should certainly modify the process when it’s wholly broken, or when you can improve the standard path for everyone, but stay wary of specializations, customization, and the bespoke.

Monitoring hiring progress and problems

Once you’ve built the hiring process, your job as an executive is generally to monitor and debug it, rather than serve within it. This is particularly true after Engineering grows past 100 members, at which point you’ll be directly involved in the process for a small fraction of the senior-most hires.

Here are the mechanisms I’ve found effective in monitoring and debugging hiring, in the order that I’d consider rolling them out if I joined a company without much oversight:

• Include Recruiting in your weekly team meeting: your Engineering leadership team should have a weekly team meeting, and I strongly encourage including a tech recruiter in that meeting. Their presence makes it possible to troubleshoot recruiting topics quickly and transparently. Some topics may not be particularly interesting to the recruiters, but that’s true for some members of most standing working meeting.

  In particular, this is by far the easiest place to change hiring priorities without anyone feeling left out of the loop.

• Hiring review meeting: meet once a month with the Engineering recruiting lead, and talk through their priorities, as well as any problems that have come up. Keeping this meeting small, typically just the two of you, means you can troubleshoot difficult issues that may be hard to discuss in your team meeting.

• Visibility into hiring approval: although you should likely not be approving every hire in your organization, it’s extremely valuable to have a single place where you can see all the approvals. Often this is a private chat with Engineering’s hiring managers and recruiters, where each offer is approved.

• Out-of-band compensation approval: this is discussed more below, but similarly to seeing all hiring approval, it’s even more helpful to be an approver on all atypical candidate offers. This gives you visibility into the places where your standard operating process isn’t working for some reason.

• Monthly hiring statistics: have Recruiting report on hiring statistics for each role they’re currently hiring. It’s particularly helpful to understand throughput (hires per recruiter), time to hire, offer rate, and acceptance rate. Those four metrics, cohorted by each role, should be enough for you to identify the next set of questions to dig into.

There are, of course, always more meetings and tools that you can introduce. I’d recommend starting with a couple and going from there. As you’ve probably picked up by now, my experience is generally that you can go faster by making incremental changes than by introducing massive changes, even when your goal is transformation.

Helping close key candidates

Sometimes executives insert themselves as a final interview in the hiring process, even after their organization becomes quite large. Executives who do this tend to swear by it as an essential step, where only they can ensure the quality bar for hires remains exceptionally high. However, it tends to significantly slow down the hiring process, and even executives who believe in the most strongly will eventually scale back on this practice.

However, while it’s unscalable to remain as an interviewer across all loops, it is particularly valuable to remain engaged in helping close senior candidates. As an executive, you should be able to tell Engineering’s story and how it contributes to the larger company story, and why that makes for interesting work. You’re also best placed to address strategic concerns the candidate raises.

The approach I’ve found helpful here is three-fold:

• Letting the recruiting team know that you’re available for sell calls with good candidates.
• Updating the Staff-plus and Engineering Manager hiring loops to offer you as a sell call by default for candidates who ask for it. Many candidates won’t, but some will, and these are the candidates who will shape your company the most quickly after they’re hired.
• Inserting yourself into the selling process for particularly high importance roles. Even if you don’t say something new, often the same message from an executive will carry more weight, and your involvement is a clear signal to the candidate that they’re considered an important hire.

I’ve only found this counterproductive in two scenarios. First, some executives add a sell call as a mandatory part of the hiring process, which often creates more friction than it’s worth. There are candidates who are excited to accept without meeting you, and for them the additional sell call will slow things down, and executives are particularly painful to schedule. Second, there are some executives who are exceptionally bad at selling their organization. A friend once did a sell call with an executive who turned out to be watching online videos during the call, which unsurprisingly did not make them feel like a valued candidate.

Leveling candidates

Within the hiring process, the two most contentious topics tend to be determining compensation details for each candidate, and determining the candidate’s level. You can’t determine appropriate compensation for a candidate without knowing their level, so we’ll start there.

The first question to answer is when you level candidates in your process. The obvious answer is that you level candidates after seeing their interview performance, but there are a few issues with that. Most importantly, you likely want to conduct a different process to evaluate very senior candidates than to evaluate early career candidates. At a minimum, you’d want the interviewers to be different, as it’s relatively rare for a panel of mid-level interviewers to decide a candidate is Staff-plus, and you likely wouldn’t be confident in their evaluation even if they did.

Generally I recommend provisionally leveling candidates before they start the bulk of your interview process. For example, you might provisionally level them after they complete the technical phone screen, allowing you to customize the remainder of their process for that provisional level. You can then finalize the leveling decision as part of deciding whether to make an offer. I recommend relying on a simple provisional leveling heuristic such as a combination of technical phone screen performance and years of prior experience. This is far from perfect, but there’s generally enough signal there to determine the range of plausible levels.

The final leveling decision should be guided by a written leveling framework, which looks at the candidate’s holistic interview performance to determine a level. Part of that framework is handling disagreement around leveling, which is particularly common. The most common approach is that:

• Final leveling decision is made by the hiring manager then escalated for approval.
• Approval is done by the hiring manager’s manager for levels at or below your career level (also known as “terminal level” at some companies, this is Senior Software Engineer at many companies), and approval for more senior levels is done by the Engineering executive.

Some companies, particularly larger ones, rely on a committee rather than individual hiring managers for these decisions. My experience is that committees appear less biased, but generally introduce a bias of their own, and are less efficient than wholly accountable individuals. The counter-balance is that at a certain scale, it’s simply impossible to centralize these decisions without significantly slowing down your hiring process. I recommend introducing committees only after relying on individuals has proven too slow at your rate of hiring.

Determining compensation details

Compensation is a broad topic, which I’ll write about more in my next post, but a quick overview on determining compensation details for your offers. There are two particularly important questions which should be detailed in your hiring role definitions: who calculates the initial offer, and what are the approval steps once an offer has been calculated?

The approach that I’ve found effective is:

• Recruiter calculates the offer and shares it into a private chat channel with the hiring manager.
• Offer approvers are added to the channel to approve decision to make an offer, the candidate’s leveling, and the offer details.
• Offers following standard guidelines, e.g. at or below 1.0 compa-ratio, require approval from hiring manager’s manager, and those outside the guidelines require approval from both the hiring manager’s manager and the Engineering executive.
• Any escalations or modifications to the offer occur within the same private chat, to ensure all relevant parties are aware.

A centralized approach where recruiters follow a structured process to calculate offers has many benefits. First, it facilitates training on the shared process, and retraining on that process as your compensation bands adjust, you experiment with offer strategy and so on. Second, it avoids less effective hiring managers leaning on compensation, such that those hired by worse hiring managers get outsized compensation packages. (Which is surprisingly common, although of course it’s almost always framed as the weak hiring manager pursuing exceptional candidates.) Finally, you can still design a process to break bands if you want to, but with a centralized mechanism to make it easier to both manage costs and drive consistency.

Some managers argue that this approach doesn’t give them enough flexibility to make compelling offers to the best candidates. That’s true, but I’ve consistently found that there’s always another way to close a candidate other than more compensation. Further, outsized compensation packages will always create ongoing problems in your annual compensation process, which will be designed to normalize compensation across individuals with similar performance ratings at a similar level. Your broader perspective as the Engineering executive is necessary to balance these incentives, whereas an individual hiring manager is almost always incentivized to hire even if it creates a long-term mess for the wider organization.

Managing hiring prioritization

The intersection of headcount planning and hiring is discussed in How to plan as an engineering executive, but is worth mentioning here as well. In practice there are two fundamental modes of prioritizing hires:

• In rapidly growing companies, there is so much headcount that you’ll be constrained by recruiter assignment rather than headcount.
• At slower growing companies, headcount is the more likely constraint.

In both cases, you’ll frequently have teams pushing for higher priority for their roles. I’m a believer in forcing leaders to solve within their constraints rather than frequently shifting those constraints, but my preference is just one of many ways to approach these tradeoffs. The most important thing to highlight is that both recruiter assignment and headcount are global constraints that you must control as the Engineering executive.

This control can either be something you do personally, or something you delegate to one individual to do on behalf of the wider organization, but they must be made centrally. Making these decisions centrally doesn’t mean that you have to spend a lot of time on it. The simplest way to sidestep this is to determine the headcount and recruiters for each Engineering sub-organization (roughly, each area corresponding to one of your direct reports) and then allow those sub-organization to optimize within their boundaries and allocations.

The biggest trap to avoid is prioritizing recruiters based on hiring need will often steer all recruiting capacity towards your least effective hiring managers. My learned belief is that slow hiring is almost always an execution issue by the hiring manager or the recruiter, and only infrequently the consequence of limited staffing. The exception is when you’ve opened too many concurrent roles for your current recruiter staffing, which is easy to diagnose by looking at the ratio of recruiters to roles (if you have more than three open roles per recruiter, something is very likely going wrong). If you really want to help, first consider spending time training the individuals involved rather than shifting headcount or recruiter staffing.

Training hiring managers

Earlier, I mentioned shadowing and reverse-shadowing as an effective mechanism to train interviewers. That is a crucial part of an effective hiring process, but there’s a second component of training that’s often ignored: training your hiring managers.

There’s a handful of particularly common hiring problems that are usually due to untrained or inexperienced hiring managers:

• Demanding unrealistic, unicorn, candidates from the recruiting team, such that you never make offers
• Allowing any concern from any interviewer to block a candidate from getting an offer, such that you never make offers
• Pushing for non-standard compensation on every candidate rather than learning to close candidates within the standard compensation bands
• Being indecisive on candidates, asking for additional interviews until the candidate withdraws from the process
• Refusing to talk to candidates early in the process and then blaming recruiting team that senior candidates aren’t interested in finishing the process

If you identify one of these, then I do recommend running focused trainings for your hiring managers on the specific topic that’s coming up. These are all topics that I’ve devoted a session of my Engineering Managers Monthly meeting to, talking through examples of why it’s problematic, why it’s not a sign of strong hiring, explaining what reasonable pass-through rates look like for a healthy hiring loop, and recommending strategies for overcoming the issue.

Once you’ve done a training session, you and the recruiting team should point out the issue to individuals who are running into it, and hold them accountable for fixing it. Folks making these mistakes will often have conviction that they’re doing the right thing, but don’t get swayed by their conviction. Effective hiring processes hire candidates. Hiring managers are accountable for their hiring process. Any argument suggesting one of these is false is a flawed argument.

Hiring internally and within your network

When I worked at Yahoo!, our team needed another engineering manager. We didn’t run a hiring process, or even do interviews. Instead, our Director brought on a colleague he’d worked with before. That new manager soon decided he needed a tech lead on his team. We didn’t run a hiring process, do interviews, or consider candidates on the existing team. Instead, our new manager brought over one of his previous colleagues. A third previous colleague reached out to our Director, and without a single interview we’d soon hired a new Chief Architect who would ultimately never write or read a technical specification about our product, nor contribute a single line of code.

One of my teammates–one who had joined the team through the more traditional route of interviewing–described this pattern as the flying wedge, and it’s emblematic of the worst sort of network-hiring. Hiring exclusively from your network will convince your existing team that they and their networks aren’t wanted for important roles at your company.

A similar, somewhat common, scenario is one where your company exclusively fills important roles with external hires. Each individual external hire may make sense, but in aggregate the pattern of prioritizing external hires will encourage your team to seek career advancement elsewhere, draining your organization of context and continuity.

When it comes to internally or externally hiring and hiring within or without your network, the ideal path is always moderation. Hire some internally, some externally, some within and some without. Too much of any path will either isolate your culture from valuable opportunities to evolve, consolidate it onto the culture that worked at a former employer, or prevent it from coalescing to begin with.

In my experience, almost everyone agrees with the above statement, but quite a few don’t follow its advice. As I’ve dug into that, it’s generally because of a missing hiring skill:

• Inability to hire outside their network (or within it)
• Inability to fairly evaluate internal candidate (or external candidate)

Periodically look at the number of internal versus external hires for senior roles within your organization, and dig into areas where there are exclusively hires of one sort. If you find a lopsided pocket of your organization, talk with the relevant leader and push them to make one hire of the sort they’re currently ignoring. Even one will force them to acknowledge the skill gap, and start the process of fixing the imbalance.

If the person with a significant imbalance is you, then take it seriously! Don’t hide from it by justifying the unbalance with philosophical or intellectual rationales, and instead push yourself to make one hire of the other sort. Particularly for new executives, I often find there’s an underlying belief that they cannot close strong external candidates, and disproving that belief is an important part of your personal growth.

Building an Engineering brand

The details of building an engineering brand are discussed in Building personal and organizational prestige, which I’ll avoid repeating here in full. Instead, I’ll briefly repeat its conclusion regarding building Engineering brands in particular:

• Some companies do a tremendous amount of investment into their Engineering brand, those companies will generally tell you that their branding efforts are a fundamental part of their success
• However, most companies, including some very successful ones, do very little Engineering branding, and rarely find themselves stymied by its absence. This doesn’t mean you shouldn’t do any, but rather than you shouldn’t view it as a fundamental necessity
• If you choose to invest into Engineering branding, a small investment can capture most of the upside. The biggest exception is for companies who sell to engineers in addition to hiring them, where you may prioritize a larger brand effort from a lead generation perspective

In general, if you’re already finding enough top-of-funnel candidates for your hiring process, don’t spend more time here unless you can connect that time to another business objective, or have internal folks who find this work energizing enough to take it on as a side project.

Should you introduce a hiring committee?

Many companies introduce centralized (Engineering-scoped) or semi-centralized (Product Engineering or Infrastructure Engineering-scoped) hiring committees as part of maintaining a consistent hiring process. I’ve seen this happen frequently enough in Silicon Valley companies that some executives have come to believe that hiring committees are a natural or ideal landing spot.

Hiring committees are a useful tool, but I’d caution against introducing them as the obvious solution. They’re useful, but come with their own problems.

I generally dislike committees as they introduce ambiguity in who should be held accountable for outcomes. In this case, they also mean that hiring decisions are made further from the particular team, which often degrades individual decisions. These committees are also vulnerable to misaligned members. I was once in a hiring committee where a new member joined who relied very heavily on the universities that candidates attended, even when we clarified to the member that we didn’t hire that way, they refused to change and our Engineering executive was unwilling to hold them accountable to our hiring practices.

On the positive side, they are also a great mechanism for training hiring managers’ judgment on what makes a good candidate. They also introduce more consistent hiring practices across an inconsistent organization, solving a similar problem as Amazon’s Bar Raiser program. Committees are slower than a responsive hiring manager, but faster than a disengaged or very busy hiring manager.

Remember that the system exists to support you

If you came up as a rules-minded leader, you can almost certainly think of examples where your executive responsible for designing the hiring process also ignored that process to accomplish an immediate goal. Personally, I was most annoyed by executives who steamrolled the process to hire former colleagues who performed poorly in our interview process. Each time I’d complain to colleagues, “Why did we build this comprehensive hiring process if we don’t even trust its decisions?”

As is often the case, as I switched into the role of the executive responsible for Engineering hiring, I began to appreciate why perfectly following the process was difficult. When I vowed to loyally follow the hiring bands, sometimes I’d find peer executives paying far outside the bands, implicitly penalizing hires in Engineering. When I endeavored to respect each negative hiring review, sometimes I’d encounter interviewers who refused to use the stated rubrics. When I hired for a brand new role, I’d sometimes find interviewers who interpreted the role’s requirements very differently, even if I pulled together materials explaining the role.

Each of those challenges can be solved over time with better training, but as an executive you rarely control the timeline you’re working in. Sometimes your problem is urgent today. In those scenarios, the question to answer is sometimes whether the company will be better off if you solve the underlying problem (e.g. missing a leader for a key role) or if you respect the process (e.g. don’t break the rules you created). You should try to solve your problem within the process you’ve designed, but don’t get so blinded by your process that you think the process is always more important than your problem at hand. Sometimes the process is clearly less important than the current problem.

That doesn’t mean you should always ignore your process. If your interview process indicates a candidate has gaps, there is usually a valuable signal when our hiring processes decline a candidate. Even if we’re confident the negative signals are wrong, it still undermines a hired individual when their new colleagues know they performed poorly in the hiring process but were nonetheless hired. There is a cost to defying your process, just as there is a cost to following it, and as an executive you need to make that tradeoff deliberately.

Summary

This post has covered your role as an executive in your organization’s hiring, the components you need to build for an effective hiring process, and provided concrete recommendations for navigating the many challenges that you’re likely to run into while operating the hiring process. There are an infinite number of questions to dig into, but this coverage will give you enough to get started, build a system that supports your goals, and start evolving it into something exceptionally useful.

That's all for now! Hope to hear your thoughts on Twitter at @lethain!

This email was sent to you why did I get this?    unsubscribe from this list    update subscription preferences Will Larson · 77 Geary St · co Calm 3rd Floor · San Francisco, CA 94108-5723 · USA