Brought to you by:

• Pendo—The only all-in-one product experience platform for any type of application
• Explo—Embed customer-facing analytics in your product
• Attio—The powerful, flexible CRM for fast-growing startups

—

Brian Tolkin is the Head of Product at Opendoor. Previously, he was one of the early employees at Uber, where he was instrumental in launching and growing UberPool, UberHop, and UberExpress and started one of the first product operations teams in tech. In our conversation, we dive into:

• How to enable product and ops to work well together

• How to run great product reviews

• How to make good decisions with limited data

• How he uses the jobs-to-be-done framework at Opendoor

• How to stay calm under pressure as a leader

• Wild stories from his time at Uber

• Challenges faced at Opendoor during the pandemic

• Much more

Listen now on Apple, Spotify, and YouTube.

Find the transcript at: https://www.lennysnewsletter.com/p/scaling-uber-and-opendoor-brian-tolkin

Some takeaways:

1. Product and ops can be a powerful combination, if they are set up well. Operations teams are typically great at implementing short-term, unscalable solutions quickly, iterating fast, and uncovering qualitative insights from direct experience with customers. Product teams are great at building tech that provides scalable solutions to well-understood problems. The key to leveraging the strengths of each team is to create tight feedback loops between them. Operations teams should feed insights to the product team that help them pick the highest-leverage opportunities for tech, and to execute on those opportunities based on a nuanced understanding of the customer.

2. To run effective product reviews:

   1. Set two primary goals for product reviews: accountability (informing stakeholders about progress) and improvement (enhancing the product).

   2. Create a constructive atmosphere: Foster a culture where product reviews are seen as collaborative rather than confrontational. Make sure that feedback is intended to improve, not criticize.

   3. Set context early: At the start of the review, outline the context and objectives clearly.

   4. Encourage open dialogue: As a leader, engage by probing, asking questions, and suggesting ideas in a non-imposing manner. Present suggestions as possibilities rather than directives.

3. When planning experiments with low sample sizes, be honest with yourself about what would be required for a conclusive result. Too often, teams run experiments for a long time only to find that the data is inconclusive, when this barrier could have been identified up front. When data is limited, don’t waste time seeking “false precision” but turn to other means of building conviction, like talking to customers, running diff-in-diff techniques, or a more intuitive, gut-driven approach.

4. When A/B testing isn’t practical due to low sample sizes, explore alternative methods to validate hypotheses:

   1. Talk to more customers

   2. Use observational data

   3. Implement diff-in-diff analysis

   4. Segment by geography

   5. Reduce confidence levels (e.g. 80% instead of 95%)

   6. Run long-term holdouts

5. You can be both “intense and chill.” Leaders can achieve this mix by creating an environment where challenging conversations about work are separated from personal interactions. Outside of work discussions, be friendly and show genuine interest in people’s lives. When it’s time to talk about work, make it clear you’re shifting gears to focus objectively on the task at hand.

Where to find Brian Tolkin:

• X: https://x.com/briantolkin

• LinkedIn: https://www.linkedin.com/in/briantolkin/

In this episode, we cover:

(00:00) Brian’s background

(02:14) Career beginnings at Uber

(02:49) Transitioning from product operations to product management

(06:47) Product and operations synergy

(10:00) Surge pricing at Uber

(12:18) Scaling challenges, and stories

(15:47) Opendoor and Covid adaptations

(25:38) Product reviews and Jobs to Be Done

(40:30) The challenges of A/B testing

(42:23) Increasing conviction in solutions

(44:33) Leveraging intuition in product decisions

(47:07) Partnering with Zillow

(52:55) Staying calm under pressure

(56:25) Finding the “kernel of truth” in product management

(01:00:21) Failure corner: Early days of Uber Pool

(01:06:11) Lightning round and final thoughts

Referenced:

• Twitter’s former Head of Product opens up: being fired, meeting Elon, changing stagnant culture, building consumer product, more | Kayvon Beykpour: https://www.lennysnewsletter.com/p/twitters-former-head-of-product-kayvon-beykpour

• Opendoor: https://sell.opendoor.com/

• How to sell your ideas and rise within your company | Casey Winters, Eventbrite: https://www.lennysnewsletter.com/p/how-to-sell-your-ideas-and-rise-within

• Thinking beyond frameworks | Casey Winters (Pinterest, Eventbrite, Airbnb, Tinder, Canva, Reddit, Grubhub): https://www.lennysnewsletter.com/p/thinking-beyond-frameworks-casey

• Zigging vs. zagging: How HubSpot built a $30B company | Dharmesh Shah (co-founder/CTO): https://www.lennysnewsletter.com/p/lessons-from-30-years-of-building

• FlashTags: A Simple Hack for Conveying Context Without Confusion: https://www.onstartups.com/flashtags-a-simple-hack-for-conveying-context-without-confusion

• Jobs to Be Done Theory: https://www.christenseninstitute.org/theory/jobs-to-be-done

• The ultimate guide to JTBD | Bob Moesta (co-creator of the framework): https://www.lennyspodcast.com/the-ultimate-guide-to-jtbd-bob-moesta-co-creator-of-the-framework/

• Zillow: https://www.zillow.com/

• Zillow, Opendoor announce multi-year partnership: https://investor.opendoor.com/news-releases/news-release-details/zillow-opendoor-announce-multi-year-partnership

• Building product at Stripe: craft, metrics, and customer obsession | Jeff Weinstein (Product lead): https://www.lennysnewsletter.com/p/building-product-at-stripe-jeff-weinstein

• Stripe Atlas: https://stripe.com/atlas

• Founders podcast: https://www.founderspodcast.com/

• Uber will deliver ice cream to you today: https://www.dispatch.com/story/lifestyle/food/2016/07/13/uber-will-deliver-ice-cream/24201840007/

• UberKittens: https://www.uber.com/newsroom/uberkittens/

• UberPuppies: https://www.uber.com/blog/uberpuppies-want-to-play/

• Shoe Dog: A Memoir by the Creator of Nike: https://www.amazon.com/Shoe-Dog-Memoir-Creator-NIKE/dp/1471146723

• The Black Swan: The Impact of the Highly Improbable: https://www.amazon.com/Black-Swan-Impact-Improbable-Incerto/dp/1400063515

• The Design of Everyday Things: https://www.amazon.com/dp/0465050654

• Shantaram: https://www.amazon.com/Shantaram-SHANTARAM-Paperback-GregoryDavidRoberts/dp/B00QPVJESC

• Full Swing on Netflix: https://www.netflix.com/title/81483353

• Formula 1: Drive to Survive on Netflix: https://www.netflix.com/title/80204890

• Break Point on Netflix: https://www.netflix.com/title/81569920

• Air on Prime Video: https://www.amazon.com/AIR-Matt-Damon/dp/B0B8Q3JMCG

• Fi smart dog collar: https://tryfi.com/

• Particle: https://particlenews.ai/

• Sara Beykpour on LinkedIn: https://www.linkedin.com/in/sarabeykpour/

• A new-parent gift guide for product managers: https://www.lennysnewsletter.com/p/a-new-parent-gift-guide-for-product

• Jeff Holden on LinkedIn: https://www.linkedin.com/in/jeffholden/

• Travis Kalanick on X: https://x.com/travisk

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

Lenny may be an investor in the companies discussed.