Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor)
Brought to you by Sanity—The most customizable content layer to power your growth engine | Hex—Helping teams ask and answer data questions by working together | Eppo—Run reliable, impactful experiments — Ramesh Johari is a professor at Stanford University focusing on data science methods and practice, as well as the design and operation of online markets and platforms. Beyond academia, Ramesh has advised some incredible startups, including Airbnb, Uber, Bumble, and Stitch Fix. Today we discuss: Listen now on Apple, Spotify, Overcast, and YouTube. Find the transcript for this episode and all past episodes at: https://www.lennyspodcast.com/episodes/. Today’s transcript will be live by 8 a.m. PT. Where to find Ramesh Johari: • LinkedIn: https://www.linkedin.com/in/rameshjohari/ • Website: https://web.stanford.edu/~rjohari/ • X: https://twitter.com/rameshjohari Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ In this episode, we cover: (00:00) Ramesh’s background (04:31) A brief overview of what a marketplace is (08:10) The role of data science in marketplaces (11:21) Common flaws of marketplaces (16:43) Why every founder is a marketplace founder (20:26) How Substack increased value to creators by driving demand (20:58) An example of overcommitting at eBay (22:24) An easy litmus test for marketplaces (25:52) Thoughts on employees vs. contractors (28:02) How to leverage data scientists to improve your marketplace (34:10) Correlation vs. causation (35:27) Decisions that should be made using data (39:29) Ramesh’s philosophy on experimentation (41:06) How to find a balance between running experiments and finding new opportunities (44:11) Badging in marketplaces (46:04) The “superhost” badge at Airbnb (49:59) How marketplaces are like a game of Whac-A-Mole (52:41) How to shift an organization’s focus from impact to learning (55:43) Frequentist vs. Bayesian A/B testing (57:50) The idea that learning is costly (1:01:55) The basics of rating systems (1:04:41) The problem with averaging (1:07:14) Double-blind reviews at Airbnb (1:08:55) How large language models are affecting data science (1:11:27) Lightning round Referenced: • Riley Newman on LinkedIn: https://www.linkedin.com/in/rileynewman/ • Upwork (formerly Odesk): https://www.upwork.com/ • Ancient Agora: https://en.wikipedia.org/wiki/Ancient_Agora_of_Athens • Trajan’s Market: https://en.wikipedia.org/wiki/Trajan%27s_Market • Kayak: https://www.kayak.com/ • UrbanSitter: https://www.urbansitter.com/ • Thumbtack: https://www.thumbtack.com/ • Substack: https://substack.com/ • Ebay: https://www.ebay.com/ • Coase: “The Nature of the Firm”: https://en.wikipedia.org/wiki/The_Nature_of_the_Firm • Stitch Fix: https://www.stitchfix.com/ • A/B Testing with Fat Tails: https://www.journals.uchicago.edu/doi/abs/10.1086/710607 • The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon): https://www.lennyspodcast.com/the-ultimate-guide-to-ab-testing-ronny-kohavi-airbnb-microsoft-amazon/ • Servaes Tholen on LinkedIn: https://www.linkedin.com/in/servaestholen/ • Bayesian A/B Testing: A More Calculated Approach to an A/B Test: https://blog.hubspot.com/marketing/bayesian-ab-testing • Designing Informative Rating Systems: Evidence from an Online Labor Market: https://arxiv.org/abs/1810.13028 • Reputation and Feedback Systems in Online Platform Markets: https://faculty.haas.berkeley.edu/stadelis/Annual_Review_Tadelis.pdf • How to Lie with Statistics: https://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/0393310728 • David Freedman’s books on Amazon: https://www.amazon.com/stores/David-Freedman/author/B001IGLSGA • Four Thousand Weeks: Time Management for Mortals: https://www.amazon.com/Four-Thousand-Weeks-Management-Mortals/dp/0374159122 • The Alpinist on Prime Video: https://www.amazon.com/Alpinist-Peter-Mortimer/dp/B09KYDWVVC • Only Murders in the Building on Hulu: https://www.hulu.com/series/only-murders-in-the-building-ef31c7e1-cd0f-4e07-848d-1cbfedb50ddf Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. Lenny may be an investor in the companies discussed. Invite your friends and earn rewardsIf you enjoy Lenny's Newsletter, share it with your friends and earn rewards when they subscribe. |
Older messages
Introducing DRICE: A modern prioritization framework
Tuesday, November 7, 2023
Boost your experiment win rate and help your team drive more impact
Mastering product strategy and growing as a PM | Maggie Crowley (Toast, Drift, Tripadvisor)
Sunday, November 5, 2023
Listen now (83 mins) | Brought to you by Productroadmap.ai—AI to connect your roadmaps to revenue | Composer—the AI-powered trading platform | Eppo—Run reliable, impactful experiments — Maggie Crowley
Unpacking Amazon’s unique ways of working | Bill Carr (author of Working Backwards)
Thursday, November 2, 2023
Listen now (93 mins) | Brought to you by AssemblyAI—Production-ready AI models to transcribe and understand speech | Coda—Meet the evolution of docs | Wix Studio—The web creation platform built for
Lessons learned from a startup that didn’t make it
Tuesday, October 31, 2023
What not to do when building a B2B startup
Reflections on a movement | Eric Ries (creator of the Lean Startup methodology)
Sunday, October 29, 2023
Listen now (134 mins) | Brought to you by Sanity—The most customizable content layer to power your growth engine | Jira Product Discovery—Atlassian's new prioritization and roadmapping tool built
You Might Also Like
SaaSHub Weekly - Sep 19
Thursday, September 19, 2024
SaaSHub Weekly - Sep 19 Featured and useful products Schedul Threads logo Schedul Threads Boost your Threads following, reach monetization status fast and automate your Threads content publishing for
95 new Shopify apps for you 🌟
Thursday, September 19, 2024
New Shopify apps hand-picked for you 🙌 Week 37 Sep 9, 2024 - Sep 16, 2024 New Shopify apps hand-picked for you 🙌 What's New at Shopify? 🌱 New granular staff permissions for Gift cards Improvement ⸱
[SaaS Club] How firing customers built a 7-figure SaaS
Thursday, September 19, 2024
The SaaS Club Newsletter Hey Reader Here's a quick round up of what's been going on at SaaS Club: In this week's newsletter: 🎧 Discover how firing customers fueled SaaS growth 📈 Turn
The MrBeast Guide
Thursday, September 19, 2024
Today's newsletter is brought to you by LCA. We help companies build their future. From new products to websites, apps, and AI solutions. We work with businesses doing $10M-$10B in revenue. We
Writing Software for Robots
Thursday, September 19, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Writing Software for Robots In a few years, most feature flags
KoppieOS, ScrapX, Notch, Magic Inspector, and Entyx
Thursday, September 19, 2024
KoppieOS, where Ai meets Personality. BetaList BetaList Daily Magic Inspector Automate browser testing in natural language KoppieOS KoppieOS, where Ai meets Personality. ScrapX Monitor your competitors
“I’m waiting for the right time”
Thursday, September 19, 2024
Read time: 1 min. 03 sec. "Hey Pat, can I talk to you for a second?" My buddy pulled me aside at a wedding last weekend. I already knew what he wanted to ask... "I really want to do what
Mastering Portfolio Construction
Thursday, September 19, 2024
How to build a strategy that maximizes returns. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
join me: Big Tech in Fintech
Thursday, September 19, 2024
thought you would be interested Hi there, Laura here, Principal Analyst at CB Insights. Thought you would be interested in this new briefing on big tech's activity in financial services. In under
Today's featured projects in 10 words
Thursday, September 19, 2024
Today's projects: Code Diagram • PhotoFairy • Triple Whale • Revnue • Modelbit • VideoTrim.app 10words Discover new apps and startups in 10 words or less Code Diagram: Visual note taker for VSCode