Product Collective - The power of product experimentation

The power of product experimentation

Thomas Edison ran experiments and found 10,000 ways not to build a lightbulb. You can invoke your inner Edison and experiment to learn more about what works and doesn’t work with your product, and it probably won’t take 10,000 iterations to figure that out. This week, we shared some resources to help you run effective experiments.

Meanwhile, in product news, the “as a Service” world is moving to BaaS and LaaS, it’s now possible to remove image backgrounds, and you may soon have to remove TikTok from your phone.

In-app experiments in B2B: Navigating challenges in product discovery. Conducting experiments is an essential part of product discovery for product managers. It goes hand-in-hand with user interviews, when your goal is to understand the user’s behavior patterns. However, conducting in-app experiments in B2B products, especially when targeted at enterprise persona, can be quite tricky and challenging. Niraj Upadhyay shares some challenges he’s faced conducting experiments and the lessons he learned.

How to do product concept testing in B2B. The earlier you test something, the sooner you identify potential opportunities for improvements—or errors in judgment that need correcting. The same is true of concept testing for products and services. Sometimes, a small tweak can unlock additional revenue and market share, and other times, a design or positioning flaw will harm its uptake. Chris Wells discusses the steps involved in B2B product concept testing and validation research projects, including some practices they’ve found helpful.

Continues below...

Set yourself up for success by transforming your product strategy into dynamic quarterly OKRs with guidance from expert Tim Herbig. Learn to craft precise strategy statements, select the right metrics, and pinpoint critical success indicators. Join this insightful airfocus webinar. Can't make it? No worries; all registrants receive a recording. ​ REGISTER HERE

This Week’s Video

Product experimentation at scale with Claire Vo. Product people are mostly all in favor of some type of experimentation. But it often feels easier to experiment at the beginning stages of a company or product. But what happens when you have lots of customers? How can you experiment when many of those customers are looking to your company to provide stable, reliable, and trustworthy products? Claire Vo of Optimizely can help answer this question. In this discussion, Claire shares how experimentation is not only beneficial for everyone, but can truly be done at scale.

This video and many others like it are available on our Member Hub. If you don’t already have access to the Member Hub, you can join the community today for free.

Wendy's Experiment

If you get frustrated whenever Uber raises the prices during rush hour – you may not want to stop at Wendy’s once you get in. Recently, the fast-food restaurant chain announced some product experiments it planned to do involving dynamic pricing (read: surge pricing?!). Depending on the time of day, the price of Dave’s Single could be more… or less… depending on the demand. At least, that’s how it all sounded when Wendy’s made their announcement.

More recently, it seems that Wendy's has done a bit of a walk-back on their dynamic pricing plans after the initial announcement caused some backlash. The fast food chain has clarified that they will not raise prices during peak demand hours, contrary to how many interpreted their original "dynamic pricing" statement. Instead, Wendy's now frames it as offering discounts during slower periods, allowing flexibility to promote items and lure customers with deals when traffic is lighter. So, some form of variable pricing will occur, not the dreaded "surge pricing" model that stirs up visions of $20 Baconators during the lunch rush.

It’s an interesting experiment to run. As a product person, I think it makes complete sense. But does the added complexity and potential customer confusion outweigh the benefits of simple night discounts or happy hour deals? There's something to be said for keeping pricing super straightforward, especially in the fast-food world where consumers have certain expectations.

For product leaders, Wendy's experiment will be a fascinating case study to monitor over the coming years. How will they balance new digital capabilities with preserving the familiar conventions that made fast food successful? Are customers hungry for this kind of innovation? (Sorry, I had to).

Keep an eye on your Wendy’s app in the coming months and find out for yourself!

​

Do you have a license for that BaaS? The banking-as-a-service company Griffin Bank got a banking license. Banking licenses are difficult to come by, especially for tech companies - Griffin’s took a year. Now that Griffin has a banking license, it offers a full-stack platform for FinTech companies to offer banking, payments and wealth solutions via automated compliance and an integrated ledger. The company will likely offer banking accounts to businesses rather than consumers. Leadership as a Service, anyone? Generative AI video startup Tavus has developed a “replica API” that allows your company to create a replica of key leadership team members and embed them anywhere. If you’re wondering why you want a copy of your VP of Sales, consider including them in your in-product onboarding or marketing campaigns. They could also be set up to tirelessly ask your product team when they introduce teleportation into the product. Can I have a little less background on that? If you’ve ever been working on a presentation and wish you could take the floral print background out of the picture you liked, you’re in luck. Google Slides now allows you to remove unwanted backgrounds from images you use in your Google Slide presentations. If that sounds like something that you’d find helpful, make sure you have either Gemini Enterprise, Gemini Business add-ons, or Google One AI Premium The clock is tik(tok)ing. TikTok faces an uncertain fate in the U.S. once again. In a 352 to 65 vote Wednesday, the House passed a bill that could lead to a national TikTok ban. Now, the bill’s fate lies in the Senate, but that chamber of Congress is decidedly more mixed on the prospect of a forced sale of the app — or a full-fledged TikTok ban in the U.S.

Prices increase on April 1st!

The New York Product Conference (a Product Collective/INDUSTRY event) returns to the Times Center in Manhattan on April 18th for a packed day of:

• Keynote talks on product leadership, AI, product positioning, and more.
• Interactive working sessions on AI for PMs and Customer Research.
• Small group discussions, networking, an end-of-day reception, and more!

BONUS: Purchase a Plus Pass or Superpass, and get access to a half-day workshop on April 17th (Influencing Without Authority) and/or access to online training programs.

The art of experimentation for product managers. Experimentation is a crucial component of product management. Product managers can test different ideas and approaches to identify the most effective solutions and gain valuable insights into their target audience. Bhavya Singh explores the art of experimentation for product managers, why it is important, and how to do it effectively, and provides real logical examples to illustrate each method.

Top 7 Expert-backed product experimentation tips. Experimentation can be as far-reaching as changing the structure of your organization to optimizing a tiny part of the funnel. Through Matheus Mello’s years of experimentation, he’s observed that optimizing the funnel can yield significant results, especially when playing with a large amount of traffic. While there are ample opportunities to do this in a B2C environment, you might have to get a little more strategic in B2B SaaS marketing environments. With that in mind, Matheus provides seven vital tips and lessons he’s learned in his five years of leading experimentation with various product-led teams.

Resources and news curated by Kent J. McDonald.

​

​

​