Product Collective - 🆎 Running AB Tests

A/B Testing

On the surface, A/B Testing may look straight forward. Create two options, show those options to a subset of your audience, and see which option performs best. Easy peasy, right? Not so fast. There are several things to consider when you structure your A/B Tests to make sure that you’re getting accurate results upon which you can make decisions. Here are some resources explaining those considerations and also when you use A/B testing in specific situations.

The complete guide to A/B Testing: Expert tips from Google, HubSpot, and more. Although there’s been plenty said about A/B testing in marketing, many people still get it wrong. The result? People make major business decisions based on inaccurate results from an improper test. Most content about A/B testing often over simplifies the technique. Shanelle Mullin provides everything you need to know to get started with different types of A/B testing for ecommerce, explained as plainly as possible.

Via Shopify

Complete guide to A/B Testing design, implementation and pitfalls. A/B testing also called split testing, originated from the randomized control trials in Statistics, is one of the most popular ways for Businesses to test new UX features, new versions of a product, or an algorithm to decide whether your business should launch that new product/feature or not. Tatev Karen Aslanyan prepared this end-to-end guide to describe the things that both technical and non-technical audiences should consider when performing an A/B test. 

Via TK Aslanyan

Continues below...

Our latest episode of Rocketship.FM hit a touchy topic that's causing even more tech wars at the dinner table: when should kids get their first phone?

The force behind this episode? My 10-year-old son, Edison, apparently decided that between making me keep up with him on the pitch for his travel soccer season and proving that he has a better jump shot than me on our backyard basketball court, he needed a new challenge - convincing me to buy him a smartphone.

And let's be clear. This isn't about if my son will get a phone – but when. I’m a realist and know that it’s only a matter of time. It’s clear to me that kids these days can make a much better case of getting a phone much earlier than kids in my generation did. Heck, I didn’t get my first phone until college… and even then, it was for “emergencies only.”

But there are some real concerns – from the potential for cyberbullying, screen addiction, online predators, and even the murky waters of online privacy. And yes, perhaps I watch too much true-crime TV. But these are real issues that I’m worried about.

On the flip side, I can see the merits too. My son has good friends that have moved away and who he wants to stay close with. And with his busy sports schedule, it is good for us to have a way for him to get a hold of us whenever he needs… and for us to know exactly where he’s at. 

There’s just a lot to process. Check out the episode… and let us know what you think if you’re in a similar conundrum. As for what I’m going to do… well… I’ll just keep deflecting his pursuits for as long as I can. 

But… it’s only a matter of time. 😭

The ultimate guide to mobile A/B Testing. There are tons of app A/B testing platforms that help brands quickly see what is driving mobile user engagement, retention, and conversions faster. While running mobile A/B tests may be easier than ever before, it's hard to know where to start and how to incorporate app A/B testing into your product development process. Will Lam provides an in-depth look at A/B Testing for mobile apps. 

Via DevCycle

A/B Testing for search is different. While A/B testing has limitations and shouldn’t be the only analytical method you use, it’s the single most valuable and versatile tool for determining whether the impact of a change to your search product is positive. There’s no single way to perform A/B testing for search. Daniel Tunkelang explains how to choose the type of tests you run and the characteristics of each test. As Daniel points out,  A/B testing search isn’t just a switch that you flip on — it’s a science.

Via Daniel Tunkelang

10 best A/B testing tools for websites and mobile apps in 2023. A/B testing tools allow you to compare two versions of a webpage or mobile app against each other to see which one converts better. A/B tests can also backfire, leading you to incorrect conclusions. Most A/B testing tools help you avoid testing issues. Martin Bolf explains how to verify the accuracy of your testing results and reviews ten A/B testing tools for running website and mobile app experiments.

Via Smartlook