CXL - Summary of User Feedback

In our last newsletter, we announced we will be sharing some stories from the last few years at CXL.

Today we share a little bit about how we collect feedback from you and what we do with it. 

How We Collect User Feedback From You

We have multiple inputs throughout the CXL app to collect feedback. At the end of courses and throughout the course itself. We also conduct regular customer interviews and chat with many of you on Intercom.

Thousands of you have given up precious time to give us feedback, good and bad.

Not only does it help us identify and fix hundreds of content issues, bugs, and improvements, but it also feeds into our opportunity database. That is where we codify and then identify the big opportunities for improvement.

We funnel all our user feedback, interviews, and Intercom chats into this. We strip out all the user stories.

As many Elearning companies did, CXL grew quickly during the COVID years.

We experimented, we built MVPs, and we went from a narrow niche to a wider scope of content. This brought with it benefits but many issues of scaling fast. The feedback was clear that this happened.

As a small company with limited resources, we needed to prioritize ruthlessly.

Weighted Opportunity Mapping

(Source: Product talk)

Many of you will be familiar with opportunity maps. We have been experimenting with a weighted opportunity map for the last year to help us use our resources more efficiently. 

We combined components of the scoring from PXL, ResearchXL’s thematic analysis, and the Opportunity mapping stage from continuous Product discovery by Teresa Torres.

Opportunity Scoring Pillars 

(Source: CXL blog)

PXL Framework uses 3 pillars of scoring metrics. Impact, Confidence, and Effort.

For our opportunity weighting we needed to change the angle so we opted for:

• Source - Who gave the feedback, where are they based, what type of user are they, are they internal or external? All these things could have a significant impact on how important the feedback is to your customers and your goals as an organization
• Severity - Very similar to the impact of PXL, but has a component of count to it. How many times is a specific popping up and where is it coming from?
• Core experience - As an Elearning business, our Centre of the World issues are things like being able to find and watch content. Rather than obscure or more specific learning path features or content requests. Marking these out in their dedicated category allows you to prevent some glaring omissions and reduce prioritization mistakes. Something we have seen happen with other scoring and count systems in the past.

One thing we are still working on is how to deal with silence. We have come across several issues that we know exist, the data is there to prove and validate them, and yet we see zero evidence of customer complaints or feedback. 

A word of caution - Weight is only an indicator and not an absolute measure. You have to keep iterating and validating your scoring system to ensure it stands up to scrutiny.

You will make mistakes, but please ensure you have a system in place to self-correct and make your framework more robust.

As a new framework. We validated what we found out, again, with interviews and new user feedback. We ran both internal and external UI/UX audits of our flows to make sure that we were focusing on the right things.

And our work isn’t close to being done

Opportunity Scoring Pillars 

The next step was to start mapping the user stories into themes and sub-topics.

We used a very similar method to thematic analysis from ResearchXL. Pulling in different stories from different sources and organizing them into themes.

(Source: Strategic research for experimentation)

Unsurprisingly, our main themes are to improve the courses and to improve the UX.

As any good opportunity map should, it aligns with our Product Goal tree map.

So far, simple.

These themes then break into multiple subtopics like:

• Requests for certain content types
• New course + Updates
• Improvements to format and Instructors
• Video, Navigation, and discovery improvements
• Self-service improvements

Probably all familiar topics if you are a paying member of ours.

There Is No One-Size-Fits-All

Where things got interesting though was the breakdown of how people wanted our courses to improve.

What was clear as day. High-level, generic content 👎

No one cares for it.

On the flip side, actionable content seems to be a pretty consistent request.

What does actionable mean though? 

Depending on the topic or level of the user they ask for different things. Practical examples, assignments, exercises, step-by-step. It is quite a wide range, and each has its application, pros and cons.

The hardest part to figure out was the Level of detail. We know this very well from producing 2500 playbooks and 10 Minidegrees.

For each playbook step, there were always people who needed more detail. The more detail you gave, the more that step expanded and branched into its own playbook. Till enough people want you to zoom back out.

We kept hearing that Minidegree programs are too long. I don't have time. It is overwhelming. Make it shorter.

You make it shorter and the pendulum swings the other way almost immediately and it lacks detail or depth.

Getting the level of detail and in combination with length right will take some testing.

So, What's Next?

We are running several experiments simultaneously to experiment with this and will follow up with learnings on each as the feedback comes in.

If you are a member with us you probably have been witnessing some of this in action already. If not keep an eye out in the next few weeks.

1. 90-180 min courses to challenge and inspire your actions - For example the AppSumo performance marketing course and incoming course on Bottom of Funnel SEO strategies.
2. Shorter programs of ~10hrs - New Growth Strategy program coming in a few weeks.
3. 5-15min Quick fire actionable lessons on specific topics to keep you up to date - We have pro tips from Fred Pike, Rob Muldoon, Ruben de Boer and others on a whole array of topics to keep your finger on the pulse.
4. Workshops with our instructors on topics such as Attribution

In the next newsletter, we will share more about our UI and UX roadmap.

New Blog: Leveraging Irrationality for Experimentation

Human beings are irrational creatures influenced by context and emotions. 

Understanding the complexity of human behavior can guide your experimentation effort if you create solutions that benefit both the client and your business.

Rubén Nesher García, Head of Experimentation at Bitso, shares how to leverage irrationality for experimentation and how to study human behavior to find new hypotheses, design better solutions, and increase the effectiveness of your communications. 

Read the full blog.

New Course: Landing Page Optimization

Only optimizing your ads, email, or banners isn't enough. 

They need to work together with an optimized landing page.

Figuring out what to optimize can be difficult.

Our Landing Page Optimization course has been updated to teach you how to conduct research to base your optimization efforts on real insights, and the fundamental characteristics, components, and structure of an effective landing page.

Check it out.

New Blog: Data Storytelling and Decision Intelligence

With the rise of data-driven marketing, the ability to understand and interpret data has become essential for everyone. 

Data literacy, data storytelling, and decision intelligence have become skills that can enhance content marketing efforts and drive better business outcomes. 

David Ciommo, Speaker at MADS 2023, is diving into the reasons behind the demand for these skills and the actions you can take to enhance or broaden your skillset.

Read the full blog.

Unlimited Team Plan - Start a 15-Day Free Trial

✔ Unlimited team seats – big team, big savings.

✔ 120+ highly practical and detailed marketing courses.

✔ 10 deep skill-building programs.

✔ Learn from vetted top 1% marketing practitioners.

If you have a team of 10, that’d be only $672/yr per team member. $336/yr per member for a team of 20.

Start a 15-day free trial. 

CXL, 901 S Mo Pac Expy, Suite 150, Barton Oaks Plaza One, Austin TX 78746.

Follow us on LinkedIn.

Unsubscribe from future emails like these.