Competitive User Research: Turn Your Competitors Into a Benchmark

When you launch into a saturated market, you have to find out fast whether your value proposition lands and whether users care at all. We bet that first impression mattered most, so we made our key metric the percentage of meaningful interactions during first-time use.

We started at 20%. Which meant 80% of users walked away at first sight, and we had a long list of hypotheses and very little time to work out why.

Why do user tests without context mislead you?

Because a score on its own tells you nothing. This was not our first user test, but every previous round was missing the same ingredient: context. Was 4 out of 5 good? Bad? We might already have hit the jackpot without knowing it, or been far behind and equally unaware.

Absolute numbers from your own product cannot answer that. You need something to measure against, and the fastest benchmark available is a product your users could choose instead of yours.

How do you use a competitor as your benchmark?

Run the same test on both products at once. We designed a competitive user research study: a two-step test where participants were split into two groups, the first using our product and the second using our biggest competitor's. Both groups used their app for a week, recorded themselves during first use and across the week, and filled in an identical questionnaire.

The results were eye-opening, because they gave us two things a solo test never does: our own moments of disappointment, and our competitor's Aha moments. One finding landed hard. We had promised personalized, quality content, and users described what they got as generic and irrelevant. We were not creating value for them, we were annoying them.

The iterations we built off that study redirected the roadmap and moved our key engagement metrics. It also gave us something durable: a cheap, repeatable way to test each new version against a real external reference.

A hand-drawn four-part infographic on competitive user research: a score with no benchmark cannot be judged, so split users across your product and a competitor's, run an identical survey over a week of use, and read the gap between their Aha moments and your Oh-no moments.
Competitive user research in four beats: without a benchmark a score means nothing, so run the same study on a competitor and read the gap.

What is the SUSHI model for competitive user research?

The method worked well enough that we turned it into a repeatable one, which we call SUSHI:

  1. Split and compare. Split users into two segments. Give one your product and the other your biggest competitor's. Use the same questionnaire for both.
  2. Uncover. Find the Aha and the Oh-no moments in the first impression, using questionnaire answers alongside recorded sessions of users narrating their experience.
  3. See trends. Let users return to the product across a week, then measure the same metrics again after seven days.
  4. Hypothesize. Name the factors most likely to improve the first and second impressions.
  5. Iterate. Run the same test against newer versions, always with the benchmark in place so the insight keeps its context.

What does this change about when you can start?

You can start before you have anything to test. You do not need your own MVP to run this, because half the study is your competitor's product. That means you can gather real product insight into what wins and loses users in your category before you write a line of code, and arrive at your first build already knowing where the category disappoints people.

Key takeaways

  • Test your competitors first. You do not have to wait for your MVP. Insight from your biggest competitor is available immediately.
  • Insight needs context. Finding your competitor's Aha moments alongside your own disappointments is what lets you judge your product objectively and locate your real differentiator.
  • Do not iterate without a benchmark. A reference point tells you whether an iteration worked, and it also tells you when to stop iterating and move on.
  • Measure both impressions. First and second impressions differ, and the change between them is its own signal. Track both.

If you are entering a crowded category and cannot tell whether your product is genuinely better or just familiar to you, a benchmarked study answers it faster than another internal debate. That is the kind of research design I set up with teams in discovery workshops, or start with the research templates in resources.