Stop Tracking NPS (Net Promotion Score)

Good research is always rooted in the past, not in the future. Net Promotion Score has many flaws: in how it asks users to predict their future behavior, the Likert scale and the statistically unreliable score calculation. There are better alternatives.

1. How Net Promoter Score (NPS) Works #

The main assumption behind NPS is that business success depends on positive word of mouth. NPS is a number in the range of -100% to 100%. We ask users to answer a question "How likely are you to recommend [X] to a friend or a colleague".

Responders are grouped into detractors (0-6), passives (7-8) and promoters (9-10). NPS score = %Promoters – %Detractors. The higher, the better.

2. The Science Behind NPS Is Wacky #

NPS doesn't use the mean average of the numbers received. It raises red flags: NPS leaves a lot of survey data out, it has weak statistical properties and it’s more susceptible to random noise. Also, gaming the values is relatively easy.

3. You Need Large Sample Sizes #

To get to statistical significance, you need to gather feedback literally from thousands of users. Because NPS survey response rate is 4–7%, you need to send it to hundreds of thousands of customers. It’s unpractical and makes NPS mostly irrelevant for experimentation.

4. NPS Is Fragile And Hides UX Success #

Even if the product team has been moving mountains to improve customer experience, NPS won’t accurately reflect it. Small incremental improvements should bring incremental score increases, but NPS doesn’t work that way. It has very high variance and very abrupt jumps in scores.

5. NPS Captures Only Most Recent Activity #

Customer loyalty is a long play. It’s about how someone behaves over a long period of time, yet NPS is skewed to the most recent user’s experience. As Jared Spool suggests, “the best research questions are about past behavior, not future behavior.”

6. We Can’t Reduce UX To A Single Number #

We need to combine qualitative and quantitative insights to capture the quality of UX. We can use top tasks methodology and design KPIs to measure individual interactions with our products, sites and the brand.

7. Choose Metrics That Matter For Your Product #

Itamar Gilad suggests that the core mission of every company is to deliver value and to capture value back. We need to define both. In his article, Itamar reviews the North Star Metric (customer value) and Top KPI (business value).

Further reading #

• NPS Considered Harmful, by Jared Spool

• Net Promoter Score – Helpful or Harmful? + Part 2 by Itamar Gilad

• Design KPIs and UX Metrics, by yours truly