How do I interpret preference scores in claims tests?

Preference scores used in claims tests are essentially the same as partworth utilities in conjoint or MaxDiff studies. They measure how much customers liked each claim or product variant. Items that are strongly preferred by customers are assigned higher scores, items that perform poorly (in comparison to other items) are assigned lower scores. In this article, we deep-dive into how preference scores work.

Identifying a significant difference between claims

The “Summary of preferences and diagnostics for each claim” module displays your claim’s score and ranking, and it’s here that you can determine if there is a significant difference between any two of your claims. We use 90% bootstrap confidence intervals across all significance checks on the system.

In a Claims Test report, to check the significance of the difference between each of your claims:

• Select two values under the row labelled “Score”.

• Refer to the text displayed in the blue box which appears in your left-hand menu.

You can also view the confidence interval of each value by hovering your cursor over it.

Comparing a claim across different segments

In a claims test, scores should be compared within each segment (because each score is relative to the other scores within each segment). With this in mind, you should only compare scores across segments at a high level, as the difference in sample size alters the confidence interval across segments.

However, it is possible to compare how claims are ranked differently across segments which can be achieved through the crosstab feature.