A/B Test

A/B Test is an experimental method for comparing the performance of two stimuli. This experimental method exposes respondents to each stimuli individually to allow for focussed comparison between the two stimuli. These stimuli may be anything from claims, to packaging, to two different products. A/B testing can be used for:

  • Determining which ad is more effective at increasing brand awareness
  • Finding which packaging will increase purchase intent
  • Deciding between two claims

Main outputs of A/B tests

Table of outputs for each question:

Table of outputs for each question:

Summary metrics for each question included in the monadic for each stimulus for easy comparison.

At-a-glance summary allowing for quick comparison between stimuli with the option of drilling down into detailed metrics of any question for any stimulus.

Detailed question output for a particular stimulus:

Detailed question output for a particular stimulus:

Displays detailed outputs for an in depth look into responses for a particular question.

With Conjointly, we provide detailed statistics into each question for each stimulus if you are interested in the nitty gritty details such as distribution of responses, medians, ranges.

Segmentation of the market

Segmentation of the market

Find out how preferences differ between segments.

With Conjointly, you can split your reports into various segments using the information collected automatically by our system, respondents' answers to additional questions (for example, multiple choice), or GET variables. For each segment, we provide the same detailed analytics as described above.

How to set up an A/B Test on Conjointly

Step 1

Start by selecting the A/B Test experiment type from the list of experiment types. Then, insert the name for the first item (not displayed to respondents) and insert what will be shown to respondents. You can insert an image, description, or short video.

Step 2

Insert the name and respondent display for the second item

Step 3

Import questions from another experiment. If you have a previous Conjointly experiment such as a Monadic Test or Concept Test your can import questions from these experiments to quickly set up an A/B Test. You can skip this step as you will be able to add new questions later on.

Step 4

Review both stimuli, including adding and modifying views. Views allow you to insert different representations of the concept in the diagnostics question. You can add images, long descriptions, use fancy formatting and lots more. You can also add or replace a stimulus in this step

While an A/B test is primarily designed to compare the performance of two concepts, including the top selling products in your categories will allow you to benchmark your concepts against existing products.

Step 5

Add or modify the questions that will be asked of respondents. you can adjust the questions found in the Monadic block section. To add questions to this block, simply drag and drop any of the additional question types into Monadic block.

Step 6

Customise the rest of the experiment. You are given complete control over the survey flow respondents will see.

Modify the following parameters to change the flow of the survey:

  • Randomise questions allows the system to vary the order of the questions inside the monadic block in order to reduce respondent bias.
  • Maximum number of questions to display per respondent lets you control the survey length by limiting the number of questions for each respondent.
  • Maximum number of stimuli per respondent helps you get more detailed feedback for each concept without introducing survey fatigue.
  • Sequence of questions lets you choose between lets you choose between sequential testing and random monadic.

Step 7

Preview the survey as a participant to test your setup and prepare to launch.

Complete solution for product testing