How Does Conjointly Ensure High-Quality Panel Responses?

Conjointly ensures sample quality with a dedicated team of researchers and fieldwork experts who conduct extensive automated and manual quality checks throughout the fieldwork process.

Before you launch your study

During the process of recruiting respondents for your study, Conjointly includes a set of bias check questions alongside the qualifying questions. This approach helps mitigate the risk of acquiescence bias, which is the inclination of respondents to select favourable answers.

When you launch a study

Once a study is launched, Conjointly’s fieldwork team will review the launch request and perform several steps, including but not limited to the following:

• Coding the screener survey based on your selected pre-defined panel and setting up quotas.
• Reviewing your survey inputs and providing you with suggestions for fixes and improvements.
• Implementing survey redirects to ensure a smooth survey flow from the screener survey to your main experiment and appropriate end pages.
• Conducting multiple rounds of pre-launch testing to ensure that:
  • Display and screen-out logic, along with quota flows are working as intended.
  • Survey inputs are displaying correctly, especially in multi-language setups.
  • Thorough testing of alpha features in use, while notifying you of essential considerations for using these features.
  • GET variables, including participant IDs and quota variables, are recorded correctly.
  • Redirection of participants back to their respective panels for claiming their participation rewards.
• Preventing duplicated and fraudulent responses by enabling Disable respondents from the same IP address and Disable respondents from the same cookie settings.

To learn more, you may also check out this intuitive guide on Conjointly’s respondent recruitment process.

During the soft launch of your study

After completing all the steps outlined above, your project will proceed to the soft launch phase. During this phase, Conjointly invites 50-100 respondents, depending on your study, to participate in the survey.

Throughout the soft launch phase, the fieldwork team carefully examines the collected data and monitors:

• Survey completion time to assess participants’ attentiveness and ensure they are not speeding through the survey.
• Sample parameters such as location, age, gender, etc., to ensure they are consistent with your request.
• Consistency in responses across questions to prevent contradictory answers.
• Open-ended answers to identify if there are copied or repeated comments, nonsensical answers, etc.

Alongside manual checks, Conjointly employs automated measures to identify signs of fraudulent or inattentive behaviour during the survey. Respondents are automatically marked as low quality if any of the following signs are detected:

• Quick completion of the full survey (e.g., within 20 seconds).
• Rapid selection of conjoint choice sets (e.g., within 1 second).
• Presence of a mouse on the device, but no mouse movement is detected.
• Lack of scrolling when it’s needed to read the full question.
• Submission of unusual answers in open-ended questions.

Additionally, the system will automatically remove participants with 70% duplicated open-ended answers.

During the full launch of your study

Your study is fully launched, and Conjointly invites the remaining sample you have ordered. The fieldwork team will be:

• Monitoring the overall progress of data collection to ensure qualified participants consistently enter the survey.
• Monitoring and filling out the required quotas as specified by the predefined panel. During this process, the speed of data collection may slow down, especially when we encounter challenging quotas. This expected slowdown in speed is typically reflected in the average speed of the panel.
• Notifying you of any potential delay or request to relax specific quotas when the quotas are more challenging than initially expected.
• Reviewing the preliminary results of your studies to ensure response quality is consistent with the established standards.

Closing fieldwork

Upon reaching the target sample size and meeting all the required quotas, the fieldwork team will:

• Perform another round of quality checks similar to those in the soft launch phase. Poor-quality responses will be removed and replaced, which may lead to fluctuations in the response count.
• Pre-set up key segments based on your specified quotas.
• Upload other respondent variables, which are typically answers to the screening questions along with additional respondent background data if available.