Avoid These 5 Mistakes to Write Better Surveys
You depend on survey data to provide valuable insights that inform future needs, take the pulse of the industry, and help to navigate marketing strategies. But survey research is complicated and poorly written questions lead to bad data and even worse decisions. In this article, we’ll explore five of the most common mistakes people make when writing questions to make sure you avoid bias and make better decisions for your business.
1. Social desirability bias makes your survey unpopular
Nobody likes to be judged. But too often, survey questions ask about controversial or personal issues that respondents might not want to answer honestly, a phenomenon known as social desirability bias. Rather than a forthright response, you might instead receive one that the respondent feels is socially acceptable. This is especially likely if you ask about finances, vices, or politics.
If you do need to ask sensitive questions, you can reduce social desirability bias by stressing the survey’s confidentiality, as seen here and being considerate with your wording (i.e., avoid accusatory language). Another strategy is to ask which scenario the subject most closely identifies with instead of asking them directly.
2. Combined messages send mixed messages
Be sure that you only ask one question at a time. All too often, survey questions are difficult to answer because they include multiple or conflicting variables.
Example: Do you feel fulfilled in your role and valued as an employee?
This question is a common mistake, and not only confusing for the respondent, but it’s also useless for your business because there’s no way to know whether the data refers to the respondent’s feelings of fulfillment or value as an employee. Keep an eye out for combined questions (which often have the word “and” in them) and separate them when found.
3. Leading questions send respondents in the wrong direction
A leading question is one that hints at the response a survey writer is looking for. These questions tend to be less interested in gaining impartial information and instead look to validate preconceptions without considering the full opinion of the reader.
Example: “Our company was rated as a top 100 employer for our second year in a row. How has your experience been thus far?”
The first statement reads like an advertisement while the ensuing question assumes that the employee enjoyed their experience when they might not have. Instead, leave out the excessive exposition and keep the question neutral:
“Has our onboarding program met your expectations?”
Avoid leading questions by allowing your subject to tell their story to you, rather than telling it to them. Consider whether your question includes assumptions, opinions, or superlatives that might unduly influence your survey taker and cause mistakes in your data.
4. Order bias causes chaos for your data
Order bias is the tendency for respondents to choose answers based on the order in which they appear. Let’s say you have an extended multiple choice question with options A-H; some people will naturally select among the first choices while others gravitate toward the last ones.
To mitigate order bias, try to minimize the number of options for each question and check to see if your survey platform allows answer randomization so that choices appear in a different order for each survey taker.
If you notice that some respondents are repeatedly selecting the same choice, a behavior known as flatlining, consider cleaning them from your survey results as this generally indicates a lack of engagement.
5. Every question isn’t for everyone
Remember that some questions might not make sense for every respondent. If you want to continue a line of questioning for a subset of respondents (e.g., those who had to remain working onsite), use branching logic to address follow-up questions to that group specifically before returning to the full group.
Similarly, your answer choices might not apply to everyone. Think carefully about your answer choices and make sure all potential responses are covered. In many cases, you might need to add a catchall such as “not sure” or provide an “other” option followed by an open response that allows the survey taker to respond in their own words.
Finally, some questions might not be for anyone. Unnecessary questions needlessly extend your survey and can cause respondents to disengage or drop off before finishing. And while it’s always tempting to keep adding one last question, less is usually more when it comes to surveys to avoid survey fatigue. If your survey seems overstuffed, have a colleague run through the questions with fresh eyes and note any that appear redundant, excessive, or contain any of the mistakes outlined above. Whether it’s validating custom questions or offering recommendations for your vision, the survey experts at WorkTango can recommend the right practices for you beyond just “best practices”.
The better the survey the better the insights
Some might assume that writing a survey is a simple process, when it actually involves a lot of careful planning and strategic thinking. By taking time to learn common survey writing mistakes, your surveys will result in stronger data, sharper insights, and better decisions for your business.
Check out our guides on workplace culture, employee engagement, and employee surveys. Learn about every aspect of a successful employee voice initiative!
We write on the current challenges HR and organizations are facing in order to support our community. Check out more of our articles here.