Photo Credit: freepik.com/storyset
Welcome to the third and final installment in our Survey Bias series where we talk about all the different sorts of survey biases that can occur during a survey. We’ve covered different stages of conducting a study and have noted that bias can occur at any stage: planning, execution, and inferences.
Three unique types of survey biases have emerged as the main culprits that can affect the validity and generalizability of a survey or research. These are:
We’ve covered the two earlier in the series and today we tackle the last one: interviewer bias.
Interviewer bias is also known as researcher bias and (similar to the sampling and response biases) further divides into sub-categories.
Before we go there, let’s take a quick refresher about our broad topic: survey bias.
Bias, in general terms, refers to unfair support or opposition extended to a person, thing, or goal due to personal reasons and opinions.
In surveys and research, bias is any deviation from the truth. As we’ve already said, “Any act or decision that can potentially influence the survey results and drive them away from the true picture is known as survey bias.”
While experts agree that survey biases cannot be fully eliminated from research (uncontrollable external factors can always play a role), the goal is to minimize unintentional bias as much as possible and exclude intentional bias completely.
Remember, a biased study will always produce inaccurate results that would be invalid and could not be applied to the broader public.
Interviewer bias occurs when survey creators or interviewers consciously or unconsciously try to influence the survey process, resulting in skewed findings.
Based on what causes it, interviewer bias can take many forms. Some major ones are listed here:
When the interviewer uses nonverbal behaviors — body language, facial expression, and gestures — to either intentionally or unintentionally influence the respondent to produce certain answers.
The interviewer nods or raises an eyebrow or shifts uncomfortably in the seat when hearing a certain response or asking a certain question. This can lead the respondent to align their answer to fit the interviewer’s apparent expectation.
When the interviewer’s tone or attitude displays their approval or disapproval of a certain question or answer. It happens when the interviewer becomes animated when talking about a certain topic or shows disinterest when hearing something else. Respondents who match the interviewer’s preferences may alter their responses to fit the displayed preferences.
In an awareness and usage survey, a respondent may choose not to elaborate on their negative experience if the interviewer shows displeasure about the topic. This may result in a study that highlights positive experiences and downplays negative ones.
This is the type of interviewer bias that results from asking leading or suggestive questions that steer towards a certain response. Interviewers may unintentionally provide hints that influence a respondent on what to say.
During an HR survey, the employee is asked about instances of successful conflict resolution with a difficult boss. The condition of ‘successful’ may tell employees to only focus on experiences that result in a positive outcome.
When the interviewer allows their personal opinions, beliefs, or prejudices to influence the research process in any way or to interpret the final results in a certain light. It is also often referred to as favoritism or discrimination.
Instead of simply asking, “What is your stance on same-sex marriage?”, the interviewer leads with, “Do you believe in traditional marriage values?”
These loaded or emotionally charged questions are a huge no-no. They may give you the answers that you’re looking for but they won’t show you the real picture.
This type of interviewer bias involves sharing survey results that either partially or completely omit certain findings. This happens when the study wants to downplay a certain reality or when they are unsatisfied with the results.
If a researcher studying Gen-Z’s social media habits and mental health has a preconceived negative notion about this dynamic, they may emphasize data that highlights this correlation while oversimplifying more nuanced findings.
Mary K. Salazar, a University of Washington professor has talked of 8 preventive techniques that address correcting interviewer bias. Here is a quick and simplified version of those suggestions:
Biases can creep into the most well-intentioned surveys, so our job as researchers is to try and minimize these occurrences.
We hope our Survey Bias series will help you stay on the right track when designing your business surveys and research studies. Tell us what you think of the series, and any topic you’d want us to do next!