Biases that are tarnishing the results of your employee surveys
It is February. It is that time of the year.
Organizations are in the process of initiating performance conversations with people (if they haven’t yet). The next two months will see employees doing a self-assessment, giving upward feedback, celebrating or groaning over their performance review, and setting goals for the next year with their manager – regardless of the performance cycles (quarterly, bi-annually, or annually) of companies. Most of these organizations will conduct employee surveys for a variety of purposes – such as performance reviews, peer feedback, upward feedback, engagement scores, etc. A majority of the time of Human Resource Professionals is being given to designing and perfecting these surveys, partnering with business managers, customizing surveys based on the teams’ needs, selecting questions that will yield the correct answers, and eventually serve the purpose for which the survey is being designed in the first place.
One thing that may possibly be worrying HR teams is whether the surveys are designed well enough to yield the correct results. Falsified results translate into wrongful outcomes, and puts a question mark on the sanctity of such surveys. Incorrectly designed surveys can invite response bias.
Response bias is a general term for a wide range of cognitive biases that influence the responses of participants away from an accurate or truthful response. ~Wikipedia
Here are the different kinds of bias that can affect the responses of your employee surveys:
1. Positive Negative Asymmetry (PNA) & Negative Bias
Academic research has proven that most open-ended comments are negative in nature. A related phenomenon, Positive Negative Asymmetry (popularly known as PNA) effect, refers to the “difference in the way that individuals process and attend to positively and negatively valued information.” This leads to what academia calls ‘Negativity bias’. This bias means that one is expected to have a higher negative impact of a stimulus compared to a positive stimulus which is equitable to the former. Research says, “The negativity effect is a reaction to specific stimuli and means the higher impact of negative than of positive stimuli of the same intensity of behavior, affect and cognitive representations of evaluated objects.” To understand how it emerges in survey responses (especially open-ended), envision this – For a question, one employee has a positive response and another employee has a negative answer up his sleeve. But the individual who has a negative response will be really negative and the positive response cannot match up to the former for its intensity. PNA theory suggests that “personnel who are asked about workplace issues on a survey should be more mobilized by negative than positive events.” “People who experience negative events may be stimulated to respond more strongly than those who experience positive events,” says the research. To give an organization-level example, in a study where employees were asked to recall workplace interactions that resulted in either positive or negative emotional reactions – the employees recalled significantly more negative interactions than positive ones.
2. Social Desirability Bias
Under the influence of this bias, the respondent selects options or gives responses which present them in positive light in front of their bosses. Employees may deny undesirable characteristics and ascribe to socially desirable traits. Jennifer Cullen, Industrial/Organizational Psychologist and Director or People Science at Culture Amp, calls this approach an exercise of “impression management” where employees indulge in managing their image (example not admitting burnout or challenges in work), instead of giving truthful responses. People do not want to admit if they are having trouble with their work, because it reflects an uncompetitive image of theirs; hence they give socially desirable answers, such as, “No challenge at work.” Cullen suggests 2 approaches to counter this bias:
- Better structure of questions
Shift the focus to the organization and the way their work has been structured by the organization from the individual and her feelings
- ‘We’ questions are more effective than ‘I’ questions
This removes judgment in the statement about an individual and makes it a collective assessment.
Under the influence of the ‘Social Desirability Bias’, the respondent selects options or gives responses which present them in positive light in front of their bosses
3. Acquiescence Bias
Another bias that disrupts a survey’s effectiveness is the tendency of the respondent to agree to a statement when she has limited knowledge of the subject. This is what research defines as acquiescence bias. There are multiple reasons why people have a tendency to say, “I agree”, as a response to many questions. Academic research suggests that it may be a “personality disposition some individuals have to be polite”; and to avoid social friction, they agree to what’s asked. Another reason is when people do not have enough knowledge about what is being asked, they feel inferior to the experimenter, and agree to his “expertise” in the form of an “agreement” in the survey. In an organizational context, this bias can easily creep in employee responses. Employees may be asked if they feel if they find the leadership team to be inspirational and role models. The employees may choose agree because of acquiescence bias even if they haven’t met any of the people in the leadership team.
4. Demand characteristics
This bias arises when people understand the purpose behind the study, and alter their responses so that the study meets its desired objectives. From an organizational perspective, it can happen in engagement surveys. There are certain stakeholders who would want the engagement score to be high, hence they give higher ratings to all the parameters.
Once the psychological biases are countered and kept away from the survey, it is important for the survey designer to look at aspects such as the order of questions, the language (like metaphorical, ambiguous) used, the kind of questions asked (double-barrelled, agree/disagree/ likert scale/ ratings, etc.). A survey design also leads to response bias, and all these aspects can trigger biased responses. It is important for HR to be cognizant of their existence, and then counter them well.