Are Employee Engagement Surveys Really Anonymous? Where It Breaks

In this article

1. Anonymous is not the same as confidential
2. How you can identify employees without their names
3. When can anonymous become a legal issue?
4. What do I do if an employee writes something in their response that is concerning?
5. Clearly communicating the nature of the promise to employees
6. Ensuring your vendor contract aligns with the terms of the employee invitation
7. Communicating the limits of anonymity to managers
8. Transparently communicating the limits of anonymity while showing employees the change they helped create

Imagine this: An employee on a three-person team opens your employee survey and reads a question about their manager.

They want to answer this question, but they’re the only person on their team who works remotely, and the survey just asked them to select their work arrangement.

Will their manager know it was them?

You should know the answer to this question, before you ask your employees to be open and honest. Most HR teams don’t know the answer to this question until they’ve gone through the exercise of trying to break their own reporting structure.

Aside from removing names, there are other ways that companies fail to protect employee data. This includes using filters that whittle a group down to one person, comparing two reports, comparing two survey cycles, creating live dashboards that update as each person submits their response, or including identifying details in comments.

There’s no need to promise more than you can deliver - it’s better to make a very limited confidentiality promise that you can keep, than to make an empty promise of anonymity.

But it is important to share what information is being collected, who it can be viewed by, and under what conditions the response will be shared.

You should share an anonymity/ confidentiality promise in writing, get legal review, and ensure you have the necessary safeguards to back up your promise.

1. Anonymous and confidential aren’t the same promise

Strict anonymity means no one, including the vendor, can connect an answer to a person. Most enterprise surveys aren’t built that way. They use unique tokens so they can send reminders, track participation and attach HRIS attributes such as department, tenure and location without asking employees for them.

If the tool knows your org unit without asking, it knows who you are.

Confidentiality is a promise about access. The vendor holds the link between person and response, and the employer sees only grouped results above a threshold. This is a legitimate design.

It’s also the only design that lets you link survey results to later attrition or analyze cohorts over time. If you promise "anonymous," you give up that analysis.

Pick strict anonymity when the population is small, trust is low or the survey covers sensitive topics such as harassment or ethics. Pick confidentiality when you need retention analysis, longitudinal trends by team, or HRIS attributes you’d otherwise have to ask for.

Neither word tells employees enough on its own.

Watch for a common middle case. The vendor deletes the link between token and response after HRIS attributes are attached, but keeps completion status by name. That design is defensible if the deletion is irreversible and happens before anyone at your company can query the data.

Even then, "anonymous" overstates it, because the attributes attached before deletion still narrow who answered. Your invitation should describe the data flow, not how the final dashboard looks.

Confidentiality also has limits the system can’t control. A manager never sees a named response and still recognizes who wrote a comment about last Tuesday’s shift.

2. Your employees might get identified in reports and data views, even when you don’t display their names

While it’s obvious that small teams pose a risk for identifiability, there might be other more complex ways for people to be identified based on filters and reports.

Suppose you have a department of 40 employees, and one of them is a remote worker. Applying a filter for remote workers reduces your large group to a group of one. Most survey tools suppress this filtered view.

But what if the tool provides a suppressed view AND the complementary view?

Consider this. We have the total number of employees in the department (40 multiplied by the department average) and we have the other number of employees on site (39 multiplied by the on site average). We can subtract these numbers to obtain the remote worker’s score.

This is called a differencing attack by statistical agencies. They protect against this by applying complementary suppression rules. If one slice of data is below the threshold, the survey tool also suppresses enough related data slices that a user can’t calculate that data slice.

You should ask your survey software vendor if they do this or if they only do individual view threshold checks. You should also ask your vendor if they assess time-based differencing attacks. Suppose a manager refreshes a live dashboard before and after a specific person uploads their responses.

They may be able to glean that person’s responses by comparing the two presentations without applying any filters.

Applying basic arithmetic, you can see how employees can be identified across survey cycles. Suppose a team has six people. They add a new member, taking the team total to seven.

By comparing the total for this survey cycle to the last survey cycle, and eliminating the responses from the six people for both cycles, you can get a glimpse into the responses of the incoming employee. This is especially true if the team was just above the threshold and only one person switched. You should ask your survey vendor if they account for data changes from one survey to the next or if they just apply a threshold based on the size of the group for each survey.

Employees can also be identified through skipped questions. Suppose a team of eight passes the threshold, but only three people answer an optional question. This optional question might be about their manager.

If the survey tool applies thresholds to the entire survey, rather than ensuring thresholds to individual questions, the responses to this optional question will be based on the responses of three people.

Research on re-identification shows how easy it is to identify individuals with minimal data. Latanya Sweeney showed that most Americans can be identified with just their ZIP code, birth date, and sex. In a well-known study looking at mobility data, researchers were able to identify almost everyone in the dataset with a few points of location and time data.

While these were not employee surveys, it speaks to the risks of survey metadata. You don’t need a name to identify somebody if you have a few attributes that only one person shares.

This suggests that your organization should collect less data or slightly adjusted data. For example, tenure data separated into bands of under two years, two to five, and over five will protect more people than asking, "how many years have you been here?" If a request for a specific attribute can be justified with how it will make a difference in action, then the organization should also consider whether those tenure bands of numbers can be eliminated if the bands contain fewer people than the threshold across the entire organization.

Other data such as unique links, SSO sign-ins, IP addresses, and timestamps can also lead to identifiability. Your identity provider probably logs who opened the survey and when, even if your survey tool doesn’t. The same goes for an email platform with open tracking or links with tracking parameters (added by marketing or communications tools).

If the data is captured about who clicked on a link, in combination with timestamps of submissions, it can be enough to identify people in small groups. The key takeaway here is that these details may be logged and that shouldn’t be a worry. What you need is to know who has those logs and data, if they can connect those logs to survey responses, if they can connect them in a time span that is concerning, and when those logs get deleted.

You should also be concerned about participation live dashboards. If managers can view participation completions in real-time by teams, then non-response to a survey can be identified by team members. This creates an enormous pressure that undermines the integrity of the entire exercise.

Consider presenting participation responses at the same threshold as survey results and only after a minimum number of submissions.

Consider also open text comments. Employees might add a comment that mentions a client, references a one-on-one with a manager, or says something that only two people witnessed. Comment metadata can identify people as well.

Suppose a filter exists for language and one employee responds in Portuguese. The language filter would identify that employee. One way to negate this is to translate all comments to one reporting language.

While this creates a loss of nuance, it removes risk of identification.

Finally, consider testing your reporting tool in the way a curious administrator would. Pick a small team, apply filters like an employee’s location and their tenure, open the complementary view, compare it to the parent view, compare it to the previous survey cycle, and consider exporting comments. Does the minimum submission threshold still apply to exports?

What about API access?

3. When the promise becomes a legal problem

What if you promised your employees that their answers were anonymous, but you later found out you had the ability to tie employees to their survey answers?

Depending on the market, there could be some significant implications.

First, let’s consider GDPR. Let’s say you pseudonymise data, but you’re technically able to re-identify employees. Under GDPR, this data would still be considered personal data.

In Germany, works councils have co-determination rights when it comes to technical systems that have the ability to monitor employees’ behavior or performance. Other European countries have similar employee protections.

So while you may have globally decided on a threshold of five employees for anonymity, the threshold that is set during negotiations in a works agreement may be different.

In the US, there is a risk that employees’ criticism in a survey about working conditions could be viewed as protected concerted activity under labor regulations.

It’s best to have your survey invitation and all survey language reviewed by local legal counsel for each country where you plan on distributing your survey, not just your headquarters.

How the survey results are used can also have legal implications.

Suppose an employee submitted critical comments about their manager in an employee survey, only to later experience a reduction in shifts or a missed promotion. If that employee files a claim of retaliation, your earlier assertion of anonymity will come into question.

Even if the initial decision was sound, your ability to sort fact from fiction will have been compromised by a loose anonymity assertion.

And your credibility in relation to all other communications about the employee survey process will be tainted.

There’s also the issue of survey records being used later on in litigation or an investigation.

While there may be confidentiality agreements in place among people who regularly access that data, those confidentiality agreements do not protect survey records from a subpoena or a litigation hold.

If the vendor still has the token to response link, that data may be subject to discovery. You want to manage your data retention policies with this consideration in mind.

Helen Nissenbaum, author of Privacy in Context, argues that privacy hinges on the idea of whether information moves in a way that’s aligned with the reasonable expectations of people within a given setting.

For example, employees might have different reasonable expectations when sharing information with their manager versus their HR department versus the vendor managing their survey.

While you may technically be telling the truth by stating one thing in a privacy notice, if you never acknowledge these different reasonable expectations in the privacy notice, you’ll present information to employees that’s not aligned with what they believed they were signing up for.

This idea of reasonable expectations can help you manage your data flows. You can evaluate each of your survey data flows to see whether it aligns with employee reasonable expectations.

For example, from the vendor managing the survey to the company’s human resources team, or from the human resources team to the employee’s manager, or from the manager to the manager’s boss.

If any of these data flows do not pass the reasonable expectation test, you have the option to call that out in the survey invitation or change the data flow configuration.

This is evidence that employee surveys should not avoid tough questions.

4. What if a response describes harm?

Receiving comments describing a worker’s experience with harassment or a statement that makes you concerned for someone’s safety is jarring. It might put your organization on notice for something, but if your survey is genuinely anonymous, it may not give you a reliable way to follow up with that person.

In this case, you can’t promise a private follow-up. You can, however, begin an investigation based on what you know, look for a pattern in other data, and remind people of the reporting channels that require them to provide their name in all of your survey communications. There are different opinions about how extensively you should investigate based on an anonymous comment.

Some view it as a formal complaint while others view it as a lead. You should discuss this with your Legal team ahead of time to understand how your organization will treat it, so you aren’t making decisions under pressure.

A good rule of thumb is to consider the type of comment you receive. If you receive a comment that includes specific information like a location, a time, or a person, you’ve got enough information to start looking into it while still maintaining the respondent’s anonymity. If you receive a general comment like, "harassment is ignored here," it’s a good sign of the culture in your organization, but the right response is to openly remind people of the reporting channels and to look more closely at the results of that group.

If there are circumstances under which you can link survey responses to people, make sure you explain those circumstances to survey respondents before the survey is launched. If you receive a troubling comment, it’s important to understand that making a change to the circumstances - for example, adding an exception - would breach the promise you made to survey respondents.

And if you need to make such an exception you’ll want to be specific about it. Something like, "We may need to share responses if required" is too vague to be useful. You’ll want to specifically say who will make that decision, who the response might be shared with, and the type of concern that would trigger such a process.

If you plan to scan comments for keywords (e.g. threats, self-harm) you’ll want to let employees know this.

5. Put your promise into words that employees can test

Before employees start the survey, let them know what to expect by including a short statement next to the survey link and an optional, longer statement one click away.

These statements should answer the questions your employees may have, such as:

• Can your vendor tie my invitation or account to my survey responses?
• What demographic details and technical information will the survey collect?
• Who will have access to my responses, comments, and grouped survey results?
• How many responses does a group need to have before survey results are displayed?
• What happens if I include something in my comments that identifies me or someone else?
• When will my organization and vendor delete survey data?
• Are there any circumstances that a member of my organization can share my personal survey response for investigation or legal reasons?

Would you be satisfied if your organization only published survey results based on responses rather than overall headcount? A group of 20 with 4 respondents only has 4 people represented in their survey results.

You should also clarify if there’s a minimum threshold for comments, exports, filtered views, individual questions, and complement views.

Will there be a minimum response rate for filtered results? This is often where organizations break their promises.

To make these decisions, you need to look at your org chart, not accept a vendor’s default setting. For example, a minimum threshold of 5 may work when most manager teams have 8+ people, because a typical response rate would keep them above the minimum. But if you have many teams of 5-7, your minimum threshold of 5 would render half your managers’ results invisible every survey cycle.

You may want to roll smaller teams into their parent unit instead of lowering the minimum threshold.

Best practice is to increase your minimum response rate for sensitive topics and comments. One sentence in a comment that can identify a respondent is more dangerous than a whole page of survey scores!

Avoid declaring things like, "Your answers are completely anonymous and will never be shared." This is likely to be untrue at some point, either when linking the token to the employee or when complying with a subpoena.

Here’s an example of what you can say, based on the most common enterprise survey setup (where the vendor can link individual survey invitations to respondents, but managers only receive grouped reports):

"The survey provider uses individual survey links to deliver the survey and track completion. However, when survey responses are shared with managers, they will not be named. We only show group results when enough people have responded to the survey.

This rule also applies to reports with filters applied. Comments may include information that identifies someone. A limited number of members within Human Resources will review comments before they are shared.

We may use or disclose information if required by law or if we need to respond to a serious safety concern. Please refer to the survey privacy notice to understand who will have access to your survey data and when your data will be deleted."

This is a good starting point, but you need to review the language and make edits based on your survey setup. We recommend you edit the phrase "enough people" to reflect YOUR minimum threshold, include the retention period you’ve agreed upon with a vendor, and remove any information you’re unable to support within your survey setup.

You can also add a line to the employee instructions that advises respondents to avoid sharing details only you would know that can link the comment to them, if they wish for their comments to be anonymous. This is a transparent way to manage expectations since nothing in a survey may stop someone from recognizing a comment.

You should also have your legal team review your final text to ensure it matches your live survey configuration. Your legal team should also review the survey instructions if you make any changes to links, filters, hierarchy, or comment sharing.

Keep in mind that changes to your org chart, such as splitting one team into two, can impact your "minimum threshold" settings and may require re-approval from legal.

6. Make sure the vendor contract matches the invitation

When you enter into a data processing agreement with a vendor, make sure the contract accurately reflects your data processing needs. The contract should specify the roles of data controller and data processor, the purposes for which data will be used, and what happens to the data after your organization and the vendor no longer work together.

In particular, you’ll want to pay attention to clauses about secondary use of data. Many vendors will include a clause that allows them to use aggregated customer data for purposes like creating benchmarks or developing products. This is generally acceptable as long as the vendor aggregates data at a cross-customer level and above a certain threshold, and that your notification to employees mentions this.

Finally, discuss data access. Are there specific roles within the vendor company, such as support engineers or implementation consultants, who will have access to response-level data? Will that access be logged?

Will your team administrators have access to that data? Can data be exported by anyone?

If your response to the last question is no, then you’ll want to ensure your vendor contract and the permission access controls implemented by your vendor reflect that requirement.

Ensure your data processing system has reporting thresholds that can be logged and tracked, rather than relying on "let’s be mindful of this" type HR policies.

Request that your vendor demonstrate how reporting thresholds hold up when users apply stacked filters, compare themselves to complementary groups, or view cycle-to-cycle trends. You should also assess how comments are managed. If a tool hides the scores of a small team, but still allows users to view the comments for that team, this creates a loophole.

Your vendor contract should also address use of sub-processors, breach notification timelines, certification of data deletion, and how the vendor handles requests from the legal system. Will they notify you before disclosing data to a legal request?

You’ll also want to determine how the vendor handles legal requests. Can you include a clause in your contract that addresses this? You don’t want to wait until the first request to have this discussion.

How Sparkbay can help you keep the survey promise

With Sparkbay, you can customize the wording of your surveys and dashboards to ensure you are using the exact language approved by your Legal team.

This way, your use of the words "anonymous" and "confidential" in your survey invitation, in the survey itself, and in the reports managers receive, is not done casually.

Sparkbay also hides survey results that receive less than a minimum number of responses (default is 5 but can be changed by the administrator).

This prevents results for small groups from being displayed. Of course, it’s up to your team to decide what threshold makes sense based on your group sizes and to decide how to review comments for identifying information.

In a large organization, it’s important to know who can see what. With Sparkbay, we automatically map report access to your org hierarchy, so managers only see survey results for their teams and cannot access reports for other teams.

Before rolling out your survey, have your team test the views described above that managers will use to access their reports.

If you’re interested in learning how Sparkbay can help you build a more engaged workforce, you can click here for a demo.

7. Provide managers with guardrails they can safely operate within

Oftentimes, when a data privacy breach occurs, it’s not a deliberate attempt to violate data privacy rules.

It may be a manager who reads a comment from the report out loud during a team meeting.

It could be a manager who shares a screenshot with another manager.

It may be a manager who tells their own superior, "I think I know who wrote this."

Make it clear to managers what they cannot do. Make it clear that they cannot ask team members who wrote a specific comment, that they cannot try to match writing styles to identify the author, or that they cannot question team members about their answers.

Make it clear that if they have serious concerns, they can escalate those concerns to HR.

The key here is to make this clear, to make this part of the policy, and to make it clear that there are consequences for violating this rule.

Otherwise, managers won’t take the rule seriously.

For instance, guessing who wrote a specific comment may not have to be right to cause a data privacy breach. If a manager guesses and is wrong, an innocent employee is now under suspicion. If they are right, then the entire team now knows that the promise of privacy may be compromised because someone knows who wrote what.

Keep tabs on who has access to your employee feedback data and limit access accordingly.

This may mean setting up reminders to review access permissions every time there’s a change in roles within your organization.

You want to identify people who may have access to your employee feedback data that they shouldn’t have access to.

This could be skip level managers, HRBPs with a very broad scope, or even individuals responsible for managing the last survey cycle.

Familiarize yourself with access logs, if your system keeps those.

It’s understandable that most managers want to have enough detail to be able to make changes.

But in most cases, knowing who said what won’t make the feedback more useful.

It’ll be more helpful for managers to simply receive the group result and receive support in how to prepare an effective follow-up question for their team.

8. Explain the data privacy context to employees, then highlight the resulting changes

In your invitation clearly state the data privacy context and include a link to an FAQ document that discusses small group thresholds, comment handling, access to comments, and information on where to go if they want a personal response.

Avoid making employees search through a privacy policy to find out whether their manager will see what they wrote.

After the survey closes, at the group level report back to employees what you heard and what you’re doing based on their feedback. If you aren’t acting on something, explain why. Employees appreciate a clear "no" more than no communication at all.

The level of participation in the survey, and the number of comments submitted, helps you understand whether employees trust the survey process. Pay attention to the tone of responses as well. If there’s an increase in neutral midpoint answers, a decrease in comments on manager related questions, but everything else remains the same, it could be a sign that people are providing guarded responses instead of disengaging entirely.

If you notice a decrease in participation after a survey cycle that doesn’t include visible changes, no matter how strong your anonymity notice is, it won’t solve the problem.

If you’re interested in learning how Sparkbay can help you build a more engaged workforce, you can click here for a demo.