Passive Listening in Employee Experience: Limits, Privacy, Trust

Slack messages are not an engagement survey - despite what some vendors say.

These vendors claim to have found a way to solve survey fatigue by using chat, calendar, and HRIS data to infer employee sentiment so companies don't have to ask people any more pesky questions.

They go further and say that these behavioral signals are more objective than self-reported data. This bold claim is usually the selling point of the pitch deck, but it's actually the weakest part of the argument.

For instance, a packed calendar doesn't tell you whether someone feels valued at your organization. On the contrary, analyzing what they say might reduce what they're willing to tell you directly.

Even if these tools are developed after proper consultation with the relevant units, there's a risk they'll contaminate your future employee feedback exercises. If people think they're being watched, they may not be as open in your next survey.

They may also not take your future surveys seriously since you now have an alternative way to "check up" on them.

This means you need to be really clear about what "measures" each of your data sources before you roll out anything.

Procurement or IT teams are unlikely to sort this out for you.

In this article, we'll walk you through where you can and can't use passive data, what the legal and trust implications are, and what you should run by your legal team before embarking on a stealthy passive data project.

What can and can't passive listening do for your organization?

Passive data collection is meant to bridge the gap between measured behaviors and attitudes such as employee engagement, but this is challenging since the relationship is not straightforward.

Passive data inputs can include:

• Data aggregated from your HRIS related to turnover and absenteeism
• Collaboration metadata like number of meetings or after hours activity
• Network patterns data detailing who works with whom
• In some cases, the contents of messages from employees

Traditional, self-reported methods of measuring employee engagement suffer from biases such as social desirability bias, recency effects, or non-response from employees who have mentally checked out.

Passive data collection methods introduce different biases. For instance, these methods can only account for work that generates a measurable digital footprint and then assume this is representative of an employee's work.

For example, suppose two managers have identical meeting schedules. One is in back-to-back meetings because they have a new team that requires support. The other is in back-to-back meetings because they can't leave the office without decisions being made.

Similarly, after-hours messages may drop because employees are respecting their work-life balance or because they're disengaging from the organization.

A higher rate of cross-team contact could indicate a healthy level of collaboration or an indication that teams aren't communicating effectively requiring employees to follow up.

It's also important to remember that the ultimate goal is to create an engaged workforce. According to Gallup's global research, only a small minority of employees are engaged in the workplace. Passive data collection won't help you determine which of the rest are just coasting and which are actively looking.

Why the "we're just interested" argument is not settling the debate

It's an appealing argument. After all, declining survey response rates, the limitations of snapshot surveys, and underutilized HRIS data are all valid points.

But "We're Just Interested" doesn't address the big question that determines whether an employee experience analytics program will succeed: Are employees comfortable with how their data is collected?

A popular point made by vendors to ease employees' worries about employee experience analytics is that their research shows employees are more comfortable with text analysis than HR assumes. But take a closer look at this research.

For starters, even though these studies may state that employees are more comfortable with text analysis, these conclusions are based on hypothetical survey questions. Moreover, Qualtrics' own research found that senior leaders were far more willing to opt into an employee experience analytics program than individual contributors. In other words, the people signing off on a program - who may be less concerned about being analyzed - are the ones approving a program that more wary employees provide the majority of data for.

One study by the American Psychological Association looking at workplace monitoring found that monitored workers were more likely than unmonitored workers to report stress and poorer mental health. While this isn't proof that all analytic approaches have negative impacts on employee wellbeing, this is a sign that employee monitoring may not be received well.

Plus, claiming that an analytics program is "objective" or "fair" ignores measurement validity issues. For example, a metric like response time rewards employees who spend a lot of time interacting with work via chat, but penalizes those who do their best work when not at their computer.

People who work on the frontlines, who have shift jobs, who use their phone a lot, or who are working in a second language are all more likely to generate less or different data. Not only does this make these individuals less visible in a passive employee experience analytics model, it also creates the potential for adverse impact on already vulnerable employee populations (any employment lawyer will tell you this is an important consideration!).

Finally, there's the issue of Goodhart's law which states that when a measure becomes a target, it ceases to be a good measure. It's worth revisiting Jerry Z. Muller's book, The Tyranny of Metrics, which explains that when numbers are tied to judgement or reward, people focus on "working the numbers".

In other words, once metrics like "number of hours in a meeting" or "average response time to emails" are introduced into performance reports for managers, these managers will start optimizing these metrics. Within a quarter or two, these reports will reflect how well managers know how to manage these metrics, rather than how hard their teams were actually working.

Clearly differentiate "headcounts" from message monitoring

Companies often lump "passive listening" techniques together. This is problematic since they are very different and should not be approved as a package.

At the lowest end of the spectrum are aggregate, de-identified HRIS data like regrettable-attrition rates, absence trends, internal mobility, or time-to-promotion rates by cohort. This data is largely lagging. But the data is already collected for a clear purpose within the HR department, making the argument easier to limit that data's use.

That said, small cell sizes can lead to identification risks. You might have a differencing attack where individuals are identified by subtracting a filtered subgroup from a larger parent group. Combining different datasets amplifies this risk.

The next level up is collecting metadata related to work like meeting volume, focus time, number of days to respond, or network analysis. This can help companies test specific hypotheses related to workload at the organization or function level. But this increases privacy risks, especially if individual employees can be zoomed in on and identified.

Even if metadata collection doesn't involve content collection, it can be quite revealing. A network graph can display who frequently communicates with the HR department, a worker representative, or a colleague who recently raised a grievance. This is sensitive information that doesn't require the company to actually look at any messages or communications.

For example, Microsoft's research from the Work Trend Index found that most workers struggled to find enough uninterrupted focus time. So if you see rising hours of meetings in one function, this helps you identify a potential workload problem. What this doesn't tell you is whether employees feel supported.

And it can't tell you whether those meetings are causing the issue or a symptom of unclear decision rights within that function.

The highest level of data collection is analyzing content within messages. This includes collecting chats, emails, or running sentiment models on these communications. This processes what employees say to one another and might also collect health disclosures, union discussions, or personal matters that go way beyond your stated purpose.

In several jurisdictions, this isn't just unethical - it's illegal.

Monitoring changes the behavior you're measuring

What happens when employees learn that a tool is monitoring their workplace conversations?

Chances are high that employees will avoid discussing sensitive topics in public forums or channels by switching to direct calls, using their personal phones, or WhatsApping each other. They may continue to post in the public forums, but limit their comments to avoid scrutiny.

Suddenly, the data you're collecting only measures how much employees are willing to write publicly while assuming a model is reading their words. This diminishes the quality of data right at the point where you most need good data.

What's more, you may not even recognize that this has happened. If employees stop discussing relevant topics on a monitored platform, your sentiment score may actually increase since there are no negative comments to bring the sentiment score down. From a dashboard perspective, things look calm and collected, when in reality people have taken their discussions elsewhere.

Furthermore, machine learning models have limitations. They may struggle to accurately interpret sarcasm, phrasing from different language backgrounds, regional and cultural differences, or neurodivergent communication styles. All these things may impact sentiment analysis scores.

For instance, a chatbot may receive a message that says "great" after a difficult meeting about a restructuring. A bot may not recognize the nuanced sarcasm a human reader would understand. This offers challenges for sentiment analysis.

Finally, the impact of sentiment analysis monitoring can have a broader impact on your business. Once people begin to question what others will be able to trace back to them, they may view your next pulse survey with suspicion. This can manifest as low participation or a higher number of neutral midpoint (e.g. 'neither agree nor disagree' responses).

People need to feel safe enough to answer

As Amy C. Edmondson explains in The Fearless Organization, psychological safety is the belief that you can raise a concern or admit a mistake without being punished. It's different from simply feeling comfortable.

People only speak up when they believe it's safe to do so. When they're unsure, they remain silent.

When employees are analyzed covertly, they don't get a chance to decide what they want to disclose about themselves to an employer. When data collection is disclosed, but the rules around it are vague, it creates anxiety and uncertainty among employees.

Even if you say something as vague as, "We may use collaboration data to improve the employee experience," the average person hears, "Assume everything is being used."

According to Gallup, only a minority of employees strongly agree that their opinions count at work. Creating more listening channels won't improve that statistic. Taking visible action based on what people have told you will.

A survey, at least, makes it clear what the rules of the game are. By conducting a survey, you can let employees know what the topic is, who will see what, what the reporting threshold is, and what the timeline you're working towards is.

That said, if someone on their team sees that you're trying to figure out who wrote a specific comment, your promise of anonymity on that survey may be blown for all future surveys with that team. Recognizing this, it's a good idea to create explicit consequences for this kind of behavior before distributing the surveys and ensuring employees are aware of these consequences.

What to take to legal and privacy reviewers

Start with a written hypothesis. "Understand engagement better" can't pass a necessity or proportionality test. "Test whether meeting load in Operations rose after the reorg" might.

Bring that hypothesis to legal, privacy, and employee representatives where they have standing. Include someone from the population whose data would be used, not only the executives sponsoring the tool. Rules differ sharply by jurisdiction, so treat the questions below as a way to structure the review, not as legal advice.

If your footprint is global, expect the strictest works council or data protection authority to set the floor.

Can employees make an informed choice?

Pin down the actual data element. Is it a department's absence rate, the timestamp on a message, or the words inside it? Vendors blur these distinctions.

Your privacy notice needs to name them.

Check whether existing notices cover this new purpose. A notice written for payroll and security logging rarely stretches to behavioral analytics without a fresh purpose-compatibility assessment.

Be sceptical of consent. European regulators have long held that the power imbalance in employment means consent is rarely freely given. Legal should identify the real lawful basis.

If it's legitimate interest, document the balancing test. A tick-box doesn't settle anything.

Test the notice on a few people outside HR and legal.

Is there a less intrusive way to answer the question?

This is the question that tends to kill content-monitoring proposals, and it should. Could an HRIS aggregate show the pattern? Could two targeted survey items answer it without touching communications?

Fix the reporting grain before anyone sees output. A function-level trend is a different processing activity from a ranked list of individuals with low sentiment, and it needs different justification.

Write down the exclusions too. If the question is workload, individual message content is almost never necessary.

What happens after collection?

Put access, retention and deletion in writing, along with a rule against secondary use. The most common failure isn't the original purpose. It's the investigations team asking for the data six months later.

Ask who at the vendor can access the data, where it's processed, and what the data processing agreement says about model training. Check whether you need a formal impact assessment.

Check whether co-determination rights apply. In Germany, for example, a technical system capable of monitoring behavior or performance triggers works council involvement, whatever you say you intend to do with it. Also check whether using the tool to monitor or evaluate workers could bring it within scope of newer AI regulation.

Finally, agree how you'll know the signal is wrong. Before the pilot starts, choose a group-level measure you already trust, such as later attrition or a direct survey item, and set the level of agreement the passive signal must reach.

Set an end date and a stop condition in writing. If the signal doesn't beat the simpler measure by the review date, the pilot ends and the data is deleted. Without that clause, pilots tend to renew themselves.

Use this table to triage the proposal before that review:

Does this mean that employee surveys are dead?

Remember that intent to stay, trust in senior leadership, and sense of inclusion are all attitudinal measures. While you can make reasonable assumptions based on behavioral signals, some aspects of these attitudes can only be measured by asking your employees directly.

Anonymous employee surveys give employees the chance to opt in to participation and provide their feedback. They also allow employees to share their point of view in ways that cannot be captured through passive data, which is especially useful for frontline or deskless workers.

To reduce the potential for identification, organizations can implement measures such as requiring a minimum number of respondents before results are made available. However, organizations should recognize that there are other ways to identify respondents, such as through specific comments, differencing from overlapping filters, or combining survey reports with other employee characteristics stored in an HRIS.

There are certainly limitations to employee surveys and valid reasons for debate within the employee experience community about how weighty these limitations are. There are also valid concerns about data bias due to non-response from disengaged employees, or biases due to fluctuations in responses based on an employee's experiences that week. There are also reasons to reconsider survey item questions or improve follow-through based on a drop in survey participation.

However none of these reasons are sufficient for replacing authentic employee feedback with assumptions based on employees' calendar data.

Gallup attributes as much as 70% of the variance in engagement of teams to the manager. This is one of the strongest reasons for the importance of team-level survey results. This allows a manager to have a specific conversation with their team about survey results and what they can do to effect change.

While wondering if your "I love meetings" employee experience interventions really make a difference, Sparkbay allows you to directly ask your employees about their experience including aspects that a meeting count can't capture.

You can modify the language in the employee survey based on the hypothesis your organization is testing instead of asking employees questions to which you will infer feelings based on passive data.

You can also modify the language and content of the dashboard to reflect your organization's needs.

Once we have received enough responses for a group (default is 5 responses, but this is customizable) we will provide the group with an engagement score out of 10. If the group doesn't meet the minimum number of responses, no results are visible.

Once team leaders have access to survey results, they can review their team's results to help them decide if they should schedule a follow-up conversation. We automatically map report access to your org hierarchy, so managers only see their teams and no one else's, without any manual work from HR.

At no point can a manager see which employee provided which answer.

If you're interested in learning how Sparkbay can help you build a more engaged workforce, you can click here for a demo.

When is it appropriate to combine data sources?

Do you have to pick between operational data and survey data? No, but you do need to keep certain things separate.

Suppose you notice an increase in regrettable attrition in one department. You can use operational data to understand whether this is a local issue or a broader trend within the cohort.

You can then use survey data to ask the employees who remain in that department a few targeted questions about their workload and intent to stay.

Later, you can combine these data sets. But it's important to proceed in a way that's mindful of your employees' privacy.

For instance, you should be transparent about which data sets you're using for your analysis and why. You should also be aware of re-identification risks before combining data sets. This is especially important if you have small groups within your organization or unique attributes (e.g., the only part-time engineer in a specific location.)

It's also best practice to keep the data inputs you receive passively (like your operational data) at an aggregate level when possible. It's better to view a trend as a prompt to ask questions, rather than draw conclusions about what a particular team believes.

Finally, when you do receive data from your employees, it's important to close the feedback loop.

If you're interested in learning how Sparkbay can help you build a more engaged workforce, you can click here for a demo.