Introduction
We live in the age of Employee Experience. An unprecedented talent shortage and changing attitudes towards the nature of work have made it an absolute must for growth-minded businesses to prioritize initiatives that ensure employees are both challenged and fulfilled at work.
This Employee Experience boom has made one tool in particular a critical part of any HR leader’s toolbox: the engagement survey, a set of questions designed to regularly identify if people feel valued and involved at work, and give them a chance to provide feedback to leadership.
Platforms like Peakon and Culture Amp have made it easy for companies to analyze survey responses and summarize the data with Engagement Scores and similar metrics, which has helped finally give People & Culture initiatives representation in C-Suite discussions. Increasingly, People Leaders themselves are measured by their ability to drive engagement.
The Problem with Engagement Surveys
As the business world becomes more data-driven, HR leaders are increasingly expected to report on the success of their initiatives based on engagement. However, there are several limitations of engagement surveys that put HR at a disadvantage to the other functional leaders around the table.
The feedback loop is too long
Think for a moment about the data that each functional leader in the organization has access to. Marketing, Sales, Finance, and Product leaders are swimming in up-to-the-second data.
That’s not true for People & Culture leaders. Engagement data isn’t passively collected in the background. Surveys need to be designed, distributed, compiled, analyzed, and interpreted. Most companies conduct surveys only a few times a year, and it can take weeks or months to reach conclusions. This means it takes a long time to answer critical business questions.
Survey Fatigue
In an effort to address the feedback loop problem, many companies have tried to optimize their approach to conducting surveys, focusing on getting more frequent input. While survey vendors have gotten creative in designing ways to minimize the burden, there’s no way around the fact that being asked to constantly fill out surveys has the potential to be both distracting and annoying for employees.
Subjectivity
In addition to the practical constraints involved in measuring the success of People & Culture initiatives with surveys, there’s also a validity one. Are responses biased by the design of the survey? Do people answer them truthfully? Is there a disconnect between what people say and how they behave? These are all common concerns that make it harder for HR leaders to use data to advance their work within the C-Suite.
Natural language processing: a new way forward
While survey-based approaches have taken centre-stage in the Engagement discussion at most companies, exciting technology for alternative ways to measure People & Culture initiatives have made tremendous advances. While those in the People Analytics community have known about these tools for years, some of them appear ready to go mainstream.
Natural Language Processing (known as NLP) is one of them. NLP is a branch of artificial intelligence that deals with the interaction between computers and humans using the natural language. It’s commonly used for tasks like translation, spell-checking, and speech recognition, but it has many other exciting potential applications.
Most companies produce a tremendous amount of unstructured text data: chat messages, emails, memos, and more. NLP allows us to find patterns and meaning in the way people communicate, and apply that to our understanding of how People & Culture initiatives have impacted the company.
This works using a “dictionary” of words, phrases and patterns that are associated with different drivers of engagement or culture.
For example, here’s an illustration showing how text can be mapped to a 6-dimensional model of company culture developed by Professor Charles O’Reilly at Stanford. Try it yourself with NLP Magic.
For companies that work digitally, NLP can be an incredibly helpful tool. Here at Bunch, we use NLP to track several drivers of employee engagement, including collaboration, morale, freedom of opinion, and willingness to learn based on evidence in digital communications.
For example, here’s how morale at our own company has developed over time based on our Slack messages:
This is based largely on analysis of sentiment in text. Language associated with positivity increases the morale metric, while negative language decreases the score.
The depths of Q4 2018 show just how hard it was to pivot our business towards a new product, while our recent highs show the excitement of launching that product. This approach has given us an “early warning system” to identify problems before they would otherwise be noticed.
This data is collected in real-time without anyone having to fill out a survey.
The feedback loop is instant, and the constant input means we don’t have to worry about when we collect data. Further, employees can’t “game” the system without actually changing the way they behave (win-win!).
The system isn’t perfect. For example, computers aren’t great at detecting sarcasm. But this approach provides an objective measure to show directional shifts that can People Leaders make more informed decisions. Combining real-time metrics with occasional surveys and a dose of common sense is a great way to overcome some of the limitations of relying only on surveys to understand and measure engagement.
If you use Slack or Microsoft Teams, you can analyze your own company’s historical trends for key engagement drivers with Bunch.ai (there are free and paid options). It takes just 3 minutes to set up.
Have an opinion on how NLP will change the way we approach People & Culture initiatives? Let us know in the comments!
