Reach, engagement, understanding, behavior: A better way to measure comms

Your overall engagement numbers say your employee comms campaign is working. That’s great! But then you break down the engagement by audience segment, another story comes to the surface: not everyone is getting the message in the way you want.

Johnna Muscente, vice president of communications and PR at Corcoran, told Ragan that communicators should start with the needed outcome for each group is the best place to begin.

“Start with what each audience needs to know, feel or do,” she said. “If the desired outcome is different, the measurement should be different too. Senior leaders may need to understand the strategy behind a transformation. Managers need to feel equipped to answer questions. Employees need to understand what it means for their day-to-day work.”

Different outcomes also mean different delivery methods. Whether someone can act on a message depends on where they encounter it. And not everyone is sitting in front of the same inbox.

“You can’t judge an office employee who lives in email against a frontline employee getting information through a mobile app or workplace signage,” Muscente said. “The measurement has to reflect how people actually receive and use the information.”

Know what you plan to do with the numbers first

This step happens before a campaign ever reaches the intended audience. Muscente said there are three things she defines before anything goes out:

• The business objective of the messaging campaign.
• The employee behavior you want to influence.
• What you’ll do with the metrics you get back.

Muscente’s third rule is what separates a strategic data point from a number. She builds the response into the plan ahead of time — for instance, if employees understand the message but still aren’t acting, the problem may not be communications at all. It may be an operational barrier.

“Too many communications dashboards exist because somebody expects a dashboard, not because the numbers are actually going to change what anyone does,” Muscente said.

Follow the chain as far as you can

Once you’ve decided how you plan to act on the results of your metrics, the next question is how far to track them. Open rates are only the starting point.

“Measure people based on how they actually engage and what you need them to do,” Muscente said. “Across all of those groups, I generally think about four things: reach, engagement, understanding and behavior. The farther you can get down that chain, the more meaningful your measurement becomes.”

• Reach: Did the message get to them?
• Engagement: Did they do anything with it or just open it?
• Understanding: Did they take away what you intended?
• Behavior: Did anything change?

The four questions should remain the same across the organization. The metrics that answer them shouldn’t.

“For office employees, email engagement or intranet activity might make sense,” Muscente said. “For frontline employees, you might care more about mobile engagement, operational adoption or whether key messages are actually reaching teams through managers.”

When Muscente’s own numbers showed frontline teams lagging behind corporate employees, the response came in three parts:

• Shorten the content.
• Give managers more frequent talking points.
• Move information into the channels frontline employees already used.

Measure managers separately

Managers get treated as a distribution method for internal messaging, but Muscente said that not enough communicators check with them to figure out whether or not the message distribution worked in the first place.

“They’re one of the most important communications channels in any organization, but we often measure whether employees opened the announcement without asking whether managers understood it, felt ready to discuss it or even received it early enough to lead effectively,” Muscente said. “If managers are finding out about a major change at the exact same time as their teams, you’ve created a credibility problem before the conversation even starts.”

She recommended asking four questions about managers specifically:

• Did they attend the original message briefing?
• Did they use the materials the comms team shared with them?
• Do they feel confident answering questions?
• What are they hearing from employees?

These questions don’t need to be completely formalized. If communicators ask them at the end of a call with managers, they’ll likely get what they need.

Count the things that don’t show up in a report

Muscente said that not all of your best data is going to come from a dashboard.

“I think of quantitative data as the ‘what’ and qualitative feedback as the ‘why,'” she said. “An open rate can tell you engagement dropped, it can’t tell you whether the subject line was bad, the timing was wrong, or employees didn’t care.”

Muscente said that some great sources of qualitative feedback that can help you understand different employee populations include:

• Pulse surveys
• Manager feedback, whether formal or informal
• Frequently asked employee questions
• Topics that repeatedly surface in town hall meetings

“Numbers can look very precise while completely missing the employee experience, while qualitative feedback can sometimes overrepresent the loudest voices,” Muscente said. “The useful insights usually come from looking for patterns across both.”

She added that the dashboard isn’t the whole point.

“That’s the difference between measurement and reporting,” Muscente said. “Measurement should help you make a better decision.”

Sean Devlin is the senior internal communications editor at Ragan Communications. You can connect with him on LinkedIn here.