Should you analyze social media sentiment? 5 questions to help you decide

Not every company should measure online sentiment. Use this guide to decide whether it’s best for you, and how to evaluate your findings.

Whether they bill themselves as listening tools, measurement services or media monitoring firms, there are now more than 150 companies pitching social media measurement services. Almost all offer sentiment analysis—the art (or science) of gleaning how people feel about your brand by reading what they say.

Should you do sentiment analysis at all?

Yes, it’s the latest shiny new measurement toy, but it’s not always possible, or even useful. Carefully consider these two questions before you decide:

1. Do people express any sentiment at all in discussing your brand?

You can’t measure sentiment if it’s not there. For many sectors (marketing a B2B product, for instance) it may well be that all the conversations out there are factual discussions. It may not be possible to glean any sentiment out of the conversations.

2. Do you have any direct interaction with customers?

Measuring sentiment is worthwhile only if you can use your results. If you have no direct customer interaction, it will be very difficult to determine whether the expression of sentiment has any real impact on your business. Only if you are an online retailer, or are in a field where people make reservations or register online, can you tie sentiment to customer behavior.

If and when you decide to go ahead with social media sentiment analysis, the biggest decision you have to make is whether you are going to use human or computer-automated analysis. Before you rush out and buy a sentiment analysis system, here are four questions that will help you decide whether computers or humans will do the better job for you:

1. Do you receive more than 10,000 qualified mentions a month in social media?

That’s not counting spam, copy generated by content farms, or mentions of a similar sounding brand (for instance, Carmax the car superstore vs. Carmex the lip balm). If your volume falls below this mark it might be more expensive to program a computer than to use humans to accurately analyze your coverage.

2. What level of accuracy is acceptable to your executive leadership?

Most automated sentiment analysis tools get sentiment right about 40-60 percent of the time. If that is good enough, you can use an automated system. If not, then to ensure a higher degree of accuracy you need to (at least) use humans to check random samples of your analysis, and re-analyze (perhaps with humans) as necessary.

3. If you need a high degree of accuracy, do you have 20,000 qualified mentions?

Computer-coded accuracy increases with the number of mentions analyzed. It is easier to get a higher degree of accuracy if you have many mentions to work with. In our experience, that number is about 20,000.

4. What level of detail do you need from your sentiment analysis system?

If you need to track complex messaging, quotes, issues, positioning or other esoteric details, computers will be complex to program and slow to deliver results. Chances are you will need humans to get the job done with acceptable accuracy and reasonable speed.

5. Do you run numerous campaigns that require different search terms, different message tracking and different definitions of positive or neutral?

Again, computers can take weeks to reprogram, test and fix. If you need fast turnaround on changes to your system, use a human.

Katie Delahaye Paine is CEO of KDPaine & Partners, and publisher of The Measurement Standard newsletter, in which this article first appeared. She is the author of “Measure What Matters.”

COMMENT Daily Headlines

Sign up to receive the latest articles from directly in your inbox.