It’s all too easy to use your blog or Twitter account to only draw attention to studies that back up what you’re selling.
It’s probably a conservative estimate to state that at least half of the content social media users share informs others of how wonderful it is in one way or another.
And yet, I wonder how many people would continue to read a newspaper if half of it was dedicated to how important newspapers are and how everyone should read one.
This is why I draw your attention to a recent study by Pew Research. It suggests, on the surface at least, that social media research may offer less value than many of us working in the industry claim.
The year-long study concluded that “the reaction on Twitter to major political events and policy decisions often differs a great deal from public opinion as measured by surveys.”
The report stated that “the overall negativity on Twitter over the course of the [US presidential] campaign stood out,” adding, “For both candidates, negative comments exceeded positive comments by a wide margin throughout the fall campaign season.”
This is a rather damning indictment of the ability of social media research to capture public opinion.
Twitter isn’t a representative sample
It’s important to acknowledge that those of us who use Twitter are not a representative sample of the population. Twitter skews toward a narrow, but expanding demographic.
As far as the wider population goes, Twitter is still quite new. The relatively recent media infatuation with it is a testament to that. Social media’s early adopters are more likely to be those interested in being part of and shaping the public sphere.
I’m not sure anyone would be particularly surprised to discover that Twitter’s character limit—in addition to the fact that more extreme opinions frequently win more attention—lends it to more polarized views.
The fact that the study found that “the overall negativity … stands out” isn’t especially illuminating. It demonstrates we generally capture the opinions of people who feel strongly about issues.
Social media research doesn’t work in a traditional, quantitative sense
I don’t think it’s presumptuous to say that few of us use our social media accounts to maintain positive sentiment toward brands we like. As we’ve found in our research at Precise, people are more likely to use Twitter to complain to companies, particularly when those companies use social media.
This isn’t to undermine the importance of understanding what people say about your brand on social media. Numerous departments within an organization would want, and need, to know what their customers have to say. Ignoring one of the channels customers use to express these opinions simply because you can’t be sure who they are and what their motivation is seems foolish.
It just means social media research can’t replace or be directly comparable to what the likes of YouGov’s BrandIndex does in a more representative and repeatable way. However, a brand tracker is usually always a starting point, allowing us to hypothesize about the direction public perception is moving. Analyzing discussions about a brand within social media is one way to understand why opinions may have changed.
Tracking the size of the discussion around a particular brand or issue also provides important context, but only the context in which those conversations take place.
None of the above caveats mean insight we gain from social media research is useless. We just need to start thinking about it differently. Its main value comes from the fact that the comments we capture are spontaneous and not defined by questions.
As well as helping to identify those “known/unknown unknowns” about a brand (which you can then build into a more traditional quantitative approach by, for example, ensuring you ask better questions in your survey), it’s best to use social media research to feed into the creative process, investigate a particular issue, or answer a question that’s difficult to answer by other means—not to predict voting behavior or claim it can “accurately gauge public opinion.”
In terms of research, social media effectively acts as a giant, disorganized focus group where anyone is free to share his opinion about anything. Its potential lies in offering the chance to do qualitative research at scale. Again, it’s not an opportunity to necessarily do it better than other types of qualitative research, it’s just another approach.
We shouldn’t dismiss social media research simply because the results aren’t always representative of the population at large.
Twitter analysis isn’t the same thing as research conducted across all forms of social media. The former is a lot quicker and easier to do, but research is far more powerful if you include other sites.
Companies tend to underestimate the value of what their audience has to say on forums and blogs; their opinions tend to be more balanced. Too many analyses focus on Twitter alone, for no other reason than it’s quicker and easier to categorize posts because of the character limit. Most of the interesting and genuine conversations take place on forums anyway.
It is worth noting that Pew’s conclusion is based on two important premises:
1. Pew’s survey is an accurate reflection of true public opinion.
2. Pew did the Twitter analysis effectively.
Sentiment is subjective, and people may misinterpret what we write and say. Our opinions may also be more nuanced than the box we’re required to tick in a survey. As one PhD student highlighted on Twitter, we need to be particularly careful with the conclusions we draw from quantitative research:
wonder if Twitter liking Obama’s State of the Union less than Public Opinion is more “conservative”? maybe Twitter was to the left of Obama?
— nathanjurgenson (@nathanjurgenson) March 5, 2013