SUMMARY: Beware of software tools promising quick answers.
There are lots of companies popping up, claiming that their tool is be able to sift through the sea of internet chatter and bring marketing insights front and center in a beautiful chart or dashboard.
The only trouble is, they’re mostly full of it.
Analysis of online conversations is a complex business, not only because of the sheer volume of posts to analyze, but also because of the over-simplified way that some of the most popular tools present the data returned. And both these issues make people to want to accept the ‘figures’ provided them - after all, the data _looks_ authoritative, right?
Over on Bloomberg Business Week, there is an article about a social media analytics company that mines millions of postings on social media sites to provide ‘insights’ back to advertisers. But they tellingly tout their speed - software that reads and analyzes thousands of sentences per minute - finding “77,000 mentions of stubble online in less than six seconds.” This is tech window dressing - Google is faster. So what is being returned that is so smart? The article goes on to mention that the company’s researchers isolated all the positive comments, categorized them into themes, and built a chart for the client ‘in less than an hour’ ranking the topics of conversation and motivation for the posters.
There are a few key things to watch out for in this story:
1. You’re never going to be able to isolate _all_ the comments on any tool. Not all conversations are public, and not all conversations are online. Period. Remember that you are only going to be able to analyze what someone decided to put online.
2. The people who buy your products are not necessarily the same people as those who tend to post a lot. People who post a lot tend to want to talk (surprise), not necessarily listen or engage. Always pay attention if a company says that they can being you the voice of your user - more likely than not they want to bring you the voice of a user, but not necessarily your user.
3. If the researchers took the time to go through any of the 77,000 mentions, they would probably find that well over half the returned posts are spam, and half of those remaining are not relevant. Even with spam filters in place, it is amazing how few posts returned by the major tools are on-topic and usable.
4. Automated sentiment is notoriously inaccurate - with the vast majority of tools returning 80% of posts with no sentiment applied whatsoever, and often incorrect flagging of the balance.
5. They try to talk about the tool doing the heavy-lifting, but if you read carefully it’s clear that the research team had to synthesize the actual results. Software stinks at thinking like a human, and so it was really the research team that pulled the insight together. And pulling an insight together in an hour is not my idea of a powerful selling point. I’d rather the team spend the time to come up with an answer that changes my thinking, or changes behavior, or changes the world.
Simply put - tools and companies that tout both speed and correct answers are pulling your leg. There is no Google for insights where you just plug your question in and get a smart answer popped back to you. It takes hard work, manual review and critical thinking. Add smart tools to this kind of a mix, and you start really being able to change the world and turn the industry on its head.
Tweet This Post