To be fair, we occasionally run Twitter polls. We’ve asked about brand trackers, personal use of cannabis, Thanksgiving food, and a variety of other research topics. Twitter polls are a fun way to encourage people to think about a topic, to spark ideas, and to engage in an informal way. But rarely (never?) are they a good way to gather valid and reliable answers to questions.
Why?
And why are market researchers so adamant about it?
First, let’s look at why results from Twitter polls don’t generalize to larger populations. According to Alexa, a website ranking tool, Twitter is the 13th most popular website in the world, and the 8th most popular website in the USA. Omnicore indicates that it has 330 million monthly active users, 100 million daily active users, and that about 23% of all internet users use Twitter.
These huge numbers sound impressive. It seems like results from any Twitter poll ought to generalize well to larger population. But that’s simply not the case.
In Canada, about 83% of people use the internet. Twitter can’t reach the 17% of people who don’t use the internet. Thus, my polls are biased towards people who can afford to pay for and are interested in using the internet.
There are about 37 million Canadians of whom 7.6 million use Twitter. Twitter misses about 80% of Canadians, those who don’t use Twitter. My poll results are biased towards people who like reading a tossed salad of information nuggets interspersed with bot accounts, fake news, memes, and pictures of strangers’ cats.
About 1000 people follow my Twitter account. These people are mainly market researchers and, I expect, a healthy dose of fake accounts, spam accounts, and inactive accounts. My poll results are biased towards people who like reading my posts about market research methods and techniques, and interesting factoids about Canada.
If any of my followers choose to answer, like, or share one of my polls, the Twitter algorithm might choose to show my poll to a few of their followers. And chances are their followers are mainly market researchers, data scientists, or branding, marketing, and advertising experts. My poll results will still be biased towards people who like learning about human nature and research methods.
Even though Twitter seems to be a giant focus group of millions of people, the people who answer my polls are an extremely nonrandom, and extremely homogenous group of people. Market researchers who like reading random nuggets of information online.
Just for fun, I ran a Twitter poll to find out if people think Twitter polls are valid and reliable. Which seems like a pointless endeavor given that we’ve just demonstrated that Twitter polls are not valid and reliable.
Regardless, feel free to click through to the poll, engage (that’s reason #1!), and chuckle along (there’s reason #2) with the poll.
Are Twitter polls valid and reliable?#poll #polls #polling #MRX #NewMR @esomar @amsrs @insightsMRX
— Carol Udell @ Canadian Viewpoint #MRX fieldwork (@CanViewpoint) April 24, 2018
So in the future, when you see market researchers poo pooing Twitter polls, remember that it’s not because we’d rather you brought your business to us. Yes, professional researchers have created access panels to gather results from broader and more comprehensive samples of people and you can pay them to run polls for you. But more importantly, it’s because we understand the value of valid and reliable information that will permit you to make sound decisions that lead to successful outcomes.
Validity must come first.
