Research access panels are often the perfect way to gather opinions from hundreds or thousands of people from across the country. People have already consented to be contacted and share their opinions about a variety of topics, and they feel comfortable sharing their most personal experiences because they know and trust the company they’re helping. But a common concern often comes up when researchers are deciding which panel would best suit their needs.
Researchers might not seriously consider working with a particular panel because they believe it is too small. When some panels advertise their size in the millions, it can be hard to seriously consider a panel that advertises ‘only’ hundreds of thousands panelists. A decision to work with the larger panel seems obvious. Or is it?
To illustrate why panel size is often not relevant, let’s consider two hypothetical companies. MassivePanel says they have 1 million panelists. They launched a questionnaire to everyone on their panel and got 50 000 completes. This equates to a response rate of 5%. On the other hand, MaintainedPanel says they have 200 000 panelists. They launched the same questionnaire to everyone on their panel and also got 50 000 completes. This equates to a response rate of 25%.
Even though MassivePanel is 5 times as large as MaintainedPanel, it turns out that the effective sizes of the two panels are identical. There is no advantage to choosing the larger panel because the end result is the same. So what’s happening?
Panel rules are what’s happening. Most panels have rules about who they include when they define their panel size. MassivePanel might have decided that a panelist is ‘active’ if they clicked on a survey invitation in the past 12 months. A rule like that would result in a very large panel as many people would click on at least one email, even accidentally, over the course of a year. The survey completion rate would be low but the panel size would be large.
On the other hand, MaintainedPanel might have implemented a much stricter rule. They might have decided that an ‘active’ panelist is someone who has fully completed at least one survey in the last 3 months. A lot fewer people would meet this criterion but the panel would be full of people who are known to fully perform the research tasks asked of them. The survey completion rate of this panel would be high although the panel size would be small.
In other words, panel size is meaningless without an estimate of the research completion rate. The next time you’re considering which panel to work with, keep in mind that the starting panel size isn’t always the best indicator. The best indicator is whether you will receive the number of completed research studies you require.
You might like to read these:
