It’s October, so the decorations are up. Plastic skeletons, fake cobwebs, a rubber spider someone left in the mailbox.
Anyone who works in market research has a scarier Halloween story, and it usually happens on a regular weekday. You open the data file and find a respondent who finished a 15-minute survey in four minutes and called your new product “asdfghjkl.” Then you see the next respondent, who answered in 16 minutes and named your product correctly with a great comment about it, but three other responses in the data file have the exact same time check and comment – word for word.
Bad data is hard to deal with because it rarely looks as obvious as that first example. More often these days, it looks like the second, but even less obvious. Unless you have the right systems in place, it can stay in the data file and might nudge a satisfaction rating up or down, potentially making a difference on the recommendation built on the data.
Canadian Viewpoint Inc. (CVI) has spent 45 years in fieldwork, which means they’ve met these problems before. Their Quality Control Handbook read like a field guide to everything that goes bump in the night, along with what to do about each one. Here’s a walk through the cast of characters and how CVI deals with them.
Meet the Monsters
Every haunted house has its regulars, and every dataset does too.
Bots are the fastest of the bunch, racking up completes that no human ever touched. Speeders treat your carefully built questionnaire like a hallway to run down. Straight-liners pick the same answer across an entire grid, so they float through it without really being there. Copy-pasters drop the same paragraph into several open-ended questions. And duplicates keep coming back for another go at the same survey.
None of these are exotic. It’s the normal cost of doing research online. The real question is what’s waiting at the door when they show up.
The Invitation to the Party
CVI’s first line of defense starts with who they invite to participate in surveys. They start with SurveyLion, a proprietary consumer panel of Canadians across English and French markets, plus a Medical Panel.
Then there’s the part that sounds like it belongs to another century. CVI mails real incentive cheques to SurveyLion panelists’ home addresses. The cheque has to both reach an actual address and get cashed at an actual bank, which means the name on the cheque needs to match the name on the bank account. Bots have yet to break through that system.
Panel members aren’t sent too many surveys invitations, so they aren’t fatigued by the process. Someone who’s been pulled into a dozen surveys before lunch is going to start rushing, so keeping the volume low keeps people paying attention.
CVI also works with vetted sample partners around the world for projects when their own panel isn’t enough for the study. They verify IP and computer information to remove duplicates from the database. They also blend the panels early in the study so that they can maintain consistency throughout the study, especially important for those studies that happen wave after wave.
The Front Door
Even a verified panel needs an extra layer of quality control. When a person tries to enter a CVI survey, a set of technical checks runs before the first question loads.
Digital fingerprinting looks at device signatures, browser characteristics, geo-IP location, and past participation to score the risk of fraud or duplication, and high-risk visitors get blocked before the survey launches. Cookie-based tracking keeps the same device from completing the same survey twice. Bot detection, CAPTCHA tests, and real-time IP verification confirm that the traffic is human and coming from the right part of the world.
Most of the problem respondents never get past this point.
Zombies in the Survey
Getting in doesn’t always mean someone is paying attention. Many respondents move through a survey like zombies, physically present, and mentally somewhere else.
CVI keeps watching once the survey is underway. Questionnaires may include attention checks and red herring questions, such as a fictitious brand slipping into an awareness grid. Taking advantage of CVI’s survey programming capabilities means adding extra layers of monitoring to catch questionable responses. Completion times are compared against the median length of interview, and anyone who finishes in roughly 33% to 40% of the average time or less gets flagged for manual review. Grid questions are monitored for straight-lining, and IP addresses and device signatures are tracked across active field runs to catch repeat completions from the same network.
None of these checks is a verdict on its own. A fast respondent isn’t automatically a bad one. That’s why post-fieldwork checks are important.
After Fieldwork: AI Narrows the Field, People Decide
Once fieldwork closes, the data gets scrubbed, and this is where CVI’s operations team puts AI to work.
No single signal condemns a respondent. The open-ended answers get checked for gibberish, copy-pasted text, off-topic responses, and profanity. Those findings are weighed alongside the speeder and straight-lining flags and each respondent’s track record over the past six months. Together, they direct reviewers to the rows that deserve the closest look.
Then a person takes over. CVI verifies the flagged records manually and decides what stays and what gets replaced. As Ben Hilleli, CVI’s Director of Technology and Development, puts it, “By combining multiple different tools and angles of attack, we can scrutinize data in a fast and comprehensive way that would be infeasible without the technology.”
Software can scan thousands of rows without getting tired. People are better at context and at noticing the thing no rule anticipated. Using both means fewer good respondents get tossed by mistake and fewer bad ones slip through. Clients can also request a project disposition report showing start rates, completion metrics, and a summary of scrubbed records.
Beyond the Web
The problems aren’t limited to online surveys, and neither are CVI’s safeguards.
On the phone side, CVI runs a 60-station virtual call center in Toronto with bilingual French and English interviewers and supervisors. Supervisors monitor calls live and coach interviewers, so questions get asked consistently, and answers get recorded accurately.
When a study uses a client’s own customer list, the file is deduped, scrubbed, and given unique participant IDs. For email sends, CVI verifies online authentication, and runs a soft launch on a small slice of the sample. They use that soft launch to check on hard bounce and “complain rates,” or times that the email invitation is flagged as spam, against set thresholds before sending to everyone. It’s a bit like trying the haunted house out on a few brave friends before opening night.
In-home usage tests and central location tests carry their own risks, since real products end up in real hands. Every item, branded or unbranded, gets an individual reference tracking code so CVI can confirm each package has arrived. Full ingredient lists are required, which protects participants from undisclosed allergens and respects dietary choices like vegan, kosher, and halal.
The Foundation
All of this sits on top of industry governance and security. CVI is an accredited agency member of the Canadian Research Insights Council (CRIC), a corporate member of ESOMAR, and a long-standing member of the Insights Association. They’ve also signed the Global Data Quality (GDQ) Excellence Pledge.
On privacy, CVI complies with PIPEDA and CASL, and every staff member completes CIRA D-Zone Cybersecurity training. Project data is stored on secure, dedicated Canadian servers. Access to personally identifiable information is limited to those who need it, direct database access and exports require executive or Tech Lead permission, and automated systems monitor filesystems, network ports, and processes, blacklisting suspicious IP addresses and hosts automatically.
None of this is particularly flashy or exciting, but it’s the foundation that keeps everything else running smoothly.
Quality Comes From Partnership
Combining software controls and human-led checks stacks the game in your favour.. A mailed cheque confirms that the person is real. Device fingerprinting screens the connection. In-survey checks watch for attention. AI flags the odd text, and a human analyst makes the final call.
That’s how quality gets built into every phase of a project instead of being inspected at the end. After 45 years, CVI has a good sense of where things tend to go wrong and who needs to be watching when they do.
So, this October, enjoy the scares that come with the season. Your research data shouldn’t be one of them. With CVI, the point is better partnership; a team paying attention at every stage so you can spend Halloween on the candy.
