Online research, particularly studies using survey methodologies, is a popular research method for many research studies. However, in recent years, it is now common for researchers to encounter bots, low-quality responses, or outright fraud. The rapid increase in bots makes it crucial for researchers to remain diligent and develop quality attention checks to identify fraud, while still being mindful not to overburden actual human participants. Below are some suggestions for researchers who wish to collect data online. 

• Add tools like CAPTCHA/reCAPTCHA. UW Qualtrics and REDCap both have this feature. 

• Include attention check questions. For example, “Select “Disagree” for this question.” or “Please type the word “blue” here.” These tend to catch automated or inattentive responses. 

• Add branching logic to the survey. It can be hard for bots to follow branching logic. 

• Consider hidden questions/fields that will not be viewable by a human participant, but often bots detect these hidden items and provide a response. 

• Include follow-up questions tied to earlier questions but framed a different way (e.g. “What is your date of birth” vs “Enter your date of birth”) 

The points above are especially important for surveys that are going to be advertised publicly. 

• Develop the survey so that it is unique to the participant. Each participant is emailed a unique link to the survey. Additional verification tools can be used as well, such as SMS/text verification, if appropriate/applicable to the study. 

• Incentives increase fraud risk. If you are going to compensate participants, you may want to consider using one of the verification methods noted above, or delay payment until verification. 

• It is important that if there is an incentive, the consent document indicate something like “Eligibility for compensation will require passing data quality checks to ensure valid participation.” 

• Be diligent and monitor responses: 

• Bots have a difficult time with open-ended questions. Review the open-ended questions for non-sensical text or copy/paste patterns. 

• Bots or fraudulent responses tend to complete surveys in rapid time, have the same answer for every question, or identical or near identical answer patterns. Check for these things later when reviewing the data. 

• Check IP addresses and emails, if collected. Many time fraudulent responses will have the same email/IP/device info. Restrict the survey to one response per IP/device/browser. 

No single method is foolproof. It is recommended you use a number of these tools, as multiple techniques will make fraud difficult, detectable, and removeable.