Masters Thesis: Online Survey Design

Motivation: Online surveys and participant recruitment through crowdsourcing platforms such as Amazon Mechanical Turk (MTurk) have become increasingly popular among academic researchers. While online survey design has been extensively studied, there is little research in the context of crowdsourcing platforms; furthermore, there are certain visual design features such as using the researcher’s logo or using a background that thus far have been mostly ignored by online survey methodology literature.

Objective: To examine if the researcher’s logo or survey design template affect respondents.

Methods: Online experiment manipulating two visual design factors for online surveys: 1.) using a logo or not using a logo, and 2.) using a dark background or not using a dark background. Participants (N = 448) were recruited via MTurk and completed one of the experimental condition surveys on Qualtrics, an online survey design and hosting platform.

Main results: Provides evidence for the effect of the survey background on social desirability bias and emotional effect, while using the researcher’s logo had no effect. Using a logo and no background led to significantly lower drop- outs compared to the three other experiment conditions. Demonstrates the high quality of data that can be obtained on MTurk.

Main contribution: Expanded framework for evaluating online survey design factors and effects on respondents. Online survey design recommendations for specific visual design factors, specifically, background and researcher logo. Further evidence for the viability of conducting research and experiments, particularly research on online survey design, on MTurk.

Conclusion: This study demonstrates both the feasibility of conducting research on MTurk and what visual design factors of online surveys may affect the way participants respond. It emphasizes the importance of online survey design and the need for researchers to openly discuss their visual design choices in publication, as these choices may have had an effect on the way participants responded to their surveys.