Experiments on the design and evaluation of complex survey questions

Complex questions may appeal to survey researchers because they maximize the specificity of data while minimizing the number of distinct data items collected. This chapter provides examples of experiments designed to inform particular questionnaire design decisions in which the available guidance was limited or ambiguous. Each of the experiments begin with a dilemma involving draft questions proposed for national surveys, in which there was more than one plausible way to craft the question, and limited guidance regarding which was preferable. Initially, the dilemmas involved the optimal structure of questions that were long and grammatically complex, potentially including detailed descriptions and definitions, examples, explanations, multiple clauses, qualifiers, or including specific response categories. As the survey research progressed, it moved from matters of structural complexity to conceptual complexity - for example, considering the optimal way to manage questions containing disparate concepts, or ambiguity of question intent arising from context.