Sociodemographic Differences in Respondent Preferences for Survey Formats: Sampling Bias and Potential Threats to External Validity.
Szu-Wei Chen, Marian Keglovits, Megen Devine and 1 others
PMID 35282151WHAT IT FOUND
People with physical disabilities who are older, non-White, have lower income or education, live alone, or report poor health were significantly more likely to choose phone surveys over web-based formats.
Offering only one format risks excluding these groups from research.
Key findings
01Participants who were non-White, earned $10,008 or less annually, or lived alone were significantly more likely to choose phone surveys than web-based surveys.
02Phone survey users were older (mean 59.8 years) than web survey users (mean 57.2 years) and reported worse self-rated physical health.
03Offering only one survey format in a convenience sample may result in lower participation rates for certain sociodemographic groups, threatening external validity.
STILL TO COME
How it was doneWhat they found
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
The study is cross-sectional in its analysis of preferences, so causation cannot be inferred. Participants were recruited via convenience sampling from community organizations and social media, introducing potential sampling bias. The analysis did not control for type I error inflation due to multiple comparisons. The sample was restricted to English-speaking adults aged 45-65, limiting generalizability to other languages or age groups.
Declared interests
The study was approved by Washington University in St Louis Institutional Review Board. No specific funding sources or conflicts of interest were detailed in the provided text beyond the institutional approval.
The easy way to misread this
Do not assume that offering both phone and web options eliminates bias or ensures equal access. This study shows that preferences are strongly linked to sociodemographic factors like race, income, and health status, meaning certain groups may still be underrepresented if their specific barriers (e.g., digital literacy, cost of internet, need for human interaction) are not actively addressed.