Verasight releases new study on the limits of synthetic survey data across different topics

Bar chart titled “Mean Absolute Error by Question Category” from Verasight’s Synthetic Sampling Report IV: Can Large Language Models Replicate Survey Data Across Topics? (January 2026). The chart shows mean absolute error for single-answer survey question

Mean absolute error by question category for single-answer questions. Lower values indicate betterLLM performance.

Researchers were invited to submit survey questions that were fielded to a nationally representative sample of 2,000 U.S. adults and a matched synthetic sample.

While synthetic samples can approximate some politically polarized attitudes, they struggle with questions rooted in individual experience.”

— G. Elliott Morris, lead author of the report

SAN FRANCISCO, CA, UNITED STATES, January 22, 2026 /EINPresswire.com/ -- Verasight announced the release of its latest whitepaper, Synthetic Sampling Report IV: Can Large Language Models Replicate Survey Data Across Topics?, the fourth study in its ongoing evaluation of large language model generated synthetic survey data.

The study compares responses from 2,000 nationally representative U.S. adults to a matched dataset generated by a state-of-the-art LLM. The analysis assesses whether synthetic samples can reliably replicate human survey responses across political and non political topics.

Researchers from the polling and market research community were invited to submit survey questions for inclusion in the study. These questions covered politics, health care, technology, education, daily life, and consumer behavior. Each question was fielded to both human respondents and a synthetic sample using identical demographic inputs.

The report finds substantial variation in accuracy by topic and question type. Across single answer questions, the mean absolute error between human and synthetic responses was 14.5 percentage points. Political questions showed lower error, while questions related to lived experience and personal behavior showed significantly higher divergence.

The analysis also finds that LLMs performed poorly on multi answer survey questions, often failing to select response options chosen by large shares of human respondents. Due to the severity of these errors, multi answer questions were excluded from the report’s main accuracy analysis.

“While synthetic samples can approximate some politically polarized attitudes, they struggle with questions rooted in individual experience,” said G. Elliott Morris, lead author of the report.

“These findings suggest researchers should be extremely cautious about using synthetic data for their research." said Benjamin Leff, Verasight co-founder and Chief Executive Officer.

The full whitepaper is available here.

For more information about Verasight, visit www.verasight.io.

About Verasight: Founded by academic researchers, Verasight enables leading institutions to survey any audience of interest (e.g., engineers, doctors, policy influencers). From academic researchers and media organizations to Fortune 500 companies, Verasight is helping our client stay ahead of trends in their industry.

Verasight Media Department
Verasight
contact@verasight.io
Visit us on social media:
LinkedIn
X

Legal Disclaimer:

EIN Presswire provides this news content "as is" without warranty of any kind. We do not accept any responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you have any complaints or copyright issues related to this article, kindly contact the author above.