Synthetic Sampling Index
Measure frontier AI model performance in simulating human behavior. Applications include political science, healthcare, management, human-computer interaction (HCI), market research, and UX research.

Synthetic Sampling Reports

Did Synthetic “Polls” Beat Real Surveys in the 2026 Senate Elections? No.
The headline is that 2026 was a much better year for Democrats than synthetic data suggested. The polls did better than synthetic data.
Read Report VI →
Depth at a Cost: A Randomized Comparison of AI-Moderated Interviews and Written Survey Open-Ends
AI-moderated interviews produced far richer answers than written survey open-ends, from a smaller and less representative set of respondents.
Read the study →
Can AI “digital twins” replace human respondents?
Policy tradeoffs, institutional questions, post-training events, and topics with weak partisan signal are all hard for models to approximate.
Read Report V →
Can Large Language Models Replicate Survey Data Across Topics?
Our first conclusion is that the LLMs did a very poor job at multi-response questions.
Read Report IV →LLMs Misread Real Consumer Behavior
For questions about coffee, the mean absolute error was 19.8 points. LLM-generated market research data is not simply noisy but directionally misleading.
Read Report III →The Limits of Synthetic Samples in Survey Research
When using LLMs, it is critical to remember that verified accuracy in one area does not necessarily generalize to accuracy in other areas.
Read Report II →
Your Polls On ChatGPT
Overall, the performance of our “synthetic sample” is too poor to be useful for all of our research questions.
Read Report I →Synthetic Sampling Leaderboard
Model comparison preview
Swipe to see more →
| Model | Overall agreement | Subgroup gap | Out-of-sample error |
|---|---|---|---|
| Claude Fable 5.2 | Pending | Pending | Pending |
| Claude Opus 5.5 | Pending | Pending | Pending |
| GPT-6-sol | Pending | Pending | Pending |
Get the answers you need, from the people who matter.
Solutions
Take a survey