Scaling LLMs Improves Social Simulation Fidelity in Most Cases, But Fails on Cognitive Biases
By
[Submitted on 2 Jul 2026]
Summary
This research paper investigates whether scaling up Large Language Models (LLMs) improves the fidelity of social simulations. Using scaling laws and a suite of 85 transformer LLMs (Qwen3 architecture) trained on DCLM web text, the authors study three sub-domains: opinion modeling, behavioral simulation, and longitudinal forecasting. They find strong compute scaling in most settings — behavioral and opinion simulation tasks improve rapidly with scale, especially for populations well-represented in English web corpora. However, longitudinal forecasting and underrepresented opinions scale more slowly. Notably, scaling fails to improve model calibration with human cognitive biases like risk aversion and certain heuristics. The authors conclude that scale will improve social simulations in most settings, but outliers exist and improvements are less reliable in low-resource domains.
Source
Twitter / XScaling LLMs Improves Social Simulation Fidelity in Most Cases, But Fails on Cognitive Biasesarxiv.orgKey quotes
· 5 pulledSurprisingly, we discover strong compute scaling in all three settings, using a suite of 85 transformer LLMs with the Qwen3 architecture pre-trained on the DCLM web text corpus under fixed-compute budgets from $10^{18}$ to $10^{20}$ FLOPs.
This reveals that the majority of behavioral and opinion simulation tasks will rapidly improve with scale, particularly when they involve populations that are well-represented in English web corpora.
In behavior simulation, scaling fails to improve model calibration with human cognitive biases like risk aversion, as well as human heuristics like learning correlated rewards from related tasks.
Taken together, we conclude that scale will improve social simulations in most settings, but outliers exist, and improvements will be less reliable in low-resource domains.
Longitudinal forecasting and underrepresented opinions scale more slowly, especially when they are less correlated with general knowledge and reasoning benchmarks like MMLU.
You might also wanna read
Scaling Laws Limit Reliability of Large Language Models, Study Finds
This research paper demonstrates that the scaling laws governing large language models (LLMs) fundamentally limit their ability to improve p
arxiv.org·10mo ago
Research Shows LLMs Develop Cognitive Degradation from Social Media Training Data
This research paper introduces the concept of 'LLM Brain Rot' - a phenomenon where large language models (LLMs) experience cognitive degrada
Exploring the Limitations of Language Models as World Models
The article argues that language models (LLMs) are not world models, despite their complexity and capabilities. The author provides examples
Study Reveals LLMs' Simulated Reasoning Abilities Are Fragile and Limited
Researchers found that large language models (LLMs) exhibit "simulated reasoning" abilities, which they describe as a "brittle mirage." The
arstechnica.com·11mo ago
Applying Distributed Systems Theory to Large Language Model Teams
The article proposes using distributed systems theory as a principled framework for creating and evaluating teams of large language models (
arxiv.org·29d agoApplying Distributed Systems Theory to Large Language Model Teams
The article proposes using distributed systems theory as a principled framework for creating and evaluating teams of large language models (
arxiv.org·29d ago
Experimental Study Reveals Ideological Biases in Leading Large Language Models
The article presents an experimental investigation into whether leading Large Language Models (LLMs) from companies like OpenAI and Google e
anomify.ai·8mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.