Turing-RL: A Reinforcement Learning Approach for Training User Simulators Using Turing Test Rewards
By
[Submitted on 17 Jun 2026]
Summary
This paper introduces Turing-RL, a novel reinforcement learning approach for training user simulator models that can mimic human users in interactive settings. Unlike existing methods that train LLMs to match a single ground truth response using log probability or similarity rewards, Turing-RL uses a discriminative Turing reward with an LLM judge to score how indistinguishable a generated response is from a real user's response. The approach was tested across conversational chat and Reddit forum discussion domains, consistently outperforming baseline methods on both LLM and human evaluation metrics. The study suggests that optimizing for indistinguishability rather than direct response matching is more effective for learning user simulators.
Source
bskyTuring-RL: A Reinforcement Learning Approach for Training User Simulators Using Turing Test Rewardsarxiv.orgKey quotes
· 4 pulledWe instead propose {Turing-RL}: a Turing-Test-based reinforcement learning approach for training user simulator models.
{Turing-RL} uses a discriminative Turing reward with an LLM judge to score how indistinguishable a generated response is from the real user's given the user's history.
Across two different domains--conversational chat and Reddit forum discussion--we find that {Turing-RL} consistently outperforms baseline methods on both LLM and human evaluation metrics.
Our study suggests that optimizing for indistinguishability, rather than response matching, is effective for learning user simulators.
You might also wanna read
Reinforcement Learning to Train Large Language Models to Explain Human Decisions
ROTE: Modeling Human Behavior as Executable Programs for Improved AI Prediction
This research paper introduces ROTE, a novel algorithm that models human behavior as executable behavioral programs rather than traditional
arxiv.org·8mo ago
Exploring RLHF on every prompt for local coding models
A Hacker News user explores the idea of using Reinforcement Learning from Human Feedback (RLHF) on every prompt with a medium-sized local mo
news.ycombinator.com·3d ago
Investigating the RYS Method: Testing Layer Duplication Across Modern LLMs
This article explores the RYS (Repeat Your Self) method discovered in Part 1, where duplicating seven middle layers in Qwen2-72B without wei
dnhkng.github.io·2mo ago
Terminal-Bench-RL Project Advances Terminal Agent Training with Reinforcement Learning
The article discusses the Terminal-Bench-RL project, which extends the rLLM framework by UC Berkeley Sky Lab to train long-horizon terminal
github.com·10mo ago
2025 LLM Paradigm Shifts: Key Technological Advances in Large Language Models
The article provides a comprehensive review of major paradigm shifts in Large Language Models (LLMs) throughout 2025, highlighting key techn
karpathy.bearblog.dev·6mo ago