Anthropic Research Reveals Claude's Expressed Values Shift Across Languages and Models

A new study from Anthropic has found that its AI assistant Claude does not express consistent values across different languages or model versions, raising fresh questions about how alignment works in multilingual AI systems. The research, published Monday, analyzed a large dataset of anonymized user conversations to see how Claude's responses varied by context.

According to Decrypt, Anthropic researchers examined 309,815 anonymized conversations with Claude involving subjective tasks. The study found that Claude's behavior changed depending on which model users chose and the language they spoke, with distinct patterns emerging. OfficeChai reported that the research specifically looked at values such as deference and warmth.

OfficeChaiClaude’s Values Such As Deference And Warmth Vary By Language, Finds Anthropic Study

"Arabic responses tended to be more deferential, while English responses placed greater emphasis on caution. Claude was warmest in Hindi and Arabic, using more polite, playful, and encouraging language."

2 sources

decrypt.coClaude's Personality Changes Depending on the Model—And the Language ...demo.decrypt.coClaude's Personality Changes Depending on the Model—And the Language ...

These language-specific shifts highlight how cultural and linguistic training data can shape an AI's expressed values, a finding that has implications for global deployment of such tools.

The study's implications reach beyond simple behavioral variation. Glen Rhodes, writing on glenrhodes.com, called the findings "genuinely uncomfortable" and argued they should worry everyone working on AI alignment. The outlet noted that Claude's values shift not only by language but also across model versions, making it difficult to pin down a consistent ethical framework.

glenrhodes.comCritique of Anthropic’s finding that Claude’s expressed values shift by language and model version, and what it means for alignment as a concept

"Claude doesn't express consistent values across languages or even across model versions."

This inconsistency, according to the critique, challenges the idea that alignment can be a one-size-fits-all process and suggests it may need to be language- and context-specific.

Blockchain.news reported that the research offers insights into AI behavior and training implications, while BitRss noted that the findings apply to both the language spoken and the specific model in use. Anthropic itself, in its own write-up, referenced earlier work that analyzed 700,000 conversations to study how Claude's values change in different contexts. Together, the research underscores the complexity of building an AI that behaves ethically and consistently across the diverse linguistic and cultural landscapes of its users.

2 sources

BitRssClaude's Personality Changes Depending on the Model—And the Language You Speakblockchain.newsAnthropic Analyzes Claude's Value Variations Across Models