Experimental Sampling of LLaMA Language Model at Negative Temperature Yields Bizarre Results

Summary

This article explores an experimental approach to sampling language models (specifically LLaMA) at negative temperatures, inspired by statistical mechanics concepts. The author applies the Boltzmann distribution from thermodynamics to language model sampling, where temperature parameter T controls randomness in text generation. By setting T=-0.001 (negative temperature), the experiment produces maximally weird and unexpected outputs, demonstrating how negative temperatures in statistical mechanics correspond to inverted probability distributions that favor high-energy states. The article explains the theoretical background of temperature in statistical mechanics, connects it to language model sampling parameters, and presents experimental results showing how negative temperature sampling leads to bizarre, high-energy text outputs that defy normal language patterns.