Evolution Fine-Tuning: Using LLMs to Learn and Transfer Knowledge Across 371 Optimization Tasks
By
Young-Jun Lee ,
Summary
This paper introduces "Evolution Fine-Tuning" (EFT), a novel approach that uses Large Language Models (LLMs) integrated with evolutionary search to solve optimization tasks across 371 diverse problems. Unlike prior work that approaches each optimization task from scratch, EFT enables LLMs to accumulate and transfer experience across tasks, learning to discover solutions more effectively. The approach has produced state-of-the-art results on open mathematical conjectures, GPU kernel design, scientific law discovery, and combinatorial puzzles. The key innovation is that the model retains and builds upon search experience rather than discarding it after each attempt.
Source
Twitter / XEvolution Fine-Tuning: Using LLMs to Learn and Transfer Knowledge Across 371 Optimization Taskshuggingface.coKey quotes
· 3 pulledWould experience designing faster GPU kernels also help close in on a long-standing open mathematical conjecture?
Large Language Models (LLMs) integrated into evolutionary search have recently produced state-of-the-art solutions on optimization tasks, including open mathematical conjectures, GPU kernel design, scientific law discovery, and combinatorial puzzles.
Prior work applied search scaffolds to one target task at a time, so every new problem is approached from scratch and the experience accumulated during search is discarded once the model finishes its attempt.
You might also wanna read
OpenEvolve: Combining LLMs with Evolutionary Search for Algorithm Discovery
OpenEvolve is an open-source evolutionary coding agent that integrates large language models (LLMs) into a quality-diversity search framewor
algorithmicsuperintelligence.ai·6mo ago
Bidirectional Evolutionary Search: A New Framework for Self-Improving Language Models
This paper introduces Bidirectional Evolutionary Search (BES), a novel search framework for self-improving language models that addresses li
arxiv.org·1mo ago
New Framework Formalizes Learning from Language Feedback with Provable Performance Guarantees
This paper formalizes the Learning from Language Feedback (LLF) problem, providing a principled framework for interactive learning using lan
arxiv.org·23d ago
EvoTrainer: A Framework for Co-Evolving LLM Policies and Training Harnesses in Autonomous Agentic Reinforcement Learning
The article introduces EvoTrainer, an autonomous training framework for LLMs that goes beyond traditional recipe search by co-evolving both
arxiv.org·1mo ago
Evolution of Large Language Model Architectures: A Critical Comparison
The article discusses the evolution of large language model (LLM) architectures from GPT-2 to newer models like DeepSeek-V3 and Llama 4, que
magazine.sebastianraschka.com·11mo ago
LLM-Based Framework Translates Natural Language into Spacecraft Trajectory Optimization Code
This paper presents a framework that uses large language models (LLMs) to translate natural language descriptions of mission requirements an
arxiv.org·29d ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.