ReOPD: Making AI Training Faster and More Efficient
Replayed-Prefix On-Policy Distillation (ReOPD) offers a breakthrough in AI training, reducing costs and improving efficiency by reusing pre-collected data.
Read the full articleYou might also wanna read
DOPD: A Dual On-policy Distillation Method to Address Privilege Illusion in LLM and VLM Training
On-policy distillation (OPD) offers superior capacity transfer by supervising student-sampled trajectories with dense token-level signals. T
arxiv.org·11d ago
Feedback Distillation: A New Training Method for Improving LLM Reasoning in Theorem Proving
Post-training for reasoning models typically combines supervised fine-tuning with reinforcement learning from verifiable rewards, most commo
arxiv.org·1mo ago
RLCSD: A Contrastive Self-Distillation Method to Fix Style Drift in Reasoning Models
On-policy self-distillation (OPSD) provides dense, token-level supervision for reasoning models by aligning a model's own distribution with
arxiv.org·28d ago
The Rise of AI Distillation Amid High Training Costs
AI inference for 90% lower cost
inference.net·11mo ago
The Case for a Strategic U.S. Policy Response to Adversarial AI Distillation
In February 2026, Anthropic disclosed that roughly 24,000 fraudulent accounts had bombarded its Claude model with 16 million interactions, l
Proxy-KD: A Novel Method for Knowledge Distillation from Black-Box Large Language Models
Given the exceptional performance of proprietary large language models (LLMs) like GPT-4, recent research has increasingly focused on boosti
arxiv.org·12d ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.