Leveraging model distillation to fine-tune a model
Source
OpenAILeveraging model distillation to fine-tune a modelopenai.comYou might also wanna read
Fine-Tuned Small LLMs Outperform Larger Models at 5-30x Lower Cost with Data Curation
The article discusses how fine-tuned small language models (LLMs) can outperform larger ones at significantly lower costs (5-30x) through pr
tensorzero.com·11mo ago
Evolution Fine-Tuning: Using LLMs to Learn and Transfer Knowledge Across 371 Optimization Tasks
This paper introduces "Evolution Fine-Tuning" (EFT), a novel approach that uses Large Language Models (LLMs) integrated with evolutionary se
Proxy-KD: A Novel Method for Knowledge Distillation from Black-Box Large Language Models
This paper introduces Proxy-KD, a novel knowledge distillation method for transferring capabilities from black-box large language models (li
arxiv.org·6d ago
Study reveals why in-context learning fails on complex specification-heavy tasks and how fine-tuning can help
This research paper investigates the limitations of in-context learning (ICL) for large language models (LLMs) when applied to specification
arxiv.org·14d ago
LLM-Deflate: Reversing Model Training to Extract Structured Datasets from Large Language Models
LLM-Deflate is a novel technique that reverses the training process of Large Language Models by systematically extracting structured dataset
scalarlm.com·9mo ago
RegMix-D: A Dynamic Data Mixing Method for LLM Pretraining Using Proxy Training Trajectories
RegMix-D is a new method for dynamic data mixture selection in Large Language Model pretraining. It extends the static RegMix approach by le
arxiv.org·2d ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.