Compressing Prompts: A New Approach to LLM Efficiency
A novel method suggests compressing task-relevant information into a single activation vector. This could lead to more efficient large language models without significant accuracy loss.
Read the full articleYou might also wanna read
PromptEmbedder: A Dual-LLM Framework for Efficient, Architecture-Agnostic Text Embedding
Large Language Models (LLMs) have demonstrated remarkable efficacy in text embedding, yet current adaptation methods like LoRA face signific
arxiv.org·1mo ago
RAGP: A Graph-Based Prompt Compression Method Using Lévy Walk-Guided Pruning
Existing prompt compression methods treat text as flat token sequences, failing to capture the distributed nature of important information,
arxiv.org·7d ago
Expected Attention: KV Cache Compression Method for Efficient LLM Inference
Memory consumption of the Key-Value (KV) cache represents a major bottleneck for efficient large language model inference. While attention-s
arxiv.org·9mo ago
LK Losses: A New Training Objective to Optimize Acceptance Rate in Speculative Decoding for LLMs
Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate
arxiv.org·1mo ago
Context Tuning: Efficient LLM Adaptation via Direct Memory Representation Optimization
Context Tuning directly optimizes an LLM's memory representation for efficient adaptation without updating model weights.
agenticlearning.ai·1d ago
Verbalized Sampling: A Training-Free Method to Mitigate Mode Collapse and Improve LLM Output Diversity
Post-training alignment often reduces LLM diversity, leading to a phenomenon known as mode collapse. Unlike prior work that attributes this
arxiv.org·13d agoVerbalized Sampling: A Training-Free Method to Mitigate Mode Collapse and Improve LLM Output Diversity
Post-training alignment often reduces LLM diversity, leading to a phenomenon known as mode collapse. Unlike prior work that attributes this
arxiv.org·13d ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.