Adaptive LLM Routing Using Contextual Bandits and Shared Embedding Space
By
tdchaitanya
Not artisan, but a perfectly fine bagel. Hits the spot.
Summary
This research paper proposes a novel approach to LLM routing that treats it as a contextual bandit problem rather than supervised learning. The authors develop PILOT (Preference-prior Informed Linucb fOr adaptive rouTing), which creates a shared embedding space for queries and LLMs, initially learned from offline human preference data and refined through online bandit feedback. The system also includes an online cost policy modeled as a multi-choice knapsack problem to handle diverse user budgets for resource-efficient routing.
Key quotes
· 5 pulledLLM routing addresses this by dynamically selecting the most suitable LLM for each query/task
We thus propose to study LLM routing as a contextual bandit problem, enabling adaptive decision-making using bandit feedback
We develop a shared embedding space for queries and LLMs, where query and LLM embeddings are aligned to reflect their affinity
This space is initially learned from offline human preference data and refined through online bandit feedback
We introduce an online cost policy modeled as a multi-choice knapsack problem, ensuring resource-efficient routing
You might also wanna read
Contextual Rollout Bandits: A Neural Scheduling Framework for Efficient Reinforcement Learning with Verifiable Rewards
This paper introduces Contextual Rollout Bandits, a novel framework for Reinforcement Learning with Verifiable Rewards (RLVR) that addresses
arxiv.org·4d ago
ModelPilot: Intelligent LLM Router Optimizes AI Model Selection for Cost, Speed, Quality, and Environmental Impact
ModelPilot is an intelligent LLM router that automatically selects the optimal AI model for each prompt based on cost, latency, quality, and
Product Hunt·6mo ago