RICP: A Teacher-Student Framework for Retrieved In-Context Principles from Mistakes in LLMs
By
@ai-firehose.column.social
A second-rack bagel that's nearly first-rack. Tasty stuff.
Summary
This paper introduces Retrieved In-Context Principles (RICP), a novel teacher-student framework for improving Large Language Models (LLMs) through in-context learning. Unlike existing approaches that lack customization and error coverage, RICP has the teacher model analyze mistakes from the student model, cluster them by underlying reasons to create task-level principles, and retrieve the most relevant mistakes per question for customized guidance. The framework is orthogonal to existing prompting methods and requires no teacher intervention during inference. Experimental results across seven reasoning benchmarks show RICP enhances performance when applied to various prompting strategies.
Key quotes
· 5 pulledIn RICP, the teacher model analyzes mistakes from the student model to generate reasons and insights for preventing similar mistakes.
These mistakes are clustered based on their underlying reasons for developing task-level principles, enhancing the error coverage of principles.
During inference, the most relevant mistakes for each question are retrieved to create question-level principles, improving the customization of the provided guidance.
RICP is orthogonal to existing prompting methods and does not require intervention from the teacher model during inference.
Experimental results across seven reasoning benchmarks reveal that RICP effectively enhances performance when applied to various prompting strategies.
You might also wanna read
Recursive Language Models: A New Approach for Processing Extremely Long Prompts Beyond Standard Context Windows
Researchers propose Recursive Language Models (RLMs), a novel inference strategy that enables large language models to process prompts far b
arxiv.org·4mo ago
Comprehensive Survey of Reasoning Failures in Large Language Models
This article presents a comprehensive survey of reasoning failures in Large Language Models (LLMs), introducing a novel categorization frame
arxiv.org·3mo ago
From Prompt Engineering to Context Engineering: Evolving LLM Inference Approaches
The article discusses the evolution from prompt engineering to context engineering in LLM applications. As LLMs transition from conversation
chrisloy.dev·7mo ago
Strategies for Mitigating Context Failures in LLM Applications
This article provides practical strategies for mitigating and avoiding context failures in large language model applications, focusing on in
dbreunig.com·9mo ago
The Four Pillars of Effective LLM Prompting: Intent, Guidance, Translation, and Analysis
The article discusses effective prompting strategies for large language models (LLMs), organized around four key pillars: articulating inten
Common Anti-Patterns to Avoid When Working with Large Language Models
The article discusses common anti-patterns to avoid when working with Large Language Models (LLMs), based on 15 months of experience. It ide
instavm.io·6mo ago