Prompt Caching 101

Can JavaScript Become a Planned Runtime? — A study of AOT compilation, computation graphs, memory planning, and concurrency scheduling

Building current, a browser-based file sharing tool, with Rust and WASM

Butter Introduces Automatic Template Induction for LLM Response Caching

Butter, an HTTP proxy cache for LLM responses, has introduced automatic template induction for its response caching system. This new feature

CacheKit: High-Performance Cache Policies and Data Structures for Rust Systems

CacheKit is a Rust library providing high-performance cache replacement policies and supporting data structures for systems programming. It

xAI Launches Grok 4 Fast: Cost-Efficient Reasoning Model with 2M Token Context

xAI has launched Grok 4 Fast, a new cost-efficient reasoning model that builds on learnings from Grok 4. The model delivers frontier-level p

OpenAI rolls out improved memory system for ChatGPT to better retain user preferences across conversations

OpenAI is rolling out a new, more scalable memory system for ChatGPT that addresses challenges with staleness, correctness, and scalability