Sequential KV Cache Compression Using Probabilistic Language Tries and Predictive Delta Coding

Recent work on KV cache quantization, culminating in TurboQuant, has approached the Shannon entropy limit for per-vector compression of transformer key-value caches. We observe that this limit…