Sequential KV Cache Compression Using Probabilistic Language Tries and Predictive Delta Coding

Recent work on KV cache quantization, culminating in TurboQuant, has approached the Shannon entropy limit for per-vector compression of transformer key-value caches. We observe that this limit applies to a strictly weaker problem than the one that actuall

0

arxiv.org2mo ago