#long context

Five Papers Attack KV Cache Bottleneck With Quantization Methods

New approaches use statistical inference, rate-distortion theory, and learned eviction to reduce memory cost of long-context LLM inference.