#llm inference
5 articles
Latest
Research
Five Papers Attack KV Cache Bottleneck With Quantization Methods
New approaches use statistical inference, rate-distortion theory, and learned eviction to reduce memory cost of long-context LLM inference.
3 days ago9 min read
Research
Three Papers Advance Quantization Methods for Efficient LLM Deployment
Researchers tackle post-training quantization bottlenecks that distort model behavior under memory and latency constraints.
6 days ago8 min read
Tools
Train-to-Test Scaling Rebalances LLM Compute Budgets
New framework optimizes total AI costs by accounting for inference-time scaling alongside training.
26 days ago1 min read
Tools
APEX MoE and TurboQuant Deliver 33% Faster LLM Inference
New quantization techniques accelerate both inference and prompt processing for local model deployment.
about 1 month ago3 min read
Research
TurboQuant Compresses LLM KV Cache to 3-4 Bits Without Accuracy Loss
New quantization algorithm enables longer context windows and 3.2× memory savings for local inference.
about 2 months ago7 min read