AI · 1h ago
LOCKS technique cuts KV cache reads 10x at 1M context
LOCKS uses per-page SVD to compress KV cache summaries, reducing memory reads by 10x at 1M token context. It avoids the structural failure of shared-basis methods like ShadowKV by storing page-specific centroids and singular vectors. The technique achieves theoretical guarantees on attention mass coverage while using only 10% of original data.
Meridian48 take
A clever fix for a fundamental flaw in sparse attention, but real-world speedups depend on hardware and implementation details.
kv-cachesparse-attention