BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design

媒体报道 Hugging Face Papers 原文语言:中文

原文摘录

论文摘要(作者报告,未经本站复测):The key-value (KV) cache dominates memory capacity and bandwidth in long-context LLM serving. While 4-bit microscaling formats (NVFP4, MXFP4) shrink the cache and run attention on FP4 Tensor Cores, a fundamental gap remains between algorithmic quantization and system-aware acceleration: existing FP4 KV systems use vanilla quantization that causes severe accuracy drops, whereas…

以下是来源提供的简短摘录,完整内容请查看原文。

参考来源

← 返回每日AI报告