[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$f2qnfw4hptgk1p":3},{"code":4,"message":5,"data":6,"timestamp":32},10000,"success",{"full_name":7,"position":8,"description":9,"language":10,"stars":11,"forks":12,"stars_gained":13,"age_days":14,"hn_threads":15,"one_liner":22,"summary_md":23,"topic":24,"github_topics":27,"license":28,"homepage":29,"updated_at":30,"history":31},"deepseek-ai\u002FFlashMLA",19,"FlashMLA: Efficient Multi-head Latent Attention Kernels","C++",13180,1232,247,610,[16],{"id":17,"title":18,"points":19,"comments":20,"posted_at":21},43155023,"DeepSeek Open Source FlashMLA – MLA Decoding Kernel for Hopper GPUs",441,108,"2025-02-24","Production attention kernels for DeepSeek's MLA and sparse attention, aimed at teams serving DeepSeek models on modern NVIDIA or Ascend hardware.","## The problem\nServing DeepSeek-style multi-head latent attention at scale is not something a generic attention implementation handles well. The KV cache is large, the decode path is bandwidth-bound, and the quantized FP8\u002FFP4 formats the model expects are not what stock PyTorch or generic FlashAttention kernels consume. FlashMLA is DeepSeek's own kernel library for this: prefill, decode and backward attention kernels, plus a dense MHA path, tuned for its V4.1 model on NVIDIA SM100\u002FSM103 and Huawei Ascend 950.\n\n## What's different\nThis is a vendor-authored kernel library rather than a general-purpose attention project. Two things stand out. First, it tracks a moving model target: the sparse kernels implement DeepSeek Sparse Attention and only accept FP8 or FP4 KV caches in DeepSeek's own byte layout — unquantized bf16 KV cache is explicitly unsupported. Second, it fuses Q-norm, Q-RoPE, core attention, O-RoPE and the FP8 cast into a single kernel, removing launch overhead from the small surrounding ops at roughly the same throughput. Compared with generic kernels like FlashAttention, you get tighter integration and higher numbers, but you give up model and hardware generality.\n\n## Who it's for\nInference engineers running DeepSeek V4.1 (or maintaining a fork for older V3\u002FV4 versions) on B200-class Blackwell GPUs or Ascend 950 NPUs. It fits teams building their own serving stack who are willing to wire metadata and KV-cache layouts themselves. Look elsewhere if you serve other models, run on Hopper or older GPUs, need a plug-and-play library, or use bf16 KV cache — none of those are supported here.\n\n## Cost of adoption\nNarrow and demanding: SM100\u002FSM103 NVIDIA GPUs with CUDA 13.1+ (13.2+ recommended), or Ascend 950 with CANN 9.2.0+, plus PyTorch 2.10+ and `torch_npu` on Ascend. The build must run without isolation (`pip install -v . --no-build-isolation`) because setup.py imports torch at build time, and a register-spill check can fail the CUDA build. The 2026.09.30 release is a hard break: Hopper and V3\u002FV3.2\u002FV4.0 support were removed and the KV cache format changed, so older deployments must pin an earlier commit. The README is detailed but assumes familiarity with attention internals.\n\n## Verdict\nFlashMLA is a real, actively maintained piece of DeepSeek's inference stack, not a showcase repo: releases have shipped since early 2025 and the project keeps pace with each new model. Interest is steady rather than spiking — about 13.2k stars and a monthly C++ ranking bump of +247, with no daily trending appearance. The 441-point HN thread dates from the first release, so the attention is old and the current activity is incremental. Adopt it if you are serving DeepSeek models on the exact hardware listed; otherwise it is a useful read, not a dependency.",{"slug":25,"name":26},"machine-learning","Machine Learning",[],"MIT",null,"2026-10-25T00:35:10Z",[],1792901794]