pub const MAX_DECODE_SPLITS: u32 = 16;Expand description
Hard ceiling on the split count, and the bound the split-K workspace is sized against.
A cap rather than a pure occupancy answer because the workspace is
rows * num_q_heads * num_splits * (head_dim + 2) F32 — linear in this
number — and because a model with very few q heads (an MQA decode head, a
draft model) would otherwise ask for a split per KV block. 16 covers
2 * 148 / 24 = 13 on the widest declared target
(kernels/b200, 148 SMs) with room, and an explicit
attn_decode_splitk = N is clamped to it.