pub const GATEUP_FUSED_MAX_M: usize = 16;Expand description
The widest M the FUSED dense-FFN gate+up decode GEMM serves (#927), and
therefore the row extent ffn_gate_up_fused is sized for.
16 — the top of the decode band. The fused arm is a per-LAUNCH saving, and
the launch overhead it removes is only material while the GEMM is
weight-bandwidth bound; at the prefill widths the same two projections
already run at 68.6% of FP8 peak (nsys round 13, M=4576), where a launch
costs nothing measurable. 16 is also the largest batch H100 round 13
captured (Captured CUDA graph for batch size 16).
DECLARED HERE because the arena is sized in this crate and the dispatch
rule lives above it; spark_model::layers::dense_ffn_gateup_fused reads
THIS constant rather than restating it, so the band and the buffer cannot
disagree.