pub const DECODE_GQA_PACK_DECLARED: bool = false;Expand description
The packed kernels are OFF unless explicitly armed.
Declared here rather than in kernels/<hw>/HARDWARE.toml because there is
no receipt for this arm on any target yet: it is a launch-geometry change
whose payoff (DECODE_GQA_PACK_WIDTHx fewer KV load instructions) trades
against a measured occupancy cost — ptxas reports 227 registers for the FP8
twin and 243 for the BF16 one, zero spills, i.e. ONE resident CTA per SM,
against 48/56 registers and up to five for the unpacked kernels. That trade
can only be settled by an A/B on the box. AVAROK_ATTN_DECODE_GQA_PACK=1
is how that A/B is run — the same mechanism AVAROK_ATTN_DECODE_SPLITK=auto
uses for the policy above. A target that wins the A/B moves this to a
[defaults] row with the receipt beside it.