pub const SPLITK_TARGET_WAVES: u32 = 2;Expand description
Waves of CTAs the auto policy aims to put on the device at the
single-stream shape.
TWO, not one: the attention CTAs are memory-latency bound (a paged K/V
gather, a shuffle reduction and an __expf per position), so one CTA per
SM leaves the SM stalled on loads. Two resident waves is the smallest
number that lets one cover the other’s misses, and it is also where the
split stops being free — every extra split is another partial for the
reduce to merge and another eighth-of-a-CTA of Q load to repeat. Round 14
measures the curve: the microtest prints GB/s for num_splits 1/2/4/6 at
three context lengths.