auto_splits

Function auto_splits 

Source
pub fn auto_splits(sm_count: u32, num_q_heads: u32) -> u32
Expand description

clamp(ceil(WAVES * sm_count / num_q_heads), 1, MAX_DECODE_SPLITS).

The single-stream occupancy answer: at C=1 the grid is (num_q_heads, num_splits, 1), so this is the split count that puts WAVES * sm_count CTAs on the device with one sequence in flight. At a wider batch the same count over-subscribes — total WORK is unchanged, only its partition is — which is the trade this policy takes deliberately: a C=1 step is 24 CTAs without it and a C=16 step is already 3 waves with it.