pub fn kquant_swiglu_q8_1_rows(
gpu: &dyn GpuBackend,
kernel: KernelHandle,
gate: DevicePtr,
up: DevicePtr,
w: DevicePtr,
h_bf16: DevicePtr,
out_q8: DevicePtr,
rows: u32,
inter: u32,
limit: f32,
stream: u64,
) -> Result<()>Expand description
moe_v41_swiglu and kquant_q8_1_rows in one launch
(kquant_swiglu_q8_1_rows_bf16): h[rows, inter] = bf16(swiglu(gate, up) * w[row]), then the q8_1 blocks of those bf16 rows; byte-identical to the two
launches. w may be null. inter % 32 == 0.