kquant_swiglu_q8_1_rows

Function kquant_swiglu_q8_1_rows 

Source
pub fn kquant_swiglu_q8_1_rows(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    gate: DevicePtr,
    up: DevicePtr,
    w: DevicePtr,
    h_bf16: DevicePtr,
    out_q8: DevicePtr,
    rows: u32,
    inter: u32,
    limit: f32,
    stream: u64,
) -> Result<()>
Expand description

moe_v41_swiglu and kquant_q8_1_rows in one launch (kquant_swiglu_q8_1_rows_bf16): h[rows, inter] = bf16(swiglu(gate, up) * w[row]), then the q8_1 blocks of those bf16 rows; byte-identical to the two launches. w may be null. inter % 32 == 0.