kquant_mmvq_groups_w

Function kquant_mmvq_groups_w 

Source
pub fn kquant_mmvq_groups_w(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    w_blocks: DevicePtr,
    y_q8: DevicePtr,
    out_bf16: DevicePtr,
    n: u32,
    k: u32,
    m: u32,
    n_groups: u32,
    stream: u64,
) -> Result<()>
Expand description

kquant_mmvq_q2_k_groups_w: n_groups row-blocks of one [n_groups * n, n_groups * k]-shaped projection, group g = weight rows [g*n, (g+1)*n) (w_blocks is [n_groups * n][k/256]) against activation columns [g*k, (g+1)*k) of the block_q8_1 [m][n_groups * k / 32] row y_q8, written to columns [g*n, (g+1)*n) of out_bf16 ([m][n_groups * n]). Bit-identical to n_groups launches of kquant_mmvq_w over the slices.