kquant_mmvq_pair_w

spark_model::layers::ops

Function kquant_mmvq_pair_w 

Source 
pub fn kquant_mmvq_pair_w(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    (w0, out0, n0): (DevicePtr, DevicePtr, u32),
    (w1, out1, n1): (DevicePtr, DevicePtr, u32),
    y_q8: DevicePtr,
    k: u32,
    m: u32,
    stream: u64,
) -> Result<()>
Expand description

kquant_mmvq_q2_k_pair_w: two [n_i, k] Q2_K projections of ONE activation (y_q8, block_q8_1 [m][k/32]) in one launch, out_i = [m][n_i] bf16. Bit-identical to two kquant_mmvq_w launches.