w8a16_gemv_batch16_ncol2_strided

Function w8a16_gemv_batch16_ncol2_strided 

Source
pub fn w8a16_gemv_batch16_ncol2_strided(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    input: DevicePtr,
    weight: DevicePtr,
    block_scale: DevicePtr,
    output: DevicePtr,
    m: u32,
    n: u32,
    k: u32,
    a_row_stride: u32,
    c_row_stride: u32,
    stream: u64,
) -> Result<()>
Expand description

N_COLS=2, explicit A and C row pitches in ELEMENTS — for the multi-seq [n, per_seq_qkv] decode buffer, exactly as w8a16_gemv_batch16_strided.

Kernel: w8a16_gemv_batch16_ncol2_strided (module w8a16_gemv_ncol).