decode_w8a8_gemm

Function decode_w8a8_gemm 

Source
pub fn decode_w8a8_gemm(
    scratch: &DecodeW8a8Scratch,
    fp8w: &Fp8Weight,
    out: DevicePtr,
    plan: &DecodeW8a8Plan,
    stream: u64,
) -> Result<()>
Expand description

out[rows, n] = act_fp8[rows, k] @ weight[n, k]ᵀ at row pitch plan.ldc, both block-scale sets folded in an FP32 epilogue.

The activation must already be through decode_w8a8_quant_act; this is the GEMM alone, so N projections over one activation cost one quantize and N matmuls.