dequant_fp8_bf16_into

Function dequant_fp8_bf16_into 

Source
pub fn dequant_fp8_bf16_into(
    gpu: &dyn GpuBackend,
    fp8w: &Fp8Weight,
    dst: DevicePtr,
    stream: u64,
) -> Result<()>
Expand description

Dequantize a block-scaled OR per-row FP8 weight [N,K] → BF16 into a CALLER-OWNED buffer of n*k*2 bytes. Allocates nothing.

The kernel reads scale[(n / block_n) * sk + (k / block_k)], so the SAME kernel serves both layouts — the block geometry is what selects between them, not a second kernel:

block-scaled block_n = block_k = 128, sk = K/128 PER-ROW block_n = 1, block_k = K, sk = 1 -> offset = n * 1 + 0 = n, one multiplier per row

That per-row case is what a mixed-precision compressed-tensors checkpoint ships, and dequantising it is lossless: every FP8 E4M3 value is exactly representable in BF16, so this is the fold’s no-double-quant path even though the GEMM downstream is BF16.

SSOT for every FP8→BF16 weight expansion in this file. It takes a destination rather than producing one because the #917 H100 receipt (2026-09-11, Qwen/Qwen3.8-27B-FP8) was a gpu.alloc hidden in here: 167772160 B per GDN layer with no BufferSizes entry, invisible to --gpu-memory-utilization, which killed a 28-token prefill at layer 36 with cuMemAlloc_v2 failed: status 2. Who owns the bytes is now the caller’s decision, and the row-wise GDN arms answer it with the ledgered buffers.take_ssm_rowwise_w_bf16 slab.