launch_k3_mla_decode_token

Function launch_k3_mla_decode_token 

Source
pub fn launch_k3_mla_decode_token(
    gpu: &dyn GpuBackend,
    kernels: &K3MlaDecodeKernels,
    q: &mut [f32],
    k: &mut [f32],
    v: &[f32],
    g: &[f32],
    kv: &mut MlaKv,
    cfg: &MlaConfig,
    pos: usize,
    theta: f32,
    stream: u64,
) -> Result<Vec<f32>>
Expand description

One-token CUDA gated-NoPE MLA. Updates q/k (rope) and kv (append).