launch_k3_mla_decode_token_on_device

Function launch_k3_mla_decode_token_on_device 

Source
pub fn launch_k3_mla_decode_token_on_device(
    gpu: &dyn GpuBackend,
    kernels: &K3MlaDecodeKernels,
    q: &[f32],
    k: &[f32],
    v: &[f32],
    g: &[f32],
    kv: &mut MlaDeviceKv,
    cfg: &MlaConfig,
    pos: usize,
    theta: f32,
    stream: u64,
) -> Result<Vec<f32>>
Expand description

Device-resident KV: rope stays on device, append is one row, one D2H (output).