attention

Function attention 

Source
pub fn attention(
    x: &[f32],
    seqlen: usize,
    start_pos: usize,
    w: &AttnWeights<'_>,
    c: &AttnCfg,
    fc: &[(f32, f32)],
    cache: &mut WindowCache,
) -> AttnRun
Expand description

Attention.forward for a ratio-0 layer. x: [seqlen][dim] bf16 values; fc from freqs_cis over max_seq_len. Mutates cache exactly as the reference’s ring buffer.