hc_mixes

Function hc_mixes 

Source
pub fn hc_mixes(
    x: &[f32],
    tokens: usize,
    hc: usize,
    dim: usize,
    hc_fn: &[f32],
    scale: &[f32],
    base: &[f32],
    iters: usize,
    hc_eps: f32,
    norm_eps: f32,
) -> (Vec<f32>, Vec<f32>, Vec<f32>)
Expand description

Block.hc_mixes. x: [tokens, hc, dim] (bf16 values); hc_fn: [(2+hc)*hc, hc*dim] f32. Returns (pre[tokens, hc], post[tokens, hc], comb[tokens, hc, hc]).