gate_and_add

spark_model::layers::deepseek_v41_ref::engram

Function gate_and_add 

Source 
pub fn gate_and_add(
    x: &[f32],
    kv: &[f32],
    q_w: &[f32],
    k_w: &[f32],
    tokens: usize,
    hc: usize,
    dim: usize,
    eps: f32,
) -> Vec<f32>
Expand description

Engram.forward after the projection: x [tokens, hc, dim] (bf16 values), kv [tokens, dim*(hc+1)], q/k weights [hc, dim]. Returns bf16-rounded [tokens, hc, dim].