full_rank_output_gate

atlas_core::kimi_k3::kda

Function full_rank_output_gate 

Source 
pub fn full_rank_output_gate(
    core: &[f32],
    g: &[f32],
    head_dim: usize,
    eps: f32,
) -> Vec<f32>
Expand description

Full-rank output gate: sigmoid(g) ⊙ RMSNorm(core) per head.