pub fn bounded_gate(
z: &[f32],
dt_bias: &[f32],
a_log: &[f32],
heads: usize,
head_dim: usize,
lower_bound: Option<f32>,
) -> Vec<f32>Expand description
KDA forget-gate in log space (FLA use_gate_in_kernel).
Some(lb):lb * sigmoid(exp(A_log) * (z + dt_bias))(production-5)None:-exp(A_log) * softplus(z + dt_bias)(0.40B twin / FLA default)