pub fn expert_situ(
x: &[f32],
w1: &[f32],
w2: &[f32],
w3: &[f32],
in_dim: usize,
hidden: usize,
beta: f32,
beta_lin: f32,
) -> Vec<f32>Expand description
One SiTU-GLU expert: down( situ(w1 x, w3 x) ). Weights are [out, in].