latent_moe_forward

Function latent_moe_forward 

Source
pub fn latent_moe_forward(
    h: &[f32],
    down: &[f32],
    up: &[f32],
    norm_w: &[f32],
    logits: &[f32],
    bias: &[f32],
    experts: &[(Vec<f32>, Vec<f32>, Vec<f32>)],
    shared: Option<&[f32]>,
    cfg: &LatentMoeConfig,
    eps: f32,
) -> (Vec<f32>, Vec<usize>)
Expand description

Routed latent path + optional shared expert (identity-scale for tests).