expert

spark_model::layers::deepseek_v41_ref::moe

Function expert 

Source 
pub fn expert(
    x: &[f32],
    w1: &[f32],
    w2: &[f32],
    w3: &[f32],
    rows: usize,
    dim: usize,
    inter: usize,
    swiglu_limit: f32,
    weights: Option<&[f32]>,
) -> Vec<f32>
Expand description

One SwiGLU expert. w1, w3: [inter, dim]; w2: [dim, inter]; all bf16 values. x: [rows, dim] bf16 values; weights: per-row routing weight (f32) or None for the shared expert. Returns [rows, dim] bf16 values.