pub fn matvec_column_tp(
w: &[f32],
x: &[f32],
out: usize,
inn: usize,
world: usize,
drop_rank: Option<usize>,
) -> Vec<f32>Expand description
Column-parallel y = W x (Megatron o_proj). W is [out, inn].
world == 1 is the unsplit GEMV. world == 2 splits columns across two
in-process ranks and allreduces (sum) the hidden. drop_rank zeros that
shard (C7 known-bad: drop rank 1).