Struct GlmVitMerger 

pub struct GlmVitMerger {
    pub post_layernorm_w: DevicePtr,
    pub downsample_w: DevicePtr,
    pub downsample_b: DevicePtr,
    pub proj_w: DevicePtr,
    pub norm_w: DevicePtr,
    pub norm_b: DevicePtr,
    pub gate_w: DevicePtr,
    pub up_w: DevicePtr,
    pub down_w: DevicePtr,
}

Everything after the last block: the post-norm, the 2×2 conv downsample and the merger MLP whose output is what gets spliced into the LM.

Fields

post_layernorm_w: DevicePtr

Weight-only RMSNorm over hidden_size, applied before the downsample.

downsample_w: DevicePtr

Conv2d(hidden, out_hidden, k=2, s=2) flattened to [out_hidden, hidden*4] in (in_channel, kh, kw) order — the conv’s own layout, which is what makes the im2col below a plain reshape.

downsample_b: DevicePtr

proj_w: DevicePtr

norm_w: DevicePtr

post_projection_norm is a real mean-subtracting LayerNorm with a bias — the ONLY such norm in this tower. Every other one is weight-only RMSNorm. The checkpoint census settles it: this is the only norm that ships a .bias.

norm_b: DevicePtr

gate_w: DevicePtr

up_w: DevicePtr

down_w: DevicePtr