Enum VisionTower 

pub enum VisionTower {
    Qwen(Box<VisionEncoder>),
    Glm(Box<GlmVit>),
}

Variants

Qwen(Box<VisionEncoder>)

Qwen3-VL / Qwen3.5 / Qwen3.6 / Qwen4 / LongCat / Mistral.

Glm(Box<GlmVit>)

GLM-5.3-Flash (glm5_next).

Implementations

impl VisionTower

pub fn qwen(encoder: VisionEncoder) -> Self

pub fn glm(encoder: GlmVit) -> Self

pub fn forward_batched( &self, images: &[(&[f32], usize, usize)], gpu: &dyn GpuBackend, stream: u64, ) -> Result<Vec<(usize, usize, usize)>>

Encode a batch of (pixels, grid_h, grid_w) temporal groups. Returns (post_merge_h, post_merge_w, merged_patches) per group, in order.

pub fn out_hidden_size(&self) -> usize

Width of one merged embedding row — the LLM’s hidden size.

pub fn ensure_scratch(&self, gpu: &dyn GpuBackend) -> Result<()>

Allocate the encoder’s device scratch if it is not allocated yet.

Normally the first image does this. A TENSOR-PARALLEL WORKER never sees one — it receives the merged rows over NCCL instead — so it has to ask for the buffers explicitly before out_row can name a destination.

pub fn out_row(&self, row: usize) -> DevicePtr

Device pointer to merged row row of the packed output. Valid only after forward_batched or ensure_scratch has allocated the scratch.