Struct Qwen35DenseWeightLoader 

pub struct Qwen35DenseWeightLoader;

Trait Implementations

impl ModelWeightLoader for Qwen35DenseWeightLoader

fn prune_after_load( &self, store: &mut WeightStore, config: &ModelConfig, gpu: &dyn GpuBackend, ) -> Result<()>

Drop the SSM source tensors the native-FP8 GDN arm copied and no longer reads (#915).

That arm builds a fused [QKV|Z] FP8 weight by device-to-device appending in_proj_qkv and in_proj_z (concat_fp8_block_scaled), and builds in_proj_ba by a D2H -> host interleave -> H2D round trip (interleave_ba). All four store originals are dead the moment load_layers returns — 80 MiB + ~1 MB per SSM layer, ~3.9 GB across the 48 GDN layers of Qwen3.8-27B-FP8, which is what the fused copy costs. Keeping both is the duplicate that came straight out of the KV budget.

🪤 Narrow on purpose. out_proj.weight IS out_proj_fp8w.weight (zero-copy from the store), conv1d, A_log, dt_bias and norm.weight are aliased or conditionally aliased depending on their on-disk dtype, and every attention tensor — and mlp.down_proj — is bound zero-copy. Only the names below are freed, and only for layers where the module-private predicate that SELECTED the corresponding concat (gdn_fp8_arm_selected for the SSM [QKV|Z] weight, ffn_gateup_fused_selected for the dense-FFN gate+up weight, #927) says that arm actually ran.

fn supports_tp(&self) -> bool

Whether this loader’s weight slicing is TP-aware. No default — every loader MUST declare this explicitly so adding a new model architecture cannot accidentally inherit a false and silently regress users who pass --tp-size > 1.

fn load_layers( &self, store: &WeightStore, config: &ModelConfig, gpu: &dyn GpuBackend, layer_kv_dtypes: &[KvCacheDtype], ) -> Result<Vec<Box<dyn TransformerLayer>>>

Load all transformer layers from the weight store.

fn load_embedding( &self, store: &WeightStore, config: &ModelConfig, _gpu: &dyn GpuBackend, ) -> Result<DenseWeight>

fn load_final_norm( &self, store: &WeightStore, config: &ModelConfig, _gpu: &dyn GpuBackend, ) -> Result<DenseWeight>

Load the final RMSNorm weight used before the LM head.

fn load_lm_head( &self, store: &WeightStore, config: &ModelConfig, gpu: &dyn GpuBackend, ) -> Result<DenseWeight>

fn load_mtp_weights( &self, store: &WeightStore, config: &ModelConfig, gpu: &dyn GpuBackend, ) -> Result<Option<MtpWeights>>

Load MTP head weights (returns None if no MTP weights in store).

fn load_vision_encoder( &self, store: &WeightStore, config: &ModelConfig, gpu: &dyn GpuBackend, ) -> Result<Option<VisionEncoder>>

Load vision encoder weights (returns None for text-only models).

fn precision_schedule(&self, _config: &ModelConfig) -> PrecisionSchedule

Per-(layer, role) weight precision schedule (C.3, 2026-04-25). Default impl returns the empty schedule (every lookup yields Dtype::Inherit), preserving the existing per-checkpoint dtype logic byte-for-byte. Loader-specific implementations can override to honour MODEL.toml’s [precision] block.

fn load_ngram_embedding( &self, _store: &WeightStore, _config: &ModelConfig, _gpu: &dyn GpuBackend, _max_tokens: usize, ) -> Result<Option<NgramEmbedding>>

Build the n-gram embedding, when this architecture fuses hashed n-gram lookups into the input embedding (LongCat / Qwen3.8-Flash-Next).

fn load_mtp_weights_multi( &self, store: &WeightStore, config: &ModelConfig, gpu: &dyn GpuBackend, ) -> Result<Vec<MtpWeights>>

Load MTP weights for multi-module MTP (DeepSeek-V3 / MiniMax-M2 style: N independent transformer modules, each with its own attention + MoE + KV cache). Returns an empty Vec when the checkpoint has no MTP modules, a 1-element Vec for single-module MTP (Qwen3.5 family), or N elements for multi-module.

fn kv_layer_dims(&self, _config: &ModelConfig) -> Vec<(usize, usize)>

Per-layer (num_kv_heads, head_dim) overrides for heterogeneous attention models (e.g. Gemma-4 with sliding 16×256 and full 4×512). Default empty — homogeneous models skip per-layer dims and the KV cache allocator uses the global (num_kv_heads, head_dim). Populated by loaders whose models have different attention geometries per layer. Indexed by attention layer index (same as layer_kv_dtypes).

fn load_dflash_weights( &self, _drafter_store: &WeightStore, _config: &ModelConfig, _gpu: &dyn GpuBackend, _tp_size: usize, ) -> Result<Option<DflashWeights>>

Load DFlash drafter weights from a separate WeightStore pointing at the drafter checkpoint (z-lab/Qwen3.6-{27B,35B-A3B}-DFlash). Default impl returns None so loaders that don’t yet support DFlash silently fall through to the existing MTP path. Override in loaders whose target models pair with a DFlash drafter (Qwen3.5/3.6 family). The same drafter format works across both 27B-dense and 35B-A3B-MoE targets — only the target_hidden_size validated against the drafter’s fc input dimension differs.

fn load_lora_adapters( &self, adapters: &[LoraAdapterInput<'_>], config: &ModelConfig, gpu: &dyn GpuBackend, max_loras: usize, max_lora_rank: usize, ) -> Result<Option<LoraWeights>>

Load one or more startup-static PEFT LoRA adapters from their own WeightStores (the adapter_model.safetensors tensors, already on-device BF16) into the fixed-address rank-padded pool (one slot each).

fn binds_vision_encoder(&self) -> bool

Will this loader ever bind a vision encoder for a multimodal checkpoint?