Module weights

Expand description

Weight loading from safetensors files (SBIO IORouter for filesystem I/O).

Modules§

adapter: Split a tensor name into (everything but its last numeric path segment, that segment as a number) so names sort NUMERICALLY on the index. embedders.2 must precede embedders.10; a plain lexicographic sort puts 10 first and silently mis-maps every table after the ninth. PEFT adapter loader: adapter_model.safetensors → WeightStore.
dequant_cpu: Public for kernel oracles: the CPU reference decoders every GPU dequant is held to. CPU reference dequantization for GGUF/GGML quantized weight blocks.
expert_stream: DeepSeek-V4.1 Flash expert streaming from the split GGUF, by positional read.
mlx_int8: MLX uint32-packed 8-bit weight format support.

DeferredTensor: Where a skipped tensor lives, so a consumer can read it in place.
DerivedStore: Device buffers a loader derived from this store’s tensors.
GgufLoader: Loads weights from a GGUF file, dequantizing every tensor to BF16 on the GPU.
GgufShardSet: One resolved split-GGUF file set, in split.no order.
K3SafetensorsLoader
SafetensorsLoader: Loads weights from safetensors files using mmap.
WeightStore: All model weights loaded onto the GPU, keyed by HuggingFace name.
WeightTensor: A weight tensor on the GPU.

auto_detect_weight_prefix: Resolve the weight-key prefix a nested (multimodal) checkpoint uses.
config_from_gguf_dir: Build a [ModelConfig] from the .gguf in model_dir, with no config.json.
find_gguf: Locate the backbone GGUF weight file in dir. Returns the lexicographically-first non-mmproj *.gguf (also the first shard, *-00001-of-*, of a split file). The mmproj vision sidecar is excluded here and loaded separately (see sidecar::find_mmproj); a dir that is only an mmproj falls back to the first file so the caller still gets a path to error on.
find_gguf_shards: Resolve every shard of a split GGUF, given the first one.
is_ngram_table: Whether a tensor is an n-gram embedding TABLE — the huge blobs that are served off NVMe instead of being uploaded with the rest of the checkpoint.
parse_expert_index: Parse expert index from tensor name (e.g. “model.layers.3.mlp.experts.42.gate_proj.weight” → 42).

DeferHook: A model loader’s answer to “which tensors will I read from disk myself?”.