Module expert_stream 

DeepSeek-V4.1 Flash expert streaming from the split GGUF, by positional read.

The routed experts of the Q2_K checkpoint are 40 layers x 384 experts of 12.22 MiB each; they never fit and they are never expanded. Measured on the real shards (deepseek_v41_stream_bench_test.rs): a CPU dequant of one expert is 851 ms, and page-faulting an expert slice in through the mmap runs at 0.33 GB/s cold and cached alike, while pread of the same 3.7 MiB slices runs at 4.5 GB/s cold. So the loader’s mmap is used for the header only, and the expert bytes are read by pread straight into a page-locked arena the GPU addresses in place (see super::expert_lru::ExpertLru); the K-quant kernels consume the raw blocks from there.

Three pieces:

• ShardFiles: every shard opened once, header parsed, mmap dropped.
• ExpertSliceMap: where expert e of layer l lives for each of the three stacked tensors (blk.N.ffn_{gate,up,down}_exps.weight, GGUF dims [k, n, experts]), and the read of one expert into one slot.
• EngramRowReader (engram_rows.rs): the ~30 GiB engram tables, one Q2_K block (84 B) per row, read by row id on demand. 48 rows per token.

Oracle: bytes identical to the mmap path, held by deepseek_v41_stream_oracle_test.rs on the real shards.

Structs

DeviceArena

One device arena plus its page-locked staging ring.

DirectSeg

One file range of an expert’s slot image, for the direct miss path: bytes slot_off..slot_off + len of the slot are file_off.. of shard.

EngramRowReader

Row reads from the engram tables. The tables are ~30 GiB each and a token touches 24 rows per table, so nothing is cached: every call is ids.len() positional reads of one block.

EngramTable

One engram table: blk.N.engram_embd.weight, GGUF dims [head_dim, rows], one quant block per row.

ExpertLayerLoc

The three projections of one MoE layer.

ExpertLru

ExpertSliceMap

The stacked expert tensors of every MoE layer, located once.

ExpertSlot

Where one cached expert’s three projections are, as the GPU sees them.

LruStats

PinnedArena

One page-locked host arena with its device alias.

ShardFiles

Every shard of a split GGUF, opened once for the process lifetime. The header is parsed through the loader’s own reader and the mmap is dropped: nothing here goes through the page cache but the reads we ask for.

SliceLoc

Where one expert of one stacked projection lives.

SlotLayout

Byte layout of one expert inside an LRU slot: [gate | up | down].

Enums

ExpertArena

Either arena, as the runtime that keeps it alive holds it.

Traits

ExpertSource

Anything the LRU can fill a slot from. The on-disk map implements it; the unit tests implement it in memory.

Functions

pread

Positional read of exactly dst.len() bytes at offset. No file position is shared, so any number of threads may read one File at once.

pread_at_least

pread into dst until at least need bytes have landed (need <= dst.len()): the direct miss path reads 512 B windows whose tail may run past the end of the file, where only the tensor’s own bytes are owed.

pread_uncached

pread, then drop the range from the page cache: the bytes now live in the arena and the cache copy is dead weight. With a 100 GiB page-locked arena on a 121 GiB Spark, leaving 12 MiB of cache behind every miss made the kernel reclaim into swap in the middle of a step (single steps of 2 to 18 s on the 09-19 standard). ATLAS_DS41_KEEP_PAGE_CACHE=1 keeps the old behaviour.