Struct ExpertSliceMap 

pub struct ExpertSliceMap { /* private fields */ }

The stacked expert tensors of every MoE layer, located once.

Implementations

impl ExpertSliceMap

pub fn new(files: Arc<ShardFiles>) -> Result<Self>

pub fn layers(&self) -> &[ExpertLayerLoc]

MoE layers in ascending order.

pub fn layer(&self, layer: usize) -> Option<&ExpertLayerLoc>

pub fn files(&self) -> &Arc<ShardFiles>

pub fn slice_at(&self, loc: &SliceLoc, expert: usize) -> (usize, u64)

Absolute (shard, offset) of one projection of one expert.

Trait Implementations

impl ExpertSource for ExpertSliceMap

fn slot_layout(&self) -> SlotLayout

fn num_experts(&self) -> usize

fn read_expert(&self, layer: u32, expert: u32, dst: &mut [u8]) -> Result<()>

Fill dst (exactly slot_layout().bytes long) with expert expert of layer layer, in slot layout.

fn read_expert_range( &self, layer: u32, expert: u32, off: usize, dst: &mut [u8], ) -> Result<()>

Fill dst with bytes off..off + dst.len() of the expert’s slot image, so one miss can be read by several threads at once. The default reads the whole expert into scratch; the on-disk map seeks.

fn direct_segments( &self, layer: u32, expert: u32, ) -> Result<Option<Vec<DirectSeg>>>

The expert’s slot image as file ranges for O_DIRECT reads, or None when this source has no direct descriptor (the default): the staged miss path then reads through read_expert_range.

fn direct_file(&self, shard: usize) -> Option<&File>

The O_DIRECT descriptor a DirectSeg names.