PredictedDerived

Struct PredictedDerived 

Source
pub struct PredictedDerived {
    pub attn_fp8_twins: u64,
    pub ssm_fp8_concat: u64,
    pub ffn_gateup_fused: u64,
    pub ffn_gateup_pruned: u64,
    pub twins: TwinsBuilt,
    pub attn_twin_set: Fp8TwinSet,
}
Expand description

The derived bytes the native-FP8 dense loader will allocate and KEEP, broken out so the preflight log can name each term.

Mirrors DerivedResidency::kept term for term — see the module docs’ receipt table — so predicted.total() and the serve log’s native FP8 dense residency: ... derived X GB are the same arithmetic evaluated at two different times.

Fields§

§attn_fp8_twins: u64

Fp8Weight::transpose_for_gemm twins for the projections fp8_residency::DenseFp8Plan selects, summed over full-attention layers.

§ssm_fp8_concat: u64

The fused [QKV|Z] FP8 weight, its block-scale grid, the out_proj block-scale grid and the interleaved in_proj_ba, summed over linear-attention layers.

§ffn_gateup_fused: u64

The fused [2*inter, hidden] dense-FFN gate+up weight and its block-scale grid (#927), summed over dense-FFN layers — 0 when the target does not arm the arm.

§ffn_gateup_pruned: u64

The checkpoint bytes prune_after_load gives BACK because the fusion consumed them: mlp.gate_proj.weight + mlp.up_proj.weight over the same layers. EQUAL to Self::ffn_gateup_fused minus the scale grids, by construction — the fused weight IS those two tensors copied side by side — which is why the fusion nets out of Self::total below.

§twins: TwinsBuilt

Which twin families the prediction expects, for the log line.

§attn_twin_set: Fp8TwinSet

The twin set the attention term was priced at.

Implementations§

Source§

impl PredictedDerived

Source

pub fn total(&self) -> u64

Derived bytes ABOVE the on-disk checkpoint count, which is what headroom.rs adds to weights to build the post-load yardstick.

The gate+up fusion appears as a DIFFERENCE and not as a term: the caller’s weights is the checkpoint’s on-disk size, which still counts gate_proj.weight and up_proj.weight — and the loader releases both once the fused copy exists. Adding the fused weight without subtracting what it replaces would over-state pre-KV by 11.4 GB on Qwen3.8-27B and silently shrink — or refuse — the decode-rollback ring this yardstick exists to fit. The two terms are equal by construction, so the difference is exactly zero and every prediction taken before #927 is unchanged; both are carried so that is legible rather than asserted (the_gateup_fusion_is_residency_neutral).

Trait Implementations§

Source§

impl Clone for PredictedDerived

Source§

fn clone(&self) -> PredictedDerived

Returns a duplicate of the value. Read more
1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for PredictedDerived

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for PredictedDerived

Source§

fn default() -> PredictedDerived

Returns the “default value” for a type. Read more
Source§

impl PartialEq for PredictedDerived

Source§

fn eq(&self, other: &PredictedDerived) -> bool

Tests for self and other values to be equal, and is used by ==.
1.0.0 · Source§

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
Source§

impl Copy for PredictedDerived

Source§

impl Eq for PredictedDerived

Source§

impl StructuralPartialEq for PredictedDerived

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

§

fn equivalent(&self, key: &K) -> bool

Compare self to key and return true if they are equal.
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more