pub struct PredictedDerived {
pub attn_fp8_twins: u64,
pub ssm_fp8_concat: u64,
pub ffn_gateup_fused: u64,
pub ffn_gateup_pruned: u64,
pub twins: TwinsBuilt,
pub attn_twin_set: Fp8TwinSet,
}Expand description
The derived bytes the native-FP8 dense loader will allocate and KEEP, broken out so the preflight log can name each term.
Mirrors DerivedResidency::kept term for term — see the module docs’
receipt table — so predicted.total() and the serve log’s
native FP8 dense residency: ... derived X GB are the same arithmetic
evaluated at two different times.
Fields§
§attn_fp8_twins: u64Fp8Weight::transpose_for_gemm twins for the projections
fp8_residency::DenseFp8Plan selects, summed over full-attention
layers.
ssm_fp8_concat: u64The fused [QKV|Z] FP8 weight, its block-scale grid, the out_proj
block-scale grid and the interleaved in_proj_ba, summed over
linear-attention layers.
ffn_gateup_fused: u64The fused [2*inter, hidden] dense-FFN gate+up weight and its
block-scale grid (#927), summed over dense-FFN layers — 0 when the
target does not arm the arm.
ffn_gateup_pruned: u64The checkpoint bytes prune_after_load gives BACK because the fusion
consumed them: mlp.gate_proj.weight + mlp.up_proj.weight over the
same layers. EQUAL to Self::ffn_gateup_fused minus the scale grids,
by construction — the fused weight IS those two tensors copied side by
side — which is why the fusion nets out of Self::total below.
twins: TwinsBuiltWhich twin families the prediction expects, for the log line.
attn_twin_set: Fp8TwinSetThe twin set the attention term was priced at.
Implementations§
Source§impl PredictedDerived
impl PredictedDerived
Sourcepub fn total(&self) -> u64
pub fn total(&self) -> u64
Derived bytes ABOVE the on-disk checkpoint count, which is what
headroom.rs adds to weights to build the post-load yardstick.
The gate+up fusion appears as a DIFFERENCE and not as a term: the
caller’s weights is the checkpoint’s on-disk size, which still counts
gate_proj.weight and up_proj.weight — and the loader releases both
once the fused copy exists. Adding the fused weight without subtracting
what it replaces would over-state pre-KV by 11.4 GB on Qwen3.8-27B
and silently shrink — or refuse — the decode-rollback ring this
yardstick exists to fit. The two terms are equal by construction, so
the difference is exactly zero and every prediction taken before #927
is unchanged; both are carried so that is legible rather than asserted
(the_gateup_fusion_is_residency_neutral).
Trait Implementations§
Source§impl Clone for PredictedDerived
impl Clone for PredictedDerived
Source§fn clone(&self) -> PredictedDerived
fn clone(&self) -> PredictedDerived
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for PredictedDerived
impl Debug for PredictedDerived
Source§impl Default for PredictedDerived
impl Default for PredictedDerived
Source§fn default() -> PredictedDerived
fn default() -> PredictedDerived
Source§impl PartialEq for PredictedDerived
impl PartialEq for PredictedDerived
impl Copy for PredictedDerived
impl Eq for PredictedDerived
impl StructuralPartialEq for PredictedDerived
Auto Trait Implementations§
impl Freeze for PredictedDerived
impl RefUnwindSafe for PredictedDerived
impl Send for PredictedDerived
impl Sync for PredictedDerived
impl Unpin for PredictedDerived
impl UnwindSafe for PredictedDerived
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key and return true if they are equal.