Fp8TwinSet

Struct Fp8TwinSet 

Source
pub struct Fp8TwinSet {
    pub q: bool,
    pub k: bool,
    pub v: bool,
    pub o: bool,
}
Expand description

Which of the four attention projections get an FP8 [K, N] transposed twin built by Qwen3AttentionLayer::transpose_fp8_for_prefill.

WHY per projection and not one flag (#915): the four are reached by DIFFERENT prefill chains and only two of them are W8A8-gated.

  • K and V are read by prefill/cache_skip_qkv.rs:218 / :235, whose dispatch chain has no W8A8 arm at all — and cache_skip is the first-chunk path (trait_impl/prefill_inner.rs:138, seq_len_start == 0) taken by every request. Their twins are never dead.
  • Q on that chain is behind ATLAS_ATTN_PREFILL_Q_T=1 (cache_skip_qkv.rs:142); otherwise it is reached only after the W8A8 arm in prefill/paged_qkv.rs:220 declines.
  • O is routed to prefill/paged_oproj.rs from both chains, so it is reached only after the W8A8 arm at paged_oproj.rs:94 declines.

On 1xH100 (Qwen3.8-27B-FP8) the four cost 100 MiB/layer x 16 layers = 1,600 MB — the weight_map/quantized.rs:643 ledger row.

Fields§

§q: bool§k: bool§v: bool§o: bool

Implementations§

Source§

impl Fp8TwinSet

Source

pub const NONE: Self

Source

pub const ALL: Self

Every twin — what a loader that has not opted into the #915 plan asks for, i.e. the pre-#915 behaviour.

Source

pub fn any(self) -> bool

Trait Implementations§

Source§

impl Clone for Fp8TwinSet

Source§

fn clone(&self) -> Fp8TwinSet

Returns a duplicate of the value. Read more
1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for Fp8TwinSet

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for Fp8TwinSet

Source§

fn default() -> Fp8TwinSet

Returns the “default value” for a type. Read more
Source§

impl PartialEq for Fp8TwinSet

Source§

fn eq(&self, other: &Fp8TwinSet) -> bool

Tests for self and other values to be equal, and is used by ==.
1.0.0 · Source§

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
Source§

impl Copy for Fp8TwinSet

Source§

impl Eq for Fp8TwinSet

Source§

impl StructuralPartialEq for Fp8TwinSet

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

§

fn equivalent(&self, key: &K) -> bool

Compare self to key and return true if they are equal.
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more