DenseFp8Inputs

Struct DenseFp8Inputs 

Source
pub struct DenseFp8Inputs {
    pub ffn_fp8: bool,
    pub attn_fp8: bool,
    pub keep_nvfp4: bool,
    pub dispatch: GemmDispatch,
    pub w8a8_kernels: bool,
    pub attn_w4a4: bool,
    pub attn_prefill_q_t: bool,
}
Expand description

The inputs DenseFp8Plan::resolve decides from. Taken as a struct so the CPU decision-table test can pin every clause without touching the process environment (the OnceLock resolvers cannot be toggled per test).

Fields§

§ffn_fp8: bool

The native FP8 dense-FFN overlay will be installed on this layer (ATLAS_DENSE_FP8=1, tp_size 1, Fp8Dequanted, gate_proj native FP8).

§attn_fp8: bool

The native FP8 attention overlay will be installed on this layer.

§keep_nvfp4: bool

ATLAS_DENSE_FP8_KEEP_NVFP4 — restore the pre-#915 behaviour.

§dispatch: GemmDispatch

Resolved exactly as model/impl_a1.rs:836 resolves it.

§w8a8_kernels: bool

Whether the target’s per_token_group_quant_fp8 + fp8_gemm_t_blockscaled kernels are both loaded. Both are required by the W8A8 prefill arm in qwen3_attention/prefill/paged_qkv.rs:220 and prefill/paged_oproj.rs:94; without them those two chains fall through to the transposed W8A16 kernels, which read the FP8 twins.

§attn_w4a4: bool

ATLAS_ATTN_W4A4 is set. prefill/paged_oproj.rs:38-42 builds its W4A4 arm with NO weight-type predicate and then feeds it &self.attn.o_proj — the NVFP4 o_proj — so this one lever keeps the NVFP4 attention weights alive even under a full FP8 overlay. (The QKV side at paged_qkv.rs:51 does check as_nvfp4(), so it is already closed; the o_proj asymmetry is not.)

§attn_prefill_q_t: bool

ATLAS_ATTN_PREFILL_Q_T=1. prefill/cache_skip_qkv.rs:142 reads it per projection per prefill and, when set, dispatches Q through q_fp8w_t.

Trait Implementations§

Source§

impl Clone for DenseFp8Inputs

Source§

fn clone(&self) -> DenseFp8Inputs

Returns a duplicate of the value. Read more
1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for DenseFp8Inputs

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl PartialEq for DenseFp8Inputs

Source§

fn eq(&self, other: &DenseFp8Inputs) -> bool

Tests for self and other values to be equal, and is used by ==.
1.0.0 · Source§

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
Source§

impl Copy for DenseFp8Inputs

Source§

impl Eq for DenseFp8Inputs

Source§

impl StructuralPartialEq for DenseFp8Inputs

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

§

fn equivalent(&self, key: &K) -> bool

Compare self to key and return true if they are equal.
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more