pub struct DenseFp8Inputs {
pub ffn_fp8: bool,
pub attn_fp8: bool,
pub keep_nvfp4: bool,
pub dispatch: GemmDispatch,
pub w8a8_kernels: bool,
pub attn_w4a4: bool,
pub attn_prefill_q_t: bool,
}Expand description
The inputs DenseFp8Plan::resolve decides from. Taken as a struct so the
CPU decision-table test can pin every clause without touching the process
environment (the OnceLock resolvers cannot be toggled per test).
Fields§
§ffn_fp8: boolThe native FP8 dense-FFN overlay will be installed on this layer
(ATLAS_DENSE_FP8=1, tp_size 1, Fp8Dequanted, gate_proj native FP8).
attn_fp8: boolThe native FP8 attention overlay will be installed on this layer.
keep_nvfp4: boolATLAS_DENSE_FP8_KEEP_NVFP4 — restore the pre-#915 behaviour.
dispatch: GemmDispatchResolved exactly as model/impl_a1.rs:836 resolves it.
w8a8_kernels: boolWhether the target’s per_token_group_quant_fp8 + fp8_gemm_t_blockscaled
kernels are both loaded. Both are required by the W8A8 prefill arm in
qwen3_attention/prefill/paged_qkv.rs:220 and
prefill/paged_oproj.rs:94; without them those two chains fall through
to the transposed W8A16 kernels, which read the FP8 twins.
attn_w4a4: boolATLAS_ATTN_W4A4 is set. prefill/paged_oproj.rs:38-42 builds its W4A4
arm with NO weight-type predicate and then feeds it
&self.attn.o_proj — the NVFP4 o_proj — so this one lever keeps the
NVFP4 attention weights alive even under a full FP8 overlay. (The QKV
side at paged_qkv.rs:51 does check as_nvfp4(), so it is already
closed; the o_proj asymmetry is not.)
attn_prefill_q_t: boolATLAS_ATTN_PREFILL_Q_T=1. prefill/cache_skip_qkv.rs:142 reads it per
projection per prefill and, when set, dispatches Q through q_fp8w_t.
Trait Implementations§
Source§impl Clone for DenseFp8Inputs
impl Clone for DenseFp8Inputs
Source§fn clone(&self) -> DenseFp8Inputs
fn clone(&self) -> DenseFp8Inputs
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for DenseFp8Inputs
impl Debug for DenseFp8Inputs
Source§impl PartialEq for DenseFp8Inputs
impl PartialEq for DenseFp8Inputs
impl Copy for DenseFp8Inputs
impl Eq for DenseFp8Inputs
impl StructuralPartialEq for DenseFp8Inputs
Auto Trait Implementations§
impl Freeze for DenseFp8Inputs
impl RefUnwindSafe for DenseFp8Inputs
impl Send for DenseFp8Inputs
impl Sync for DenseFp8Inputs
impl Unpin for DenseFp8Inputs
impl UnwindSafe for DenseFp8Inputs
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key and return true if they are equal.