Expand description
Composable transformer layer traits (SDD).
Decouples the generic model loop (embed -> layers -> norm -> lm_head)
from layer-specific logic (attention vs SSM, MoE vs dense FFN).
Adding a new architecture only requires implementing TransformerLayer
for each layer type, not duplicating the model loop.
Structs§
- Attn
Layer State - Attention-layer per-sequence state. KV lives in
PagedKvCache; the only resident piece is the QSA indexer carry on the 12 qwen4_exp full-attention layers (Atlas #753 item B). - Attn
Metadata Dev - Pre-uploaded attention metadata device pointers.
- Batched
Attn Metadata - Q12 batched-prefill device-side metadata.
- Empty
Layer State - Empty state for layers that store all persistent state externally
(e.g., attention layers where KV is in
PagedKvCache). - Forward
Context - Shared context for a single forward pass step.
- GdnPrefill
Buffers - Device pointers to full-sequence GDN input/output buffers.
- Midchunk
Capture - Per-pass descriptor for mid-chunk SSM tail capture. Points at the reserved Marconi snapshot slot’s per-SSM-layer destination buffers (already offset to the slot) plus the split point in local (chunk) token coordinates.
- SsmLayer
State - SSM layer state: recurrent hidden state + conv1d sliding window.
Enums§
- MoeLora
Route - Feature-1 MoE-LoRA fold decision for a single forward pass.
Constants§
- VERIFY_
WY_ LAYER_ STRIDE_ BYTES - Bytes between consecutive GDN layers’ table slices.
- VERIFY_
WY_ TABLES_ PER_ LAYER - Tables per GDN layer: h_state + Hi0..Hi14 (K=16 verify → 15
intermediates). 4 → 16 (2026-09-01): the wyN pointer-table twins
extend the cross-sequence batched verify to K=5..16, so the staging
gate
(2..=THIS)must admit gamma-width verifies; at 4 every K>4 batch silently declined to the per-sequence loop (measured 09-01: 187,392 wy10 launches = 27.7% of GPU time in a C=16 prose window). Cost: 48 layers x 16 x 32 x 8 B = 192 KB of host+device tables. - VERIFY_
WY_ TABLE_ SEQS - Batched-verify WY pointer-table layout (SSOT — writer:
TransformerModel::upload_verify_wy_tables, reader: the qwen3_ssmGdnStates::Multibatched conv+WY arm). - VERIFY_
WY_ TABLE_ STRIDE_ BYTES - Bytes between consecutive tables within a layer slice.
Traits§
- Layer
State - Per-layer persistent state tracked across decode steps.
- Transformer
Layer