Expand description
DFlash drafter levers, resolved once per head and then carried.
§★ THE ENVIRONMENT IS READ EXACTLY ONCE PER HEAD. KEEP IT THAT WAY.
forward_block runs once per DECODE STEP, and its per-layer helpers run
num_layers times inside that. Every std::env::var on that path
allocates a String and takes the PROCESS-WIDE environment lock, so
concurrent decode threads serialise against each other on it —
MEASURED on GB10: a 30-variable resolve costs 0.57 us single-threaded but
4.00 us at 8 threads and 5.76 us at 16. The cost grows with
concurrency, which is exactly why no single-stream benchmark shows it.
Before this module forward_block read 31 variables per propose, ten
of them duplicates of a variable it had already read in the same call, and
eleven of them in a single && chain whose only job was to answer one
question: is any diagnostic armed?
These are resolved at head construction rather than in a OnceLock
static, for the reason crate::layers::ops::ModelLevers gives: a static
outlives the model whose flags it encodes, so a second model silently
keeps the first one’s branches. A field on the head cannot go stale,
because a new head is a new resolution.
Structs§
- DFlash
Levers - Diagnostic and A/B levers for one loaded DFlash drafter.