Expand description
Marconi aux state for the DSA indexer — the half of a GLM-5.3 sequence that a KV-only prefix-cache hit cannot reconstruct.
§Why a blob and not a rewind
Glm5NextDsaState is append-only with a host cursor, so rewind_to is an
exact restore within a live sequence: the rows are still there and only the
cursor moved. A prefix-cache hit is not that. It hands the prefix to a
sequence whose indexer buffer was never written, so there is nothing to
rewind to — and the rows cannot be recomputed either, because k_normed and
gate are projections of the hidden state (indexer.wk,
index_kpool_compress_gate) and wk · hidden cannot be inverted out of the
rank-512 MLA latent (see the module doc on super::state). The rows have
to travel with the snapshot.
§The precedent this copies
layers/qsa_snapshot.rs does exactly this for qwen4_exp’s QSA indexer:
serialize the ingested keys plus the cursor, and on restore re-derive by
kernel whatever is cheaper to recompute than to carry. DSA has nothing in
the second category — pooling happens inside the selector kernel each step
and is never stored — so the blob is the whole reachable cache and nothing
else.
§🔴 Fail-closed, deliberately
Every rejection here is a hard Err, never a skip. A DSA restore that
silently applied a stale but correctly sized blob would select over another
sequence’s keys and return a wrong answer under HTTP 200 — the A25/A55 shape.
The size and geometry checks below are what make a mismatched blob
unrepresentable rather than merely unlikely.
§Cost
len * (4 * index_head_dim + 1) bytes per DSA layer — 513 B/token/layer at
GLM-5.3’s index_head_dim = 128, 5,643 B/token across its 11 text DSA
layers. The blob carries len rows, NOT capacity: a 4K prefix costs 4K
rows even on a serve declaring --max-seq-len 131072.