pub fn requires_single_chunk_prefill(
model_type: &str,
kv_lora_rank: usize,
) -> boolExpand description
Must chunked prefill run as a SINGLE chunk for this model?
True only for models that reach the chunk-LOCAL MLA prefill in
qwen3_attention/prefill.rs, which attends over the current chunk’s K/V alone —
multi-chunk there silently corrupts attention output (Mistral-Small-4, 2026-05-01: 8 K
collapses to “The\nThe…”).
🔴 kv_lora_rank > 0 is a PROXY for that kernel and glm5_next breaks it: GLM-5.3 is
MLA (rank 512) but prefills through Glm5NextLayer::prefill, a per-token walk that
attends the whole paged prefix at each absolute position — chunk boundaries are
invisible to it. Answering true capped every GLM prompt at 2 × --max-prefill-tokens,
because prefill_a_step splits the FIRST chunk at the cap regardless and this gate then
made the remainder one unsplit chunk the buffer arena refused. ANOMALIES A61.
K3 also uses the default per-token prefill walk: K3BoundLayer::decode advances
its sequence-owned KDA/MLA cache at each absolute position. Its MLA decode
attends that whole retained cache, so it must honor the scheduler chunk budget.