mhc_mix_max_tokens

Function mhc_mix_max_tokens 

Source
pub fn mhc_mix_max_tokens() -> usize
Expand description

Token bound the mix scratch is actually allocated at AND checked against.

🔴 MEASURED 2026-09-22, WS1/P2: this was a bare 256 const, and it is a hard ceiling on AVAROK_GLM_PREFILL_ROWS that has nothing to do with memory. At 512 rows the serve boots fine and then every prefill dies at layer 0 with “glm_hc_pre: 512 tokens exceeds the 256-token mix scratch” — HTTP 200, empty completion, runs/ws1-p2-prefill-n1n2/out/shared-r512/. P1’s “rows=1024 is not memory-feasible” never saw this because the LOAD died first; memory was the second blocker, not the first.

The scratch is [tokens, mix_hc(hc_mult)] FP32 per site: at GLM-5.3’s hc_mult = 4 that is tokens * 96 B per site, 90 sites — 2.2 MB for the whole model at 256, 8.8 MB at 1024. The ceiling was never buying anything; it was slack that hardened into a limit.

🪤 The allocation in glm5_next_load and the refusal in glm_hc_pre MUST read the same number, which is why this is one OnceLock function and not two max() expressions. Two expressions that agree today are a buffer overrun the day one of them is edited.