Expand description
One K3 decoder layer as host graph data: mixer + MLP + AttnRes sites.
h = AttnRes(blocks, partial, attn_res_*)
mix_out = KDA | gated-NoPE-MLA (on RMSNorm(h))
partial += mix_out
h = AttnRes(blocks, partial, mlp_res_*)
mlp_out = dense SiTU-GLU | LatentMoE (on RMSNorm(h))
partial += mlp_out
at layer_idx % block_size == 0: archive incoming prefix, reset partial