Expand description
Hybrid cache: paged MLA KV + KDA recurrent/conv state.
Prefix-cache restore reuses C3 CPU semantics: snapshot/restore the
per-layer LayerCache (KDA conv/recurrent or MLA KV).
Host bytes are the prefix snapshot. Decode hot path must use
spark_model::kimi_k3::DeviceHybridCache (device-resident conv/recurrent
and MLA KV). Per-token D2H/H2D of those buffers is not this cache.
Structs§
- Hybrid
Cache - Per-sequence hybrid cache. Slot identity is the layer index; a prefix hit that writes KDA state into the wrong slot is the C4 mutant.
- MlaKv
- One MLA layer’s host KV (unpaged CPU stand-in).