Expand description
Kimi K3 host graph. This slice is KDA only.
K3-DECISION: KDA is a new backend. Do not copy GDN / Mamba-2 /
Qwen3-Next / glm5next_kda kernels into kernels/gb10/kimi-k3/.
CUDA: kda_decode.cu (default; K3_CUDA_KDA=0 CPU escape).
Re-exports§
pub use attnres::AttnResHub;pub use attnres::attnres_blend;pub use attnres::attnres_mix;pub use attnres::attnres_softmax_mix;pub use cache::HybridCache;pub use cache::LayerCache;pub use cache::MlaKv;pub use expert_backend::ExpertBackendKind;pub use expert_backend::MmapExpertStore;pub use expert_backend::PrefetchPlanner;pub use expert_backend::kind_from_env;pub use kda::KDA_L2_EPS;pub use kda::KdaConfig;pub use kda::KdaState;pub use kda::cuda_kda_enabled;pub use kda::kda_decode_token;pub use kda::kda_from;pub use latent_moe::LatentMoeConfig;pub use latent_moe::latent_moe_forward;pub use latent_moe::mix_routed_experts;pub use latent_moe::moe_from;pub use latent_moe::sigmoid_topk;pub use layer::K3Graph;pub use layer::K3LayerSpec;pub use layer::MixerKind;pub use layer::MlpKind;pub use mla::MlaConfig;pub use mla::cuda_mla_enabled;pub use mla::gated_mla_attend;pub use mla::mla_decode_token;pub use mla::mla_from;pub use situ::situ_glu;pub use situ::situ_glu_vec;pub use situ::softcap;pub use cpu_forward::AttnResStream;pub use cpu_forward::HiddenReduce;pub use cpu_forward::K3LayerCtx;pub use cpu_forward::forward_one_layer;pub use cpu_forward::forward_one_layer_with_cores;pub use cpu_forward::forward_one_layer_with_kda_decode;pub use cpu_forward::forward_one_layer_with_mla_decode;pub use cpu_forward::forward_token;pub use cpu_weights::Ablation;pub use cpu_weights::K3CpuLayer;pub use cpu_weights::K3CpuModel;pub use greedy::greedy_decode;
Modules§
- attnres
- Block AttnRes CPU reference.
- binding_
memory - Additional GPU weight storage created by the marked K3 model binder. Dense/packed layer binding aliases resident allocations. Only the three engine-facing FP32 tensors allocate BF16 copies; source allocations remain. This excludes engine workspace, KV, CUDA/NCCL and lazy inference allocations.
- cache
- Hybrid cache: paged MLA KV + KDA recurrent/conv state.
- cpu_
forward - One-token K3 CPU forward: AttnRes + mixer + MLP, then logits.
- cpu_
weights - Host-side 0.40B-shaped (or tiny) K3 weights for the C1 CPU graph.
- expert_
backend - Expert storage backend for K3 routed MXFP4 packs.
- greedy
- Token-greedy helper for C1: prefill the prompt, then decode one token at a time.
- kda
- Kimi K3 KDA CPU reference — a new backend, not GDN / Mamba-2.
- latent_
moe - Stable LatentMoE CPU reference.
- layer
- One K3 decoder layer as host graph data: mixer + MLP + AttnRes sites.
- mla
- Gated NoPE MLA CPU reference.
- mxfp4
- K3 official MXFP4 names → DeepSeek-V4 E8M0 unpack.
- ops
- Tiny host GEMV / gather helpers for the C1 CPU graph.
- situ
- SiTU-GLU CPU reference.
- tp
- K3 tensor-parallel storage plan, shared by host loading and device binding. Config head counts are rank-local; expert and dense widths remain full.
- weights
- Text decoder inventory shared by checkpoint admission and layer binding.
- weights_
geometry - Replicated checkpoint geometry checked before device allocation.