window_topk_idxs

Function window_topk_idxs 

Source
pub fn window_topk_idxs(
    win: usize,
    seqlen: usize,
    start_pos: usize,
) -> (Vec<i32>, usize)
Expand description

get_window_topk_idxs: [queries][cols] of slot indices, -1 for a slot holding nothing. Prefill: one causal window per query into the chunk itself. Decode: the whole ring, oldest first.