Struct NgramHashState 

pub struct NgramHashState<'a> {
    pub t: &'a EngramTables,
    pub batch: usize,
    pub max_seq: usize,
    /* private fields */
}

engram.py::NgramHashState. The cache holds compressed ids for every position seen, so a decode step’s look-back reaches into the prefill.

Fields

t: &'a EngramTables

batch: usize

max_seq: usize

Implementations

impl<'a> NgramHashState<'a>

pub fn new(t: &'a EngramTables, batch: usize, max_seq: usize) -> Self

pub fn forward( &mut self, input_ids: &[i64], seqlen: usize, start_pos: usize, ) -> Vec<i64>

input_ids: [batch, seqlen] row-major. Returns [batch, seqlen, n_layers, n_hash_cols].