Skip to main content

ClikaRT::nn::KVCache::StepIndices

struct

Header: ClikaRT/nn/kv_cache.h

The device-side index tensors one varlen decode/prefill step consumes, derived from the host-side step description (see prepare_step).

Member functions​

for_layer()​

const StepIndices& for_layer(int layer) const

The view layer layer binds: its row group's index tensors. On a uniform cache every layer returns this object's own fields (zero-copy); a mixed-layout cache returns the layer's group forms (a continuous row's slot_ids beside a paged row's gathered block table). An out-of-range layer returns the flat fields.

Declared in ClikaRT/nn/kv_cache.h, line 511

Data members​

cu_seqlens_q​

Tensor cu_seqlens_q

[B+1] Int32, the query cumulative.

Declared in ClikaRT/nn/kv_cache.h, line 485

cu_seqlens_k​

Tensor cu_seqlens_k

per-seq post-append KV lengths (see prepare_step).

Declared in ClikaRT/nn/kv_cache.h, line 486

kvcache_start​

Tensor kvcache_start

paged: [B, blocks_per_seq] gathered block table; continuous: the rank-1 [B] selector (see prepare_step).

Declared in ClikaRT/nn/kv_cache.h, line 489

slot_ids​

Tensor slot_ids

continuous: [B] Int32 slot ids on device, the attention op's slot_ids; undefined on a paged cache.

Declared in ClikaRT/nn/kv_cache.h, line 492

max_seqlen_k​

std::int64_t max_seqlen_k = 0

the step's max per-seq post-append KV length (host scalar).

Declared in ClikaRT/nn/kv_cache.h, line 493

groups​

std::vector<StepIndices> groups

Row-group views + the layer→group map. A group is the set of layers sharing one index shape (layout / window / K-V geometry); a uniform cache has one group and leaves both vectors EMPTY. Filled only on a cache carrying more than one group; every entry shares cu_seqlens_q, and tensor handles are shared, never copied.

Declared in ClikaRT/nn/kv_cache.h, line 501

group_of_layer​

std::vector<std::int32_t> group_of_layer

Row-group index per layer (group_of_layer[layer] -> the groups entry that layer's view binds).

Declared in ClikaRT/nn/kv_cache.h, line 504