Skip to main content

StepIndices

The device-side index tensors one varlen step consumes.

cu_seqlens_k (property)​

[B+1] Int32: the per-sequence KV lengths after this step's append, cumulative.

cu_seqlens_q (property)​

[B+1] Int32: the query cumulative sequence lengths.

kvcache_start (property)​

The KV placement of every sequence: a paged cache's gathered [B, blocks_per_seq] block table, a continuous cache's rank-1 [B] selector.

max_seqlen_k (property)​

The step's largest per-sequence KV length after the append (a host int).

slot_ids (property)​

Continuous cache: [B] Int32 slot ids on the device, the attention operator's slot_ids; an undefined tensor on a paged cache.

__init__​

__init__(self, /, *args, **kwargs)

Initialize self. See help(type(self)) for accurate signature.

for_layer​

for_layerfor_layer(self, layer: int) -> clika_runtime._core.nn.StepIndices

for_layer(self, layer: int) -> clika_runtime._core.nn.StepIndices

The index view this layer binds (its row group's tensors; the flat fields on a uniform cache).