---
title: "KVCache"
sidebar_label: "KVCache"
description: "The clika_runtime.nn KVCache class."
---

<!-- Generated by tools/api_reference/generate_api_docs.py. Do not edit. -->

The serving-side KV cache: per-layer storage rows over slot-indexed batches. Built once with make(); the attention ops append and attend through the tensors it hands out.

## `__init__`

```python
__init__(self, /, *args, **kwargs)
```

Initialize self.  See help(type(self)) for accurate signature.

## `admit`

```python
admitadmit(self, batch_idx: int, prompt_ids: clika_runtime._core.Tensor, addressable_len: int = -1) -> int
```

admit(self, batch_idx: int, prompt_ids: clika_runtime._core.Tensor, addressable_len: int = -1) -> int

Paged only: bind the longest already-cached prefix into a fresh slot; returns the number of leading tokens whose K/V is already cached.

## `conv_state`

```python
conv_stateconv_state(self, layer: int, index: int = 0) -> clika_runtime._core.Tensor
```

conv_state(self, layer: int, index: int = 0) -> clika_runtime._core.Tensor

A state layer's i-th conv window slab `[max_seqs, dim, width]`.

## `evict_batch`

```python
evict_batchevict_batch(self, batch_idx: int) -> None
```

evict_batch(self, batch_idx: int) -> None

Free a slot's stored K/V (its seq_lens entry returns to 0).

## `fill`

```python
fillfill(self, layer: int, batch_idx: int, k: clika_runtime._core.Tensor, v: clika_runtime._core.Tensor) -> None
```

fill(self, layer: int, batch_idx: int, k: clika_runtime._core.Tensor, v: clika_runtime._core.Tensor) -> None

Fill a CrossAttentionKV layer's K/V planes for one slot in one call (`[t_enc, kv_heads, head_dim]` each).

## `keys`

```python
keyskeys(self, layer: int) -> clika_runtime._core.Tensor
```

keys(self, layer: int) -> clika_runtime._core.Tensor

This layer's K storage view; bind as past AND present key into the varlen attention ops.

## `make`

```python
makemake(config: clika_runtime._core.nn.KVCacheConfig, layer_specs: collections.abc.Sequence[clika_runtime._core.nn.KVLayerSpec]) -> clika_runtime._core.nn.KVCache
```

make(config: clika_runtime._core.nn.KVCacheConfig, layer_specs: collections.abc.Sequence[clika_runtime._core.nn.KVLayerSpec]) -> clika_runtime._core.nn.KVCache

Build a cache with one spec row per layer (the list's length must equal config.num_layers).

## `mode`

```python
modemode(self) -> clika_runtime._core.nn.KVCacheMode
```

mode(self) -> clika_runtime._core.nn.KVCacheMode

The serving mode this cache was built with.

## `paged_block_table`

```python
paged_block_tablepaged_block_table(self) -> clika_runtime._core.Tensor
```

paged_block_table(self) -> clika_runtime._core.Tensor

The paged block table `[max_seqs, max_blocks_per_seq]` Int32, or an undefined tensor for a cache with no paged rows.

## `prepare_step`

```python
prepare_stepprepare_step(self, cu_seqlens_q: collections.abc.Sequence[int], past_len: collections.abc.Sequence[int], slot_ids: collections.abc.Sequence[int]) -> clika_runtime._core.nn.StepIndices
```

prepare_step(self, cu_seqlens_q: collections.abc.Sequence[int], past_len: collections.abc.Sequence[int], slot_ids: collections.abc.Sequence[int]) -> clika_runtime._core.nn.StepIndices

Convert one step's host-side description into the device-side index tensors a varlen attention forward binds (a paged cache also reserves the step's blocks). One call per step.

## `recurrent_state`

```python
recurrent_staterecurrent_state(self, layer: int) -> clika_runtime._core.Tensor
```

recurrent_state(self, layer: int) -> clika_runtime._core.Tensor

A state layer's recurrent slab `[max_seqs, heads, k_dim, v_dim]`.

## `reserve`

```python
reservereserve(self, slot_ids: clika_runtime._core.Tensor, n_new_per_seq: collections.abc.Sequence[int]) -> clika_runtime._core.Tensor
```

reserve(self, slot_ids: clika_runtime._core.Tensor, n_new_per_seq: collections.abc.Sequence[int]) -> clika_runtime._core.Tensor

Paged only: reserve this step's new tokens per slot; returns the per-sequence pre-advance write offsets.

## `reset`

```python
resetreset(self) -> None
```

reset(self) -> None

Drop all stored K/V across every slot and layer.

## `retire`

```python
retireretire(self, batch_idx: int, full_token_ids: clika_runtime._core.Tensor, addressable_len: int = -1) -> None
```

retire(self, batch_idx: int, full_token_ids: clika_runtime._core.Tensor, addressable_len: int = -1) -> None

Paged only: retire a finished session, keeping its blocks addressable for future admits; use evict_batch on the cancel/error path instead.

## `seq_lens`

```python
seq_lensseq_lens(self) -> clika_runtime._core.Tensor
```

seq_lens(self) -> clika_runtime._core.Tensor

Per-slot effective token counts, `[max_seqs]` Int32 on the cache device.

## `update_position`

```python
update_positionupdate_position(self, batch_idx: int, new_pos: int) -> None
```

update_position(self, batch_idx: int, new_pos: int) -> None

Set a slot's effective seq_len (bookkeeping only).

## `values`

```python
valuesvalues(self, layer: int) -> clika_runtime._core.Tensor
```

values(self, layer: int) -> clika_runtime._core.Tensor

The value-side sibling of keys(layer).