---
title: "ClikaRT::nn::KVLayerSpec"
sidebar_label: "KVLayerSpec"
description: "One decoder layer's storage row, the per-layer entry of the spec vector KVCache::make consumes. Two orthogonal axes: Kind says WHAT the row stores, Layout says HOW a token-indexed row is laid out."
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--struct">struct</span>

**Header:** `ClikaRT/nn/kv_cache.h`

One decoder layer's storage row, the per-layer entry of the spec vector [`KVCache::make`](./KVCache.md#make) consumes. Two orthogonal axes: [`Kind`](./KVLayerSpec.md#Kind) says WHAT the row stores, [`Layout`](./KVLayerSpec.md#Layout) says HOW a token-indexed row is laid out.

## Nested types

| Name | Description |
| --- | --- |
| [`ConvState`](./KVLayerSpec.ConvState.md) | One short-conv rolling window: [`dim`](./KVLayerSpec.ConvState.md#dim) channels × [`width`](./KVLayerSpec.ConvState.md#width) taps, the slab `causal_conv_update` binds as its state. |
| [`RecurrentState`](./KVLayerSpec.RecurrentState.md) | The delta-rule recurrent state geometry (per slot: `[`[`heads`](./KVLayerSpec.RecurrentState.md#heads)`, `[`k_dim`](./KVLayerSpec.RecurrentState.md#k_dim)`, `[`v_dim`](./KVLayerSpec.RecurrentState.md#v_dim)`]`). [`dtype`](../dtype/index.md) picks the slab's STORAGE dtype: Float32 (the default reference), or BFloat16 / Float16, half the state bandwidth and checkpoint bytes; the recurrence ops still compute fp32 and round only at the slab round-trip (bf16 keeps f32's exponent range, f16 keeps more mantissa). |

## Types

### enum Kind {#Kind}

<div class="api-signature">

`enum class` **`Kind`** `:` `uint8_t`

</div>

WHAT the row stores. Orthogonal to how it is laid out.

| Enumerator | Description |
| --- | --- |
| `AttentionKV` | token-indexed K/V (full, or windowed via [`window`](./KVLayerSpec.md#window)) |
| `ConvState` | causal-conv rolling windows only |
| `RecurrentState` | delta-rule / SSM recurrent slab only |
| `HybridState` | conv windows + a recurrent slab on ONE row |
| `CrossAttentionKV` | Encoder K/V for cross-attention: written ONCE per encoder pass via [`KVCache::fill`](./KVCache.md#fill) (length = the encoder length), read by every decode step, NEVER appended; the per-step append path rejects these rows. [`Layout::Auto`](./KVLayerSpec.md#Layout) resolves Continuous and an explicit [`Layout::Paged`](./KVLayerSpec.md#Layout) rejects at `make` (a fill-once row gains nothing from a block pool); [`preallocate`](./KVLayerSpec.md#preallocate) is implied true and the field is IGNORED (fill writes the planes in place, so the buffer must not move); [`window`](./KVLayerSpec.md#window) rejects (a sliding window on a fill-once row is a contradiction). The prefix-cache verbs (`admit`/`retire`) do not apply; encoder K/V has no token identity; an encoder-decoder model keeps its cross cache as its own [`KVCache`](./KVCache.md) beside the self-attention one. |

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 178

</div>

### enum Layout {#Layout}

<div class="api-signature">

`enum class` **`Layout`** `:` `uint8_t`

</div>

HOW a token-indexed row is laid out. Ignored by the state kinds (their slabs are slot-resident either way).

| Enumerator | Description |
| --- | --- |
| `Auto` | follow the cache's serving `mode` (the default) |
| `Continuous` | a per-slot buffer (preallocated or grow-on-demand) |
| `Paged` | blocks from the cache's shared pool |

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 199

</div>

## Data members

### kind {#kind}

<div class="api-signature">

[`Kind`](./KVLayerSpec.md#Kind) **`kind`** `=` [`Kind::AttentionKV`](./KVLayerSpec.md#Kind)

</div>

WHAT this row stores (see [`Kind`](./KVLayerSpec.md#Kind)).

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 243

</div>

### layout {#layout}

<div class="api-signature">

[`Layout`](./KVLayerSpec.md#Layout) **`layout`** `=` [`Layout::Auto`](./KVLayerSpec.md#Layout)

</div>

HOW a token-indexed row is laid out (see [`Layout`](./KVLayerSpec.md#Layout)); ignored by the state kinds.

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 246

</div>

### kv\_heads {#kv_heads}

<div class="api-signature">

`int64_t` **`kv_heads`** `= 0`

</div>

KV head count for this row. 0 = the cache's `num_kv_heads`. Each row's storage carries its own head count; a continuous row sizes its buffer with it, and paged rows sharing a (kv\_heads, head\_dim) geometry share one pool under the cache's one block table.

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 253

</div>

### head\_dim {#head_dim}

<div class="api-signature">

`int64_t` **`head_dim`** `= 0`

</div>

K-plane head\_dim for this row. 0 = the cache's [`head_dim`](./KVLayerSpec.md#head_dim). A continuous row's buffer is its own; paged rows group by geometry; rows at a divergent head\_dim get their own pool under the shared block table.

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 258

</div>

### v\_head\_dim {#v_head_dim}

<div class="api-signature">

`int64_t` **`v_head_dim`** `= 0`

</div>

> 0 sizes the row's V plane independently of the K plane's head\_dim, the asymmetric-plane row a latent-attention (MLA) layer binds, where the K plane stores the compressed latent (`ckv`, `kv_lora_rank` wide) and the V plane the shared positional rows (`kpe`, `qk_rope_head_dim` wide), typically at one KV head (the latent cache is headless; [`ops::mla_attention`](../ops/mla_attention.md#mla_attention) binds both planes). 0 = both planes at the row's head\_dim. Continuous rows only (a paged pool's planes are symmetric).

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 267

</div>

### window {#window}

<div class="api-signature">

`int64_t` **`window`** `= 0`

</div>

0 = full attention; \> 0 = sliding window (the row keeps this many most-recent tokens). A continuous windowed row is a cyclic ring; a PAGED windowed row WINDOW-RECYCLES; it retains only the blocks covering the trailing [`window`](./KVLayerSpec.md#window) tokens (at most ⌈window/block\_size⌉ + 1 blocks per slot, released back to the pool as the step advances), the memory shape a long-context sliding-attention stack pages for. Its blocks live in their own pool of `max_seqs × (⌈window/block_size⌉ + 1)` blocks per distinct window value, ON TOP of [`PagedParams::num_blocks`](./PagedParams.md#num_blocks) (logged at construction). Windowed paged rows serve the retained window only: reads below it reject, prefix caching skips them (the cache's shareable prefix rides its full-attention rows), one step takes at most [`window`](./KVLayerSpec.md#window) tokens (chunk longer prefills), and a position rewind must stay within the current retained base (up to `block_size − 1` tokens of depth just before a block boundary, zero right after one; deeper rewinds reject; recycled content is gone).

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 284

</div>

### preallocate {#preallocate}

<div class="api-signature">

`bool` **`preallocate`** `= false`

</div>

Continuous rows: allocate the full `max_tokens_per_seq` depth up front instead of growing on demand. Required when an op writes the row's planes IN PLACE (the fused-attention present-aliases-past flow, [`ops::mla_attention`](../ops/mla_attention.md#mla_attention)'s cache append); an in-place writer needs the buffer not to move under it. Default off (grow-on-demand, the `update()`-driven flow). A paged row's blocks are pool-owned; the flag does not apply there. A `CrossAttentionKV` row implies preallocation and IGNORES this field (`fill` is an in-place write).

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 293

</div>

### k\_quant {#k_quant}

<div class="api-signature">

`std::optional<`[`KVQuantSpec`](./KVQuantSpec.md)`>` **`k_quant`**

</div>

Per-row K/V quantization override. Scope: a set value rejects at construction; quantize the K/V sides cache-wide via [`KVCacheConfig::k_quant`](./KVCacheConfig.md#k_quant)/[`v_quant`](./KVLayerSpec.md#v_quant) (per-row kv\_heads / head\_dim divergence is served; a per-row quantization scheme is not). A per-row CALIBRATION SCALE is (see [`k_scale`](./KVLayerSpec.md#k_scale)/[`v_scale`](./KVLayerSpec.md#v_scale) below).

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 299

</div>

### v\_quant {#v_quant}

<div class="api-signature">

`std::optional<`[`KVQuantSpec`](./KVQuantSpec.md)`>` **`v_quant`**

</div>

V-side quantization scheme, [`k_quant`](./KVLayerSpec.md#k_quant)'s twin.

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 301

</div>

### k\_scale {#k_scale}

<div class="api-signature">

`std::optional<double>` **`k_scale`**

</div>

Per-row static calibration scales for an ELEMENT-CODED quantized side: the per-layer [`k_scale`](./KVLayerSpec.md#k_scale)/[`v_scale`](./KVLayerSpec.md#v_scale) scalars fp8-calibrated checkpoints ship, one pair per attention layer. The value is the row's DEQUANT MULTIPLIER: the attend decodes `value = code × scale` and the append encodes `code = value / scale`, the same convention as [`KVQuantSpec::scale`](./KVQuantSpec.md#scale) and the cache-wide checkpoint-convention scalars ([`KVCacheConfig::k_scale`](./KVCacheConfig.md#k_scale)/[`v_scale`](./KVLayerSpec.md#v_scale)), which these replace per row. Unset (the default) = the cache-wide side scale serves the row. Constraints, each raising at `make`:

- `AttentionKV` rows only (a state or cross-attention row carries no served per-row calibration);
- the side must be quantized element-coded: [`KVQuantSpec::block`](./KVQuantSpec.md#block) `None`, `per_token` false, no `kv_cache_scheme` (a block side's scales ride inline; a per-token side derives its own row scales; an unquantized side has no code to scale);
- the value must be \> 0;
- the side's calibration is set in ONE place: overrides may not combine with a non-default [`KVQuantSpec::scale`](./KVQuantSpec.md#scale) or the cache-wide [`KVCacheConfig::k_scale`](./KVCacheConfig.md#k_scale)/[`v_scale`](./KVLayerSpec.md#v_scale) scalar;
- coverage is all-or-nothing per side; when any `AttentionKV` row carries an override, every `AttentionKV` row must (a half-calibrated cache is malformed, never served-with-gaps).

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 324

</div>

### v\_scale {#v_scale}

<div class="api-signature">

`std::optional<double>` **`v_scale`**

</div>

V-side calibration scale, [`k_scale`](./KVLayerSpec.md#k_scale)'s twin.

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 326

</div>

### conv\_states {#conv_states}

<div class="api-signature">

`std::vector<`[`ConvState`](./KVLayerSpec.ConvState.md)`>` **`conv_states`**

</div>

[`ConvState`](./KVLayerSpec.ConvState.md) / `HybridState`: N conv windows (a layer may carry several branch convs; they take the cache's `kv_dtype`).

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 331

</div>

### recurrent\_state {#recurrent_state}

<div class="api-signature">

`std::optional<`[`RecurrentState`](./KVLayerSpec.RecurrentState.md)`>` **`recurrent_state`**

</div>

[`RecurrentState`](./KVLayerSpec.RecurrentState.md) / `HybridState`: the recurrent slab geometry.

<div class="api-declared">

Declared in `ClikaRT/nn/kv_cache.h`, line 333

</div>