ClikaRT::nn::KVLayerSpec

struct

Header: ClikaRT/nn/kv_cache.h

One decoder layer's storage row, the per-layer entry of the spec vector KVCache::make consumes. Two orthogonal axes: Kind says WHAT the row stores, Layout says HOW a token-indexed row is laid out.

Nested types

Name	Description
ConvState	One short-conv rolling window: dim channels × width taps, the slab causal_conv_update binds as its state.
RecurrentState	The delta-rule recurrent state geometry (per slot: [heads, k_dim, v_dim]). dtype picks the slab's STORAGE dtype: Float32 (the default reference), or BFloat16 / Float16, half the state bandwidth and checkpoint bytes; the recurrence ops still compute fp32 and round only at the slab round-trip (bf16 keeps f32's exponent range, f16 keeps more mantissa).

Types

enum Kind

enum class Kind : uint8_t

WHAT the row stores. Orthogonal to how it is laid out.

Enumerator	Description
AttentionKV	token-indexed K/V (full, or windowed via window)
ConvState	causal-conv rolling windows only
RecurrentState	delta-rule / SSM recurrent slab only
HybridState	conv windows + a recurrent slab on ONE row
CrossAttentionKV	Encoder K/V for cross-attention: written ONCE per encoder pass via KVCache::fill (length = the encoder length), read by every decode step, NEVER appended; the per-step append path rejects these rows. Layout::Auto resolves Continuous and an explicit Layout::Paged rejects at make (a fill-once row gains nothing from a block pool); preallocate is implied true and the field is IGNORED (fill writes the planes in place, so the buffer must not move); window rejects (a sliding window on a fill-once row is a contradiction). The prefix-cache verbs (admit/retire) do not apply; encoder K/V has no token identity; an encoder-decoder model keeps its cross cache as its own KVCache beside the self-attention one.

Declared in ClikaRT/nn/kv_cache.h, line 170

enum Layout

enum class Layout : uint8_t

HOW a token-indexed row is laid out. Ignored by the state kinds (their slabs are slot-resident either way).

Enumerator	Description
Auto	follow the cache's serving mode (the default)
Continuous	a per-slot buffer (preallocated or grow-on-demand)
Paged	blocks from the cache's shared pool

Declared in ClikaRT/nn/kv_cache.h, line 191

Data members

kind

Kind kind = Kind::AttentionKV

WHAT this row stores (see Kind).

Declared in ClikaRT/nn/kv_cache.h, line 235

layout

Layout layout = Layout::Auto

HOW a token-indexed row is laid out (see Layout); ignored by the state kinds.

Declared in ClikaRT/nn/kv_cache.h, line 238

kv_heads

int64_t kv_heads = 0

KV head count for this row. 0 = the cache's num_kv_heads. Each row's storage carries its own head count; a continuous row sizes its buffer with it, and paged rows sharing a (kv_heads, head_dim) geometry share one pool under the cache's one block table.

Declared in ClikaRT/nn/kv_cache.h, line 245

head_dim

int64_t head_dim = 0

K-plane head_dim for this row. 0 = the cache's head_dim. A continuous row's buffer is its own; paged rows group by geometry; rows at a divergent head_dim get their own pool under the shared block table.

Declared in ClikaRT/nn/kv_cache.h, line 250

v_head_dim

int64_t v_head_dim = 0

0 sizes the row's V plane independently of the K plane's head_dim, the asymmetric-plane row a latent-attention (MLA) layer binds, where the K plane stores the compressed latent (ckv, kv_lora_rank wide) and the V plane the shared positional rows (kpe, qk_rope_head_dim wide), typically at one KV head (the latent cache is headless; ops::mla_attention binds both planes). 0 = both planes at the row's head_dim. Continuous rows only (a paged pool's planes are symmetric).

Declared in ClikaRT/nn/kv_cache.h, line 259

window

int64_t window = 0

0 = full attention; > 0 = sliding window (the row keeps this many most-recent tokens). A continuous windowed row is a cyclic ring; a PAGED windowed row WINDOW-RECYCLES; it retains only the blocks covering the trailing window tokens (at most ⌈window/block_size⌉ + 1 blocks per slot, released back to the pool as the step advances), the memory shape a long-context sliding-attention stack pages for. Its blocks live in their own pool of max_seqs × (⌈window/block_size⌉ + 1) blocks per distinct window value, ON TOP of PagedParams::num_blocks (logged at construction). Windowed paged rows serve the retained window only: reads below it reject, prefix caching skips them (the cache's shareable prefix rides its full-attention rows), one step takes at most window tokens (chunk longer prefills), and a position rewind must stay within the current retained base (up to block_size − 1 tokens of depth just before a block boundary, zero right after one; deeper rewinds reject; recycled content is gone).

Declared in ClikaRT/nn/kv_cache.h, line 276

preallocate

bool preallocate = false

Continuous rows: allocate the full max_tokens_per_seq depth up front instead of growing on demand. Required when an op writes the row's planes IN PLACE (the fused-attention present-aliases-past flow, ops::mla_attention's cache append); an in-place writer needs the buffer not to move under it. Default off (grow-on-demand, the update()-driven flow). A paged row's blocks are pool-owned; the flag does not apply there. A CrossAttentionKV row implies preallocation and IGNORES this field (fill is an in-place write).

Declared in ClikaRT/nn/kv_cache.h, line 285

k_quant

std::optional<KVQuantSpec> k_quant

Per-row K/V quantization override. Scope: a set value rejects at construction; quantize the K/V sides cache-wide via KVCacheConfig::k_quant/v_quant (per-row kv_heads / head_dim divergence is served; a per-row quantization scheme is not). A per-row CALIBRATION SCALE is (see k_scale/v_scale below).

Declared in ClikaRT/nn/kv_cache.h, line 291

v_quant

std::optional<KVQuantSpec> v_quant

V-side quantization scheme, k_quant's twin.

Declared in ClikaRT/nn/kv_cache.h, line 293

k_scale

std::optional<double> k_scale

Per-row static calibration scales for an ELEMENT-CODED quantized side: the per-layer k_scale/v_scale scalars fp8-calibrated checkpoints ship, one pair per attention layer. The value is the row's DEQUANT MULTIPLIER: the attend decodes value = code × scale and the append encodes code = value / scale, the same convention as KVQuantSpec::scale and the cache-wide checkpoint-convention scalars (KVCacheConfig::k_scale/v_scale), which these replace per row. Unset (the default) = the cache-wide side scale serves the row. Constraints, each raising at make:

• AttentionKV rows only (a state or cross-attention row carries no served per-row calibration);
• the side must be quantized element-coded: KVQuantSpec::block None, per_token false, no kv_cache_scheme (a block side's scales ride inline; a per-token side derives its own row scales; an unquantized side has no code to scale);
• the value must be > 0;
• the side's calibration is set in ONE place: overrides may not combine with a non-default KVQuantSpec::scale or the cache-wide KVCacheConfig::k_scale/v_scale scalar;
• coverage is all-or-nothing per side; when any AttentionKV row carries an override, every AttentionKV row must (a half-calibrated cache is malformed, never served-with-gaps).

Declared in ClikaRT/nn/kv_cache.h, line 316

v_scale

std::optional<double> v_scale

V-side calibration scale, k_scale's twin.

Declared in ClikaRT/nn/kv_cache.h, line 318

conv_states

std::vector<ConvState> conv_states

ConvState / HybridState: N conv windows (a layer may carry several branch convs; they take the cache's kv_dtype).

Declared in ClikaRT/nn/kv_cache.h, line 323

recurrent_state

std::optional<RecurrentState> recurrent_state

RecurrentState / HybridState: the recurrent slab geometry.

Declared in ClikaRT/nn/kv_cache.h, line 325