ClikaRT::nn::KVLayerSpec::RecurrentState

struct

Header: ClikaRT/nn/kv_cache.h

The delta-rule recurrent state geometry (per slot: [heads, k_dim, v_dim]). dtype picks the slab's STORAGE dtype: Float32 (the default reference), or BFloat16 / Float16, half the state bandwidth and checkpoint bytes; the recurrence ops still compute fp32 and round only at the slab round-trip (bf16 keeps f32's exponent range, f16 keeps more mantissa).

quant stores the slab QUANTIZED instead: the element-coded per-tensor SYMMETRIC subset of KVQuantSpec (a symmetric byte code: Int8 / Float8_E4M3 / Float8_E5M2 under one static scale; zero_point / block / per_token must stay at their defaults, and dtype above is then ignored). Quarter the state bandwidth; compute stays fp32, the encode/decode round-trip is the only rounding the state sees.

Data members

heads

int64_t heads = 0

Recurrent heads (the slab's leading extent).

Declared in ClikaRT/nn/kv_cache.h, line 221

k_dim

int64_t k_dim = 0

Per-head key width of the [heads, k_dim, v_dim] slab.

Declared in ClikaRT/nn/kv_cache.h, line 223

v_dim

int64_t v_dim = 0

Per-head value width of the slab.

Declared in ClikaRT/nn/kv_cache.h, line 225

dtype

DataType dtype = DataType::Float32

The slab's STORAGE dtype (compute stays fp32); Float32 default, BFloat16 / Float16 halve the state bandwidth.

Declared in ClikaRT/nn/kv_cache.h, line 228

quant

std::optional<KVQuantSpec> quant

Optional quantized storage instead of dtype, the symmetric element-coded subset only (see the struct doc).

Declared in ClikaRT/nn/kv_cache.h, line 231