Skip to main content

ClikaRT::graph::KVCacheLayerInfo

struct

Header: ClikaRT/graph/model_graph.h

Per-attention-layer KV-cache descriptor: which graph inputs carry the layer's past keys/values in, which graph outputs carry the present keys/values out (any may be empty, a layer with no cache IO), plus the layer's attention properties. A serving loop drives its cache handling off these; generate_kv_input_specs derives compile-time shape specs from them.

Data members​

op_name​

std::string op_name

graph node name of the attention op

Declared in ClikaRT/graph/model_graph.h, line 66

past_key_input​

std::string past_key_input

graph input name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 67

past_value_input​

std::string past_value_input

graph input name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 68

present_key_output​

std::string present_key_output

graph output name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 69

present_value_output​

std::string present_value_output

graph output name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 70

layer_idx​

std::int64_t layer_idx = -1

sequential layer index

Declared in ClikaRT/graph/model_graph.h, line 71

is_self_attention​

bool is_self_attention = true

attends its own sequence

Declared in ClikaRT/graph/model_graph.h, line 72

is_cross_attention​

bool is_cross_attention = false

attends the encoder stream

Declared in ClikaRT/graph/model_graph.h, line 73

sliding_window_size​

std::int64_t sliding_window_size = 0

0 = no sliding window

Declared in ClikaRT/graph/model_graph.h, line 74

num_heads​

std::int64_t num_heads = 0

query heads

Declared in ClikaRT/graph/model_graph.h, line 75

num_kv_heads​

std::int64_t num_kv_heads = 0

K/V heads (grouped-query when fewer than num_heads).

Declared in ClikaRT/graph/model_graph.h, line 76

mask_kind​

AttentionMaskKind mask_kind = AttentionMaskKind::Causal

mask family

Declared in ClikaRT/graph/model_graph.h, line 77

bound_to_cache​

bool bound_to_cache = false

True once ModelGraph::attach_kv_cache bound this layer's past/present slots to the attached nn::KVCache: the layer's K/V live in the cache, the past inputs named above have left the graph's input list (those two fields then read empty), and the present outputs stay, each returned by run(inputs, step) as the layer's cache plane at its present position.

Declared in ClikaRT/graph/model_graph.h, line 83