Skip to main content

ClikaRT::graph::KVCacheLayerInfo

struct

Header: ClikaRT/graph/model_graph.h

Per-attention-layer KV-cache descriptor: which graph inputs carry the layer's past keys/values in, which graph outputs carry the present keys/values out (any may be empty, a layer with no cache IO), plus the layer's attention properties. A serving loop drives its cache handling off these; generate_kv_input_specs derives compile-time shape specs from them.

Data members​

op_name​

std::string op_name

graph node name of the attention op

Declared in ClikaRT/graph/model_graph.h, line 56

past_key_input​

std::string past_key_input

graph input name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 57

past_value_input​

std::string past_value_input

graph input name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 58

present_key_output​

std::string present_key_output

graph output name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 59

present_value_output​

std::string present_value_output

graph output name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 60

layer_idx​

std::int64_t layer_idx = -1

sequential layer index

Declared in ClikaRT/graph/model_graph.h, line 61

is_self_attention​

bool is_self_attention = true

attends its own sequence

Declared in ClikaRT/graph/model_graph.h, line 62

is_cross_attention​

bool is_cross_attention = false

attends the encoder stream

Declared in ClikaRT/graph/model_graph.h, line 63

sliding_window_size​

std::int64_t sliding_window_size = 0

0 = no sliding window

Declared in ClikaRT/graph/model_graph.h, line 64

num_heads​

std::int64_t num_heads = 0

query heads

Declared in ClikaRT/graph/model_graph.h, line 65

num_kv_heads​

std::int64_t num_kv_heads = 0

K/V heads (grouped-query when fewer than num_heads).

Declared in ClikaRT/graph/model_graph.h, line 66

mask_kind​

AttentionMaskKind mask_kind = AttentionMaskKind::Causal

mask family

Declared in ClikaRT/graph/model_graph.h, line 67

bound_to_cache​

bool bound_to_cache = false

True once ModelGraph::attach_kv_cache bound this layer's past/present slots to the attached nn::KVCache: the layer's K/V live in the cache, the past inputs named above have left the graph's input list (those two fields then read empty), and the present outputs stay, each returned by run(inputs, step) as the layer's cache plane at its present position.

Declared in ClikaRT/graph/model_graph.h, line 73