Skip to main content

ClikaRT::graph::KVCacheLayerInfo

struct

Header: ClikaRT/graph/model_graph.h

Per-attention-layer KV-cache descriptor: which graph inputs carry the layer's past keys/values in, which graph outputs carry the present keys/values out (any may be empty, a layer with no cache IO), plus the layer's attention properties. A serving loop drives its cache handling off these; generate_kv_input_specs derives compile-time shape specs from them.

Data members​

op_name​

std::string op_name

graph node name of the attention op

Declared in ClikaRT/graph/model_graph.h, line 48

past_key_input​

std::string past_key_input

graph input name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 49

past_value_input​

std::string past_value_input

graph input name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 50

present_key_output​

std::string present_key_output

graph output name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 51

present_value_output​

std::string present_value_output

graph output name (empty if none)

Declared in ClikaRT/graph/model_graph.h, line 52

layer_idx​

std::int64_t layer_idx = -1

sequential layer index

Declared in ClikaRT/graph/model_graph.h, line 53

is_self_attention​

bool is_self_attention = true

attends its own sequence

Declared in ClikaRT/graph/model_graph.h, line 54

is_cross_attention​

bool is_cross_attention = false

attends the encoder stream

Declared in ClikaRT/graph/model_graph.h, line 55

sliding_window_size​

std::int64_t sliding_window_size = 0

0 = no sliding window

Declared in ClikaRT/graph/model_graph.h, line 56

num_heads​

std::int64_t num_heads = 0

query heads

Declared in ClikaRT/graph/model_graph.h, line 57

num_kv_heads​

std::int64_t num_kv_heads = 0

K/V heads (grouped-query when fewer than num_heads).

Declared in ClikaRT/graph/model_graph.h, line 58

mask_kind​

AttentionMaskKind mask_kind = AttentionMaskKind::Causal

mask family

Declared in ClikaRT/graph/model_graph.h, line 59