Skip to main content

ClikaRT::nn::AttentionOptions

struct

Header: ClikaRT/nn/attention.h

Settings of Attention: chain the setters, or assign the fields. The defaults are plain non-causal attention over head-split inputs.

Member functions​

num_heads(int64_t)​

AttentionOptions& num_heads(std::int64_t v)

Sets num_heads, the query head count of hidden-folded inputs.

Declared in ClikaRT/nn/attention.h, line 37

kv_num_heads(int64_t)​

AttentionOptions& kv_num_heads(std::int64_t v)

Sets kv_num_heads, the key / value head count of hidden-folded inputs.

Declared in ClikaRT/nn/attention.h, line 39

is_causal(bool)​

AttentionOptions& is_causal(bool v)

Sets is_causal.

Declared in ClikaRT/nn/attention.h, line 41

scale(double)​

AttentionOptions& scale(double v)

Sets scale, the softmax pre-scale.

Declared in ClikaRT/nn/attention.h, line 43

softcap(double)​

AttentionOptions& softcap(double v)

Sets softcap, the logit soft-cap.

Declared in ClikaRT/nn/attention.h, line 45

sliding_window(int64_t)​

AttentionOptions& sliding_window(std::int64_t v)

Sets sliding_window, the key window each query attends.

Declared in ClikaRT/nn/attention.h, line 47

sinks(bool)​

AttentionOptions& sinks(bool v)

Sets sinks: declare the learned per-head softmax sink slot.

Declared in ClikaRT/nn/attention.h, line 49

device(Device)​

AttentionOptions& device(Device v)

Sets device, where the bound tensors live.

Declared in ClikaRT/nn/attention.h, line 51

num_heads()​

std::int64_t num_heads() const

Reads num_heads.

Declared in ClikaRT/nn/attention.h, line 54

kv_num_heads()​

std::int64_t kv_num_heads() const

Reads kv_num_heads.

Declared in ClikaRT/nn/attention.h, line 56

is_causal()​

bool is_causal() const

Reads is_causal.

Declared in ClikaRT/nn/attention.h, line 58

scale()​

double scale() const

Reads scale.

Declared in ClikaRT/nn/attention.h, line 60

softcap()​

double softcap() const

Reads softcap.

Declared in ClikaRT/nn/attention.h, line 62

sliding_window()​

std::int64_t sliding_window() const

Reads sliding_window.

Declared in ClikaRT/nn/attention.h, line 64

sinks()​

bool sinks() const

Reads sinks.

Declared in ClikaRT/nn/attention.h, line 66

device()​

const Device& device() const

Reads device.

Declared in ClikaRT/nn/attention.h, line 68

Data members​

num_heads_​

std::int64_t num_heads_ = 0

The query head count when forward receives HIDDEN-FOLDED inputs ([B, S, num_heads * head_dim], the projection output as is; the module splits the heads and folds the output back). 0 (the default) means the inputs arrive head-split, [B, H, S, D].

Declared in ClikaRT/nn/attention.h, line 74

kv_num_heads_​

std::int64_t kv_num_heads_ = 0

The key / value head count of hidden-folded inputs; 0 (the default) = num_heads. Head-split inputs carry their own key / value head count, which divides the query head count (grouped-query attention).

Declared in ClikaRT/nn/attention.h, line 78

is_causal_​

bool is_causal_ = false

Apply the causal (lower-triangular) mask. Default false.

Declared in ClikaRT/nn/attention.h, line 80

scale_​

double scale_ = 0.0

The softmax pre-scale; 0 (the default) selects 1 / sqrt(head_dim).

Declared in ClikaRT/nn/attention.h, line 82

softcap_​

double softcap_ = 0.0

The logit soft-cap, cap * tanh(x / cap) before the softmax; 0 (the default) = off.

Declared in ClikaRT/nn/attention.h, line 85

sliding_window_​

std::int64_t sliding_window_ = -1

Each query attends only the last sliding_window key positions; -1 (the default) = no window.

Declared in ClikaRT/nn/attention.h, line 88

sinks_​

bool sinks_ = false

Declare the learned per-head softmax sink as the parameter slot sinks ([num_heads]), so a checkpoint binds it by name; needs num_heads > 0. Default false (bind one with set_sinks instead, or attend without a sink).

Declared in ClikaRT/nn/attention.h, line 93

device_​

Device device_ = Device::cpu()

Where the bound tensors live (a bind moves its payload there). Default CPU.

Declared in ClikaRT/nn/attention.h, line 96