---
title: "ClikaRT::nn::AttentionOptions"
sidebar_label: "AttentionOptions"
description: "Settings of Attention: chain the setters, or assign the fields. The defaults are plain non-causal attention over head-split inputs."
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--struct">struct</span>

**Header:** `ClikaRT/nn/attention.h`

Settings of [`Attention`](./Attention.md): chain the setters, or assign the fields. The defaults are plain non-causal attention over head-split inputs.

## Member functions

### num\_heads(int64\_t) {#num_heads}

<div class="api-signature">

[`AttentionOptions`](./AttentionOptions.md)`&` **`num_heads`**(`std::int64_t` `v`)

</div>

Sets [`num_heads`](./AttentionOptions.md#num_heads), the query head count of hidden-folded inputs.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 40

</div>

### kv\_num\_heads(int64\_t) {#kv_num_heads}

<div class="api-signature">

[`AttentionOptions`](./AttentionOptions.md)`&` **`kv_num_heads`**(`std::int64_t` `v`)

</div>

Sets [`kv_num_heads`](./AttentionOptions.md#kv_num_heads), the key / value head count of hidden-folded inputs.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 42

</div>

### is\_causal(bool) {#is_causal}

<div class="api-signature">

[`AttentionOptions`](./AttentionOptions.md)`&` **`is_causal`**(`bool` `v`)

</div>

Sets [`is_causal`](./AttentionOptions.md#is_causal).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 44

</div>

### scale(double) {#scale}

<div class="api-signature">

[`AttentionOptions`](./AttentionOptions.md)`&` **`scale`**(`double` `v`)

</div>

Sets [`scale`](./AttentionOptions.md#scale), the softmax pre-scale.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 46

</div>

### softcap(double) {#softcap}

<div class="api-signature">

[`AttentionOptions`](./AttentionOptions.md)`&` **`softcap`**(`double` `v`)

</div>

Sets [`softcap`](./AttentionOptions.md#softcap), the logit soft-cap.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 48

</div>

### sliding\_window(int64\_t) {#sliding_window}

<div class="api-signature">

[`AttentionOptions`](./AttentionOptions.md)`&` **`sliding_window`**(`std::int64_t` `v`)

</div>

Sets [`sliding_window`](./AttentionOptions.md#sliding_window), the key window each query attends.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 50

</div>

### sinks(bool) {#sinks}

<div class="api-signature">

[`AttentionOptions`](./AttentionOptions.md)`&` **`sinks`**(`bool` `v`)

</div>

Sets [`sinks`](./AttentionOptions.md#sinks): declare the learned per-head softmax sink slot.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 52

</div>

### device(Device) {#device}

<div class="api-signature">

[`AttentionOptions`](./AttentionOptions.md)`&` **`device`**([`Device`](../Device.md) `v`)

</div>

Sets [`device`](../device-namespace/index.md), where the bound tensors live.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 54

</div>

### num\_heads() {#num_heads-2}

<div class="api-signature">

`std::int64_t` **`num_heads`**() `const`

</div>

Reads [`num_heads`](./AttentionOptions.md#num_heads).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 57

</div>

### kv\_num\_heads() {#kv_num_heads-2}

<div class="api-signature">

`std::int64_t` **`kv_num_heads`**() `const`

</div>

Reads [`kv_num_heads`](./AttentionOptions.md#kv_num_heads).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 59

</div>

### is\_causal() {#is_causal-2}

<div class="api-signature">

`bool` **`is_causal`**() `const`

</div>

Reads [`is_causal`](./AttentionOptions.md#is_causal).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 61

</div>

### scale() {#scale-2}

<div class="api-signature">

`double` **`scale`**() `const`

</div>

Reads [`scale`](./AttentionOptions.md#scale).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 63

</div>

### softcap() {#softcap-2}

<div class="api-signature">

`double` **`softcap`**() `const`

</div>

Reads [`softcap`](./AttentionOptions.md#softcap).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 65

</div>

### sliding\_window() {#sliding_window-2}

<div class="api-signature">

`std::int64_t` **`sliding_window`**() `const`

</div>

Reads [`sliding_window`](./AttentionOptions.md#sliding_window).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 67

</div>

### sinks() {#sinks-2}

<div class="api-signature">

`bool` **`sinks`**() `const`

</div>

Reads [`sinks`](./AttentionOptions.md#sinks).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 69

</div>

### device() {#device-2}

<div class="api-signature">

`const` [`Device`](../Device.md)`&` **`device`**() `const`

</div>

Reads [`device`](../device-namespace/index.md).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 71

</div>

## Data members

### num\_heads\_ {#num_heads_}

<div class="api-signature">

`std::int64_t` **`num_heads_`** `= 0`

</div>

The query head count when `forward` receives HIDDEN-FOLDED inputs (`[B, S, `[`num_heads`](./AttentionOptions.md#num_heads)` * head_dim]`, the projection output as is; the module splits the heads and folds the output back). `0` (the default) means the inputs arrive head-split, `[B, H, S, D]`.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 77

</div>

### kv\_num\_heads\_ {#kv_num_heads_}

<div class="api-signature">

`std::int64_t` **`kv_num_heads_`** `= 0`

</div>

The key / value head count of hidden-folded inputs; `0` (the default) = [`num_heads`](./AttentionOptions.md#num_heads). Head-split inputs carry their own key / value head count, which divides the query head count (grouped-query attention).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 81

</div>

### is\_causal\_ {#is_causal_}

<div class="api-signature">

`bool` **`is_causal_`** `= false`

</div>

Apply the causal (lower-triangular) mask. Default `false`.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 83

</div>

### scale\_ {#scale_}

<div class="api-signature">

`double` **`scale_`** `= 0.0`

</div>

The softmax pre-scale; `0` (the default) selects `1 / sqrt(head_dim)`.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 85

</div>

### softcap\_ {#softcap_}

<div class="api-signature">

`double` **`softcap_`** `= 0.0`

</div>

The logit soft-cap, `cap * tanh(x / cap)` before the softmax; `0` (the default) = off.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 88

</div>

### sliding\_window\_ {#sliding_window_}

<div class="api-signature">

`std::int64_t` **`sliding_window_`** `= -1`

</div>

Each query attends only the last [`sliding_window`](./AttentionOptions.md#sliding_window) key positions; `-1` (the default) = no window.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 91

</div>

### sinks\_ {#sinks_}

<div class="api-signature">

`bool` **`sinks_`** `= false`

</div>

Declare the learned per-head softmax sink as the parameter slot [`sinks`](./AttentionOptions.md#sinks) (`[`[`num_heads`](./AttentionOptions.md#num_heads)`]`), so a checkpoint binds it by name; needs [`num_heads`](./AttentionOptions.md#num_heads)` > 0`. Default `false` (bind one with `set_sinks` instead, or attend without a sink).

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 96

</div>

### device\_ {#device_}

<div class="api-signature">

[`Device`](../Device.md) **`device_`** `=` [`Device::cpu`](../Device.md#cpu)`()`

</div>

[Where](./Where.md) the bound tensors live (a bind moves its payload there). Default CPU.

<div class="api-declared">

Declared in `ClikaRT/nn/attention.h`, line 99

</div>