---
title: "ClikaRT::ops::attention_varlen"
sidebar_label: "attention_varlen"
description: "Variable-length (packed) form of attention: the serving riders over token-packed ragged batches."
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--function">function</span>

## attention\_varlen() {#attention_varlen}

<div class="api-signature">

[`Tensor`](../Tensor.md) **`attention_varlen`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `query`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `key`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `value`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `cu_seqlens_q`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `cu_seqlens_k`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `max_seqlen_q` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `max_seqlen_k` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `attn_mask` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `head_sink` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<bool>` `is_causal` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `q_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `softcap` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<int64_t>` `sliding_window` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<bool>` `smooth_softmax` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `k_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `v_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `kept_prefix` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `kv_position_offset` `=` `{}`<br />
)

</div>

Variable-length (packed) form of [`attention`](./attention.md#attention): the serving riders over token-packed ragged batches.

Tensors and offsets follow [`scaled_dot_product_attention_varlen`](./scaled_dot_product_attention_varlen.md#scaled_dot_product_attention_varlen); the riders (`head_sink`, `softcap`, `sliding_window`, `smooth_softmax`) follow [`attention`](./attention.md#attention).

**Parameters**

- `query`: packed queries `[Sum(S_q), H_q, D]`.
- `key`: packed keys `[Sum(S_kv), H_kv, D]`.
- `value`: packed values `[Sum(S_kv), H_kv, Dv]`.
- `cu_seqlens_q`: `[B+1]` Int32 query prefix sums.
- `cu_seqlens_k`: `[B+1]` Int32 KV prefix sums.
- `max_seqlen_q`: optional host bound (int or HOST-resident 0-D tensor).
- `max_seqlen_k`: optional host bound, same forms.
- `attn_mask`: optional, broadcastable to `[B, H_q, S_q, S_kv]`.
- `head_sink`: optional `[H_q]` per-head softmax sink.
- `is_causal`: optional causal mask; absent = not causal.
- `q_scale`: softmax pre-scale.
- `softcap`: optional logit soft-cap.
- `sliding_window`: optional window length.
- `smooth_softmax`: optional; `true` adds 1 to the denominator.
- `k_scale`: quantized-key dequant scale.
- `v_scale`: quantized-value dequant scale.
- `kept_prefix`: optional: each request's first P keys stay attended beside the sliding window (a 0-D value for every request, or `[B]` one per request; Int32 or Int64), the same law as [`attention`](./attention.md#attention)'s.
- `kv_position_offset`: optional `[B]` (Int32 or Int64): each paged row's served-key offset. When a windowed paged cache serves only a row's recent blocks, its served key at view position i sits at absolute position i + offset, so rotary and the window read true positions. Pass `KVCache::StepIndices::kv_position_offset` through as-is; it is undefined on every other cache.

**Throws**

- `ClikaRT::Error`: when the input's dtype/shape is not served for this operation (the machine-readable reason rides `code_name()`).

**Returns:** packed `[Sum(S_q), H_q, Dv]`, dtype of `query`.

<div class="api-declared">

Declared in `ClikaRT/compute/ops.h`, line 1285

</div>