---
title: "ClikaRT::ops::scaled_dot_product_attention_varlen"
sidebar_label: "scaled_dot_product_attention_varlen"
description: "Variable-length (packed) scaled dot-product attention: ragged batches ride one token-packed tensor plus prefix-sum offsets, no padding."
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--function">function</span>

## scaled\_dot\_product\_attention\_varlen() {#scaled_dot_product_attention_varlen}

<div class="api-signature">

[`Tensor`](../Tensor.md) **`scaled_dot_product_attention_varlen`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `query`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `key`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `value`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `cu_seqlens_q`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `cu_seqlens_k`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `max_seqlen_q` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `max_seqlen_k` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `attn_mask` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`bool` `is_causal` `=` `false`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `q_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `k_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `v_scale` `=` `{}`<br />
)

</div>

Variable-length (packed) scaled dot-product attention: ragged batches ride one token-packed tensor plus prefix-sum offsets, no padding.

Same math as [`scaled_dot_product_attention`](./scaled_dot_product_attention.md#scaled_dot_product_attention); the batch structure moves into `cu_seqlens_*`.

**Parameters**

- `query`: packed queries `[Sum(S_q), H_q, D]` (token-major).
- `key`: packed keys `[Sum(S_kv), H_kv, D]`.
- `value`: packed values `[Sum(S_kv), H_kv, Dv]`.
- `cu_seqlens_q`: `[B+1]` Int32 prefix sums of the per-sequence query lengths.
- `cu_seqlens_k`: `[B+1]` Int32 prefix sums of the per-sequence KV lengths.
- `max_seqlen_q`: optional host bound on the longest query run (an integer, or a HOST-resident 0-D Int32/Int64 tensor).
- `max_seqlen_k`: optional host bound on the longest KV run, same forms.
- `attn_mask`: optional, broadcastable to `[B, H_q, S_q, S_kv]`.
- `is_causal`: apply the causal mask per sequence. Default `false`.
- `q_scale`: as in [`scaled_dot_product_attention`](./scaled_dot_product_attention.md#scaled_dot_product_attention).
- `k_scale`: as in [`scaled_dot_product_attention`](./scaled_dot_product_attention.md#scaled_dot_product_attention).
- `v_scale`: as in [`scaled_dot_product_attention`](./scaled_dot_product_attention.md#scaled_dot_product_attention).

**Throws**

- `ClikaRT::Error`: when the input's dtype/shape is not served for this operation (the machine-readable reason rides `code_name()`).

**Returns:** packed `[Sum(S_q), H_q, Dv]`, dtype of `query`.

<div class="api-declared">

Declared in `ClikaRT/compute/ops.h`, line 1211

</div>