---
title: "ClikaRT::ops::group_query_attention_varlen"
sidebar_label: "group_query_attention_varlen"
description: "The varlen variant of the fused GQA above, with the same qk-norm tail: q_norm_gain/k_norm_gain (rank-1 [head_dim], post-rope, applied before the K/V append) travel with qk_norm_eps (the model's rms-no…"
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--function">function</span>

## group\_query\_attention\_varlen() {#group_query_attention_varlen}

<div class="api-signature">

`std::array<`[`Tensor`](../Tensor.md) `3>` **`group_query_attention_varlen`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `query`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `key`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `value`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `cu_seqlens_q`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `cu_seqlens_k`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `max_seqlen_q` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `max_seqlen_k` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `past_key` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `past_value` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `kvcache_start` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `rope_cos` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `rope_sin` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `position_ids` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `attn_mask` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `head_sink` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<bool>` `is_causal` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `q_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `softcap` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<int64_t>` `sliding_window` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<bool>` `smooth_softmax` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<`[`RotaryMode`](./index.md#RotaryMode)`>` `rotary_mode` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<int64_t>` `num_heads` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<int64_t>` `kv_num_heads` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `out_present_key` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `out_present_value` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `k_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `v_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `q_norm_gain` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `k_norm_gain` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `qk_norm_eps` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `slot_ids` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `kept_prefix` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `kv_position_offset` `=` `{}`<br />
)

</div>

The varlen variant of the fused GQA above, with the same qk-norm tail: `q_norm_gain`/`k_norm_gain` (rank-1 `[head_dim]`, post-rope, applied before the K/V append) travel with `qk_norm_eps` (the model's rms-norm epsilon) and require the in-op rope planes; absent ⇒ unchanged. The gains are the after-rotation order; a model that norms its heads before the rotation (the Qwen3 order) runs [`qk_rms_norm`](./qk_rms_norm.md#qk_rms_norm) on its projections ahead of this op and passes no gains. The same `slot_ids` contract: on a continuous cache it names each batch row's cache row (`[B]` Int32, in range, pairwise distinct; absent = row b for batch row b), and `kvcache_start`'s values stay unread there (the write offsets derive from `cu_seqlens_k`). The same block-table contract on a paged pool: every entry of the `[B, max_blocks]` table is `-1` or a block index in `[0, num_blocks)`, and no two entries of one row name the same physical block; an out-of-range or repeated entry refuses `INVALID_ARGUMENT` before any read or append. The same read-only rule: with `key` and `value` absent it takes no present outputs; use [`attention_over_cache`](./attention_over_cache.md#attention_over_cache), or leave `out_present_key`/`out_present_value` unbound. `kept_prefix` keeps each request's first P keys attended beside the sliding window (a 0-D value or `[B]`, Int32 or Int64), the same law as [`attention`](./attention.md#attention)'s. `kv_position_offset` (`[B]`, Int32 or Int64) is each paged row's served-key offset: when a windowed paged cache serves only a row's recent blocks, its served key at view position i sits at absolute position i + offset, so the in-op rotary and the window read true positions; pass `KVCache::StepIndices::kv_position_offset` through as-is (undefined on every other cache).

<div class="api-declared">

Declared in `ClikaRT/compute/ops.h`, line 1343

</div>