---
title: "ClikaRT::ops::attention_over_cache"
sidebar_label: "attention_over_cache"
description: "Attend a KV cache WITHOUT appending to it (a read-only re-attention)."
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--function">function</span>

## attention\_over\_cache() {#attention_over_cache}

<div class="api-signature">

[`Tensor`](../Tensor.md) **`attention_over_cache`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `query`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `cache_key`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `cache_value`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `kvcache_start`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `cu_seqlens_q`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `cu_seqlens_k`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `max_seqlen_q` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `max_seqlen_k` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `rope_cos` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `rope_sin` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `position_ids` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `attn_mask` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<bool>` `is_causal` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `q_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `softcap` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<int64_t>` `sliding_window` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<bool>` `smooth_softmax` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<`[`RotaryMode`](./index.md#RotaryMode)`>` `rotary_mode` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<int64_t>` `num_heads` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<int64_t>` `kv_num_heads` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `k_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ScalarOrTensor`](../ScalarOrTensor.md) `v_scale` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `head_sink` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `q_norm_gain` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `k_norm_gain` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `qk_norm_eps` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `slot_ids` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `kept_prefix` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `kv_position_offset` `=` `{}`<br />
)

</div>

Attend a KV cache WITHOUT appending to it (a read-only re-attention). q is packed varlen `[ΣS_q, H, D]` (or hidden-folded `[ΣS_q, num_heads*D]` with `num_heads` set); `cache_key`/`cache_value` are a cache another attention call already appended: continuous head-major `[max_seqs, H_kv, max_seq, D]` with a rank-1 `[B]` `kvcache_start` (the layout selector; its values are not read, and q's position derives from `cu_seqlens_k`), or a paged block pool `[num_blocks, H_kv, block_size, D]` with a rank-2 `[B, max_blocks]` block table (every entry `-1` or in `[0, num_blocks)`, no block index repeated within a row; an out-of-range or repeated entry refuses `INVALID_ARGUMENT` before any read). On the continuous cache `slot_ids` (`[B]` Int32, in range, pairwise distinct) names each batch row's cache row; absent, batch row b reads cache row b. `cu_seqlens_k` is each sequence's TOTAL cached length: per-seq `[B]` or cumulative `[B+1]`. The cache is never written. When `rope_cos`/`rope_sin` are bound, rotary applies to q only (the cached keys are already rotated). Use case: a q-only module re-attending a sibling layer's cache. `head_sink` is the per-head softmax sink `[H_q]`, a virtual logit folded into the softmax denominator, the same contract as [`group_query_attention_varlen`](./group_query_attention_varlen.md#group_query_attention_varlen); it rides the parameter tail here.

**Returns:** out\_attn: q's shape with the trailing dim set to the cache's value head size. [`attention_over_cache`](./attention_over_cache.md#attention_over_cache)'s qk-norm tail rotates and norms q ONLY; the cached keys were already rotated+normed by the call that appended them; `k_norm_gain` rides along untouched so one argument set serves both the appending and the read-only attention. Same travel rule: gains come WITH `qk_norm_eps` and require the rope planes; absent ⇒ unchanged. `kept_prefix` and `kv_position_offset` follow [`group_query_attention_varlen`](./group_query_attention_varlen.md#group_query_attention_varlen): the first P keys of each request stay attended beside the window, and a windowed paged cache's served-key offset places the served keys at their absolute positions for the q-only rotation and the window (pass the cache step's `kv_position_offset` through as-is).

<div class="api-declared">

Declared in `ClikaRT/compute/ops.h`, line 1369

</div>