---
title: "ClikaRT::ops::qk_rms_norm"
sidebar_label: "qk_rms_norm"
description: "Per-head RMS norm over packed attention projections: query, key and value in ONE call, no reshapes."
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--function">function</span>

## qk\_rms\_norm() {#qk_rms_norm}

<div class="api-signature">

`std::array<`[`Tensor`](../Tensor.md) `3>` **`qk_rms_norm`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `query`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `key` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `value` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `query_weight` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `key_weight` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::int64_t` `head_dim` `=` `0`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `eps` `=` `std::nullopt`<br />
)

</div>

Per-head RMS norm over packed attention projections: query, key and value in ONE call, no reshapes.

Each contiguous `head_dim` run of `query` (and `key`, when present) is its own normalization group: `out = x / `[`sqrt`](./sqrt.md#sqrt)`(mean(x^2) + eps) * w`. `value` passes through untouched (it rides along so one call serves the projection triplet). Equivalent to reshaping `[S, heads*head_dim]` to `[S, heads, head_dim]`, applying [`rms_norm`](./rms_norm.md#rms_norm), and reshaping back, with none of those steps.

**Parameters**

- `query`: the query projection, `[.., heads*head_dim]`.
- `key`: optional key projection, same trailing structure.
- `value`: optional value projection, same trailing structure; when given it is normalized too, with no gain (it has no weight slot: the scale-less per-head value norm).
- `query_weight`: optional per-head gain for `query`, shape `[head_dim]`.
- `key_weight`: optional per-head gain for `key`, shape `[head_dim]`.
- `head_dim`: the per-head run length; must divide the trailing extent.
- `eps`: stability floor; `std::nullopt` selects 1e-5.

**Returns:** `{q_normed, k_normed, v_normed}`; absent inputs come back absent.

**Throws**

- `ClikaRT::Error`: (`INVALID_ARGUMENT`) when `head_dim` does not divide the trailing extent or a gain is mis-shaped.

<div class="api-declared">

Declared in `ClikaRT/compute/ops.h`, line 3625

</div>