---
title: "ClikaRT::ops::add_rms_norm"
sidebar_label: "add_rms_norm"
description: "Fused residual-add + normalization, on either side of the norm."
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--function">function</span>

## add\_rms\_norm() {#add_rms_norm}

<div class="api-signature">

`std::array<`[`Tensor`](../Tensor.md) `2>` **`add_rms_norm`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `input`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `residual` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `residual2` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `post_residual` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`ClikaRT::Span`](../Span.md)`<const std::int64_t>` `normalized_shape` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `skip_bias` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `weight` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `bias` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `eps` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<`[`Activation`](./index.md#Activation)`>` `activation` `=` `std::nullopt`<br />
)

</div>

Fused residual-add + normalization, on either side of the norm. Which data flow runs is inferred from which addends you supply; there is no mode flag:

- `residual` only → `{ACT(`[`norm`](./norm.md#norm)`(x + residual)·w + b), x + residual}`
- `post_residual` only → `{ACT(norm(x)·w + b) + post_residual, UNDEFINED}`
- both → `{ACT(`[`norm`](./norm.md#norm)`(x + residual)·w + b) + post_residual, x + residual}`
- neither → raises (that is a plain [`rms_norm`](./rms_norm.md#rms_norm)/[`layer_norm`](./layer_norm.md#layer_norm))

The **second result is the PRE-norm sum**, the residual stream the next sub-layer reads. Supplying only `post_residual` forms no such sum, so that element comes back UNDEFINED: read only the first one in that flow.

The activation applies to the **norm result, before** `post_residual` is added; it is the norm's epilogue, not the sum's. Gated activations (SwiGlu / GeGlu / ReGlu) narrow their input and are rejected.

**Returns:** a 2-element array \{normalized, pre-norm sum\}. `residual2` is an optional second PRE-norm full-shape addend (a parallel residual stream, e.g. a gated per-layer delta): the pre-norm sum is `x + residual (+ residual2) (+ skip_bias)`. It follows `residual` (the pre-norm addends are adjacent) and precedes `post_residual`.

<div class="api-declared">

Declared in `ClikaRT/compute/ops.h`, line 3600

</div>