---
title: "ClikaRT::nn::RMSNorm"
sidebar_label: "RMSNorm"
description: "Root-mean-square normalization module. \\[ y_i = \\frac{x_i}{\\sqrt{\\tfrac{1}{n}\\sum_j x_j^2 + \\varepsilon}} \\cdot \\gamma_i \\] The RMS is taken over the trailing normalized_shape dims, the module form of…"
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--class">class</span>

**Header:** `ClikaRT/nn/rms_norm.h`  
**Inherits:** [`ClikaRT::nn::Module`](./Module.md)

A bound RMSNorm, the public face of the runtime's `ops::rms_norm`, exposed as an `nn::Module` leaf. Unlike the stateless `ops::rms_norm` free function, an `RMSNorm` binds its `weight` (gain) / `bias` ONCE and materializes them into contiguous layout at the first forward, so every later `forward` reuses the bound params; build one per norm site at load, reuse it every step.

The gain / bias stay in their OWN dtype (a fp32 gain against a bf16 activation is read at full precision, never cast at rest); the build does layout work only. The output dtype follows the input `x`.

The lifecycle (uniform across every weight-bearing `nn` module): `make(normalized_size, ...)` registers storage-free `weight` / `bias` slots under their canonical names; `set_weights(...)` (positional) or `load_state_dict(...)` (a checkpoint, by dotted name) binds them; the FIRST `forward` materializes contiguous (once, thread-safe). `LoadOptions::pack_on_load` pays the build at load time.

Held via `std::shared_ptr` (an `nn::Module` leaf); copy/move are pinned by the base.

Root-mean-square normalization module.

$$
y_i = \frac{x_i}{\sqrt{\tfrac{1}{n}\sum_j x_j^2 + \varepsilon}}
\cdot \gamma_i
$$

The RMS is taken over the trailing `normalized_shape` dims, the module form of [`ops::rms_norm`](../ops/rms_norm.md#rms_norm), holding gamma as its weight. The module owns its weights: construct with `make(...)`, or declare shapes and bind a checkpoint via [`load_state_dict`](./Module.md#load_state_dict). The first forward materializes the bound slots contiguous (or [`LoadOptions::pack_on_load`](./LoadOptions.md#pack_on_load) does it at the end of the load); the module holds each weight once and [`named_parameters()`](./Module.md#named_parameters) / [`state_dict()`](./Module.md#state_dict) stay complete; `to(dtype)` casts the slots.

```cpp
auto norm = ClikaRT::nn::RMSNorm::make(gamma);
auto y    = norm->forward(x);
```

## Static member functions

### make() {#make}

<div class="api-signature">

`static` `std::shared_ptr<RMSNorm>` **`make`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::int64_t` `normalized_size`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`bool` `bias` `=` `true`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`double` `eps` `=` `1e-6`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<`[`ops::Activation`](../ops/index.md#Activation)`>` `activation` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`DataType`](../../index.md#DataType) `dtype` `=` [`DataType::Float32`](../../index.md#DataType),<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Device`](../Device.md) `device` `=` [`Device::cpu`](../Device.md#cpu)`()`<br />
)

</div>

Construct from config with storage-free slots: registers a shape-only `weight` `[normalized_size]` (and, with `bias`, a `bias` `[normalized_size]`) parameter on [`dtype`](../dtype/index.md)/[`device`](../device-namespace/index.md); no storage is allocated. [`named_parameters()`](./Module.md#named_parameters) reports the slots, so a strict [`load_state_dict`](./Module.md#load_state_dict) both expects and satisfies them; the first [`forward`](./RMSNorm.md#forward) builds. Raises [`ClikaRT::Error`](../Error.md) on a bad normalized size.

<div class="api-declared">

Declared in `ClikaRT/nn/rms_norm.h`, line 63

</div>

## Member functions

### set\_weights() {#set_weights}

<div class="api-signature">

`void` **`set_weights`**([`Tensor`](../Tensor.md) `weight`, [`OptionalTensor`](../../index.md#OptionalTensor) `bias` `=` `{}`)

</div>

Bind the declared slots positionally: `weight` (gain, `[normalized_size]`) and, when the module was made with one, `bias`. The declared placement wins; shapes must match the declaration. Re-binding drops the built primitive; the next [`forward`](./RMSNorm.md#forward) rebuilds. Raises [`ClikaRT::Error`](../Error.md) on a geometry mismatch or a bias without a declared bias slot.

<div class="api-declared">

Declared in `ClikaRT/nn/rms_norm.h`, line 75

</div>

### \~RMSNorm() {#RMSNorm}

<div class="api-signature">

**`~RMSNorm`**() `override`

</div>

<div class="api-declared">

Declared in `ClikaRT/nn/rms_norm.h`, line 79

</div>

### to\_impl(StreamOrDevice) {#to_impl}

<div class="api-signature">

`virtual` [`Result`](../Result.md)`<void>` **`to_impl`**([`StreamOrDevice`](../StreamOrDevice.md) `where`) `override`

</div>

Move to a placement ([`Device`](../Device.md) / [`Stream`](../Stream.md)) or cast to a dtype. Rebuilds the bound primitive on the target after moving the registry slots.

<div class="api-declared">

Declared in `ClikaRT/nn/rms_norm.h`, line 83

</div>

### to\_impl(DataType) {#to_impl-2}

<div class="api-signature">

`virtual` [`Result`](../Result.md)`<void>` **`to_impl`**([`DataType`](../../index.md#DataType) `dtype`) `override`

</div>

<div class="api-declared">

Declared in `ClikaRT/nn/rms_norm.h`, line 84

</div>

### forward() {#forward}

<div class="api-signature">

[`Tensor`](../Tensor.md) **`forward`**([`Tensor`](../Tensor.md) `x`) `const`

</div>

`out = ACT( x · rsqrt(mean(x²) + eps) · weight + bias )`; `x` `[*, N]` -\> `[*, N]`, output dtype = `x`'s. The first call builds (once, thread-safe); later calls reuse the bound primitive. Raises [`ClikaRT::Error`](../Error.md) on a shape/dtype mismatch or a still-fake slot.

<div class="api-declared">

Declared in `ClikaRT/nn/rms_norm.h`, line 91

</div>

## Protected member functions

### initialize\_impl() {#initialize_impl}

<div class="api-signature">

`virtual` [`Result`](../Result.md)`<void>` **`initialize_impl`**() `override`

</div>

The pack hook ([`Module::initialize_impl`](./Module.md#initialize_impl)): materializes the bound `weight` / `bias` slots contiguous (layout-only, dtype-preserving; idempotent, thread-safe). Every declared slot must hold a real (loaded) tensor; a still-fake slot is a clean error.

<div class="api-declared">

Declared in `ClikaRT/nn/rms_norm.h`, line 103

</div>