---
title: "ClikaRT::nn::LayerNorm"
sidebar_label: "LayerNorm"
description: "Layer normalization module. \\[ y = \\frac{x - \\mathrm{E}[x]}{\\sqrt{\\mathrm{Var}[x] + \\varepsilon}} \\cdot \\gamma + \\beta \\] Statistics are computed over the trailing normalized_shape dims, the module fo…"
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--class">class</span>

**Header:** `ClikaRT/nn/layer_norm.h`  
**Inherits:** [`ClikaRT::nn::Module`](./Module.md)

A bound LayerNorm, the public face of the runtime's `ops::layer_norm`, exposed as an `nn::Module` leaf. Unlike the stateless `ops::layer_norm` free function, a `LayerNorm` binds its `weight` (scale) / `bias` ONCE and materializes them contiguous at the first forward, so every later `forward` reuses the bound params; build one per norm site at load, reuse it every step.

The scale / bias stay in their OWN dtype (a fp32 scale against a bf16 activation is read at full precision, never cast at rest); the build does layout work only. The output dtype follows the input `x`.

The lifecycle (uniform across every weight-bearing `nn` module): `make(normalized_size, ...)` registers storage-free `weight` / `bias` slots under their canonical names; `set_weights(...)` (positional) or `load_state_dict(...)` (a checkpoint, by dotted name) binds them; the FIRST `forward` materializes contiguous (once, thread-safe). `LoadOptions::pack_on_load` pays the build at load time.

Held via `std::shared_ptr` (an `nn::Module` leaf); copy/move are pinned by the base.

Layer normalization module.

$$
y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \varepsilon}}
\cdot \gamma + \beta
$$

Statistics are computed over the trailing `normalized_shape` dims, the module form of [`ops::layer_norm`](../ops/layer_norm.md#layer_norm), holding gamma/beta as its weights. The module owns its weights: construct with `make(...)`, or declare shapes and bind a checkpoint via [`load_state_dict`](./Module.md#load_state_dict). The first forward materializes the bound slots contiguous (or [`LoadOptions::pack_on_load`](./LoadOptions.md#pack_on_load) does it at the end of the load); the module holds each weight once and [`named_parameters()`](./Module.md#named_parameters) / [`state_dict()`](./Module.md#state_dict) stay complete; `to(dtype)` casts the slots.

```cpp
auto ln = ClikaRT::nn::LayerNorm::make(gamma, beta);
auto y  = ln->forward(x);
```

## Static member functions

### make() {#make}

<div class="api-signature">

`static` `std::shared_ptr<LayerNorm>` **`make`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::int64_t` `normalized_size`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`bool` `bias` `=` `true`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`double` `eps` `=` `1e-5`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<`[`ops::Activation`](../ops/index.md#Activation)`>` `activation` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`DataType`](../../index.md#DataType) `dtype` `=` [`DataType::Float32`](../../index.md#DataType),<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Device`](../Device.md) `device` `=` [`Device::cpu`](../Device.md#cpu)`()`<br />
)

</div>

Construct from config with storage-free slots: registers a shape-only `weight` `[normalized_size]` (and, with `bias`, a `bias` `[normalized_size]`) parameter on [`dtype`](../dtype/index.md)/[`device`](../device-namespace/index.md); no storage is allocated. [`named_parameters()`](./Module.md#named_parameters) reports the slots, so a strict [`load_state_dict`](./Module.md#load_state_dict) both expects and satisfies them; the first [`forward`](./LayerNorm.md#forward) builds. Raises [`ClikaRT::Error`](../Error.md) on a bad normalized size.

<div class="api-declared">

Declared in `ClikaRT/nn/layer_norm.h`, line 64

</div>

## Member functions

### set\_weights() {#set_weights}

<div class="api-signature">

`void` **`set_weights`**([`Tensor`](../Tensor.md) `weight`, [`OptionalTensor`](../../index.md#OptionalTensor) `bias` `=` `{}`)

</div>

Bind the declared slots positionally: `weight` (scale, `[normalized_size]`) and, when the module was made with one, `bias`. The declared placement wins; shapes must match the declaration. Re-binding drops the built primitive; the next [`forward`](./LayerNorm.md#forward) rebuilds. Raises [`ClikaRT::Error`](../Error.md) on a geometry mismatch or a bias without a declared bias slot.

<div class="api-declared">

Declared in `ClikaRT/nn/layer_norm.h`, line 76

</div>

### \~LayerNorm() {#LayerNorm}

<div class="api-signature">

**`~LayerNorm`**() `override`

</div>

<div class="api-declared">

Declared in `ClikaRT/nn/layer_norm.h`, line 80

</div>

### to\_impl(StreamOrDevice) {#to_impl}

<div class="api-signature">

`virtual` [`Result`](../Result.md)`<void>` **`to_impl`**([`StreamOrDevice`](../StreamOrDevice.md) `where`) `override`

</div>

Move to a placement ([`Device`](../Device.md) / [`Stream`](../Stream.md)) or cast to a dtype. Rebuilds the bound primitive on the target after moving the registry slots.

<div class="api-declared">

Declared in `ClikaRT/nn/layer_norm.h`, line 84

</div>

### to\_impl(DataType) {#to_impl-2}

<div class="api-signature">

`virtual` [`Result`](../Result.md)`<void>` **`to_impl`**([`DataType`](../../index.md#DataType) `dtype`) `override`

</div>

<div class="api-declared">

Declared in `ClikaRT/nn/layer_norm.h`, line 85

</div>

### forward() {#forward}

<div class="api-signature">

[`Tensor`](../Tensor.md) **`forward`**([`Tensor`](../Tensor.md) `x`) `const`

</div>

`out = ACT( (x − mean) · rsqrt(var + eps) · weight + bias )`; `x` `[*, N]` -\> `[*, N]`, output dtype = `x`'s. The first call builds (once, thread-safe); later calls reuse the bound primitive. Raises [`ClikaRT::Error`](../Error.md) on a shape/dtype mismatch or a still-fake slot.

<div class="api-declared">

Declared in `ClikaRT/nn/layer_norm.h`, line 92

</div>

## Protected member functions

### initialize\_impl() {#initialize_impl}

<div class="api-signature">

`virtual` [`Result`](../Result.md)`<void>` **`initialize_impl`**() `override`

</div>

The pack hook ([`Module::initialize_impl`](./Module.md#initialize_impl)): materializes the bound `weight` / `bias` slots contiguous (layout-only, dtype-preserving; idempotent, thread-safe). Every declared slot must hold a real (loaded) tensor; a still-fake slot is a clean error.

<div class="api-declared">

Declared in `ClikaRT/nn/layer_norm.h`, line 104

</div>