Skip to main content

ClikaRT::ops::layer_norm

function

layer_norm()​

Tensor layer_norm(
    Tensor input,
    ClikaRT::Span<const std::int64_t> normalized_shape,
    OptionalTensor weight = {},
    OptionalTensor bias = {},
    std::optional<double> eps = std::nullopt,
    std::optional<Activation> activation = std::nullopt
)

Layer normalization over the trailing normalized_shape dims of input.

out=x−E[x]Var[x]+ε⋅γ+β\mathrm{out} = \frac{x - \mathrm{E}[x]} {\sqrt{\mathrm{Var}[x] + \varepsilon}} \cdot \gamma + \beta

Statistics are computed per position over the trailing normalized_shape dims (channels-last layout, [N, .., C]). The output keeps input's shape AND dtype. An optional activation is fused onto the post-affine value (gated kinds are not accepted here).

Parameters

  • input: the input; its trailing dims must equal normalized_shape.
  • normalized_shape: the trailing dims that form one group, e.g. {hidden}.
  • weight: optional per-element scale (gamma), shaped normalized_shape.
  • bias: optional per-element shift (beta), shaped normalized_shape.
  • eps: stability floor on the variance; std::nullopt selects 1e-5.
  • activation: optional fused activation; absent = none.

Returns: a new tensor, same shape and dtype as input.

Throws

  • ClikaRT::Error: (INVALID_ARGUMENT) when the trailing dims do not match normalized_shape, or a provided weight/bias is shaped differently.
auto y = ClikaRT::ops::layer_norm(x, {hidden}, gamma, beta);

Declared in ClikaRT/compute/ops.h, line 3685