Skip to main content

ClikaRT::ops::rms_norm

function

rms_norm()​

Tensor rms_norm(
    Tensor x,
    ClikaRT::Span<const std::int64_t> normalized_shape,
    OptionalTensor weight = {},
    OptionalTensor bias = {},
    std::optional<double> eps = std::nullopt,
    std::optional<Activation> activation = std::nullopt
)

Root-mean-square normalization over the trailing normalized_shape dims (no mean subtraction).

out=xE[x2]+ε⋅γ+β\mathrm{out} = \frac{x}{\sqrt{\mathrm{E}[x^2] + \varepsilon}} \cdot \gamma + \beta

The transformer-style norm: statistics are the mean SQUARE only, per position over the trailing dims. Optional fused activation applies to the post-affine value (gated kinds are not accepted).

Parameters

  • x: the input tensor; float dtype.
  • normalized_shape: the trailing dims that form one group.
  • weight: optional gain (gamma), shaped normalized_shape.
  • bias: optional post-norm shift, shaped normalized_shape.
  • eps: stability floor; std::nullopt selects 1e-6.
  • activation: optional fused activation; absent = none.

Returns: a new tensor, same shape and dtype as x.

Throws

  • ClikaRT::Error: (INVALID_ARGUMENT) when the trailing dims do not match normalized_shape or an affine operand is mis-shaped.

Declared in ClikaRT/compute/ops.h, line 3728