Skip to main content

RMSNorm

y = x * rsqrt(mean(x^2) + eps) * weight (+ bias) over the trailing normalized_size dim.

__init__​

__init__(self, normalized_size: 'int', bias: 'bool' = False, *, eps: 'float' = 1e-06, activation: 'Activation | None' = None, dtype: 'DataType' = DataType.Float32, device: 'Device | None' = None) -> 'None'

Declare weight (gain) and, with bias=True, bias slots of [normalized_size].

extra_repr​

extra_repr(self) -> 'str'

One line of per-class detail for :meth:__repr__ (a layer prints its geometry here).

forward​

forward(self, x: 'Tensor') -> 'Tensor'

Subclasses define the computation here; call the module itself (m(x)), not forward directly.

set_weights​

set_weights(self, weight: 'Tensor', bias: 'Tensor | None' = None) -> 'None'

Bind the declared slots directly; raises on a shape mismatch.