Skip to main content

clika_runtime.nn.init functions

calculate_gain​

calculate_gain(nonlinearity: 'str', param: 'float | None' = None) -> 'float'

calculate_gain(nonlinearity, param=None) -> float

The recommended gain for a nonlinearity: 1 for the linear family and sigmoid, 5/3 for tanh, sqrt(2) for relu, sqrt(2 / (1 + slope^2)) for leaky_relu (param is the negative slope, 0.01 by default), and 3/4 for selu.

Args: nonlinearity: one of linear, conv, conv1d, conv2d, conv3d, conv_transpose, conv_transpose1d, conv_transpose2d, conv_transpose3d, sigmoid, tanh, relu, leaky_relu, selu. param: the negative slope of leaky_relu.

constant_​

constant_(tensor: 'Tensor', val: 'float') -> 'Tensor'

constant_(tensor, val) -> Tensor

Fill tensor with val; returns the tensor.

dirac_​

dirac_(tensor: 'Tensor', groups: 'int' = 1) -> 'Tensor'

dirac_(tensor, groups=1) -> Tensor

Fill a 3-, 4-, or 5-d convolution weight [out, *kernel, in / groups] with the Dirac delta: each output channel passes its matching input channel through the kernel center unchanged, for as many channels as the smaller of the two counts (per group with groups > 1); returns the tensor.

eye_​

eye_(tensor: 'Tensor') -> 'Tensor'

eye_(tensor) -> Tensor

Fill a 2-d tensor with the identity (ones on the main diagonal, zeros elsewhere; rectangular shapes keep as many ones as the shorter side); returns the tensor.

kaiming_normal_​

kaiming_normal_(tensor: 'Tensor', a: 'float' = 0, mode: 'str' = 'fan_in', nonlinearity: 'str' = 'leaky_relu', generator: 'None' = None) -> 'Tensor'

kaiming_normal_(tensor, a=0, mode="fan_in", nonlinearity="leaky_relu") -> Tensor

Fill tensor from N(0, std^2) with std = gain / sqrt(fan) (He initialization), where gain is :func:calculate_gain of nonlinearity with a as its slope and fan is the fan chosen by mode; returns the tensor. A zero-element tensor is left alone with a warning.

kaiming_uniform_​

kaiming_uniform_(tensor: 'Tensor', a: 'float' = 0, mode: 'str' = 'fan_in', nonlinearity: 'str' = 'leaky_relu', generator: 'None' = None) -> 'Tensor'

kaiming_uniform_(tensor, a=0, mode="fan_in", nonlinearity="leaky_relu") -> Tensor

Fill tensor uniformly from [-bound, bound] with bound = gain * sqrt(3 / fan) (He initialization), where gain is :func:calculate_gain of nonlinearity with a as its slope and fan is the fan chosen by mode; returns the tensor. A zero-element tensor is left alone with a warning.

normal_​

normal_(tensor: 'Tensor', mean: 'float' = 0.0, std: 'float' = 1.0, generator: 'None' = None) -> 'Tensor'

normal_(tensor, mean=0.0, std=1.0) -> Tensor

Fill tensor with values drawn from the normal distribution N(mean, std^2); returns the tensor.

ones_​

ones_(tensor: 'Tensor') -> 'Tensor'

ones_(tensor) -> Tensor

Fill tensor with ones; returns the tensor.

trunc_normal_​

trunc_normal_(tensor: 'Tensor', mean: 'float' = 0.0, std: 'float' = 1.0, a: 'float' = -2.0, b: 'float' = 2.0, generator: 'None' = None) -> 'Tensor'

trunc_normal_(tensor, mean=0.0, std=1.0, a=-2.0, b=2.0) -> Tensor

Fill tensor with values drawn from N(mean, std^2) truncated to [a, b] (values outside are redrawn, by inverse-CDF sampling); returns the tensor.

uniform_​

uniform_(tensor: 'Tensor', a: 'float' = 0.0, b: 'float' = 1.0, generator: 'None' = None) -> 'Tensor'

uniform_(tensor, a=0.0, b=1.0) -> Tensor

Fill tensor with values drawn uniformly from [a, b); returns the tensor.

xavier_normal_​

xavier_normal_(tensor: 'Tensor', gain: 'float' = 1.0, generator: 'None' = None) -> 'Tensor'

xavier_normal_(tensor, gain=1.0) -> Tensor

Fill tensor from N(0, std^2) with std = gain * sqrt(2 / (fan_in + fan_out)) (Glorot initialization); returns the tensor.

xavier_uniform_​

xavier_uniform_(tensor: 'Tensor', gain: 'float' = 1.0, generator: 'None' = None) -> 'Tensor'

xavier_uniform_(tensor, gain=1.0) -> Tensor

Fill tensor uniformly from [-a, a] with a = gain * sqrt(6 / (fan_in + fan_out)) (Glorot initialization); returns the tensor.

zeros_​

zeros_(tensor: 'Tensor') -> 'Tensor'

zeros_(tensor) -> Tensor

Fill tensor with zeros; returns the tensor.