ClikaRT::ops::gelu

function

gelu()

Tensor gelu(Tensor x, GeluMode approximate = GeluMode::None)

Gaussian error linear unit.

$$\mathrm{gelu}(x) = x\,\Phi(x) = \tfrac{x}{2}\Bigl(1 + \mathrm{erf}\bigl(x/\sqrt{2}\bigr)\Bigr)$$

approximate = GeluMode::Tanh selects the cheaper tanh form used by many transformer checkpoints; GeluMode::None is the exact erf form.

Parameters

• x: the input tensor.
• approximate: the GELU variant. Default GeluMode::None (exact).

Throws

• ClikaRT::Error: when the input's dtype is not served for this operation (the machine-readable reason rides code_name()).

Returns: a new tensor, same shape and dtype as x.

auto y = ClikaRT::ops::gelu(x, ClikaRT::ops::GeluMode::Tanh);

Declared in ClikaRT/compute/ops.h, line 768