Skip to main content

ClikaRT::ops::fast_gelu

function

fast_gelu()​

Tensor fast_gelu(Tensor x)

FastGELU: the tanh GELU approximation as a standalone op.

fast_gelu(x)=x2(1+tanh⁡(2/π (x+0.044715 x3)))\mathrm{fast\_gelu}(x) = \tfrac{x}{2}\Bigl(1 + \tanh\bigl(\sqrt{2/\pi}\,(x + 0.044715\,x^3)\bigr)\Bigr)

Throws

  • ClikaRT::Error: when the input's dtype is not served for this operation (the machine-readable reason rides code_name()).

Returns: a new tensor, same shape and dtype as x.

auto y = ClikaRT::ops::fast_gelu(x);

Declared in ClikaRT/compute/ops.h, line 793