Skip to main content

ClikaRT::ops::glu

function

glu()​

Tensor glu(Tensor x, int64_t dim = -1)

Gated linear unit: splits x in half along dim and gates the first half with the sigmoid of the second.

glu(x)=a⋅σ(b),x=[ a ∥ b ] along dim\mathrm{glu}(x) = a \cdot \sigma(b), \qquad x = [\,a \,\|\, b\,] \text{ along } \mathit{dim}

The size of dim must be even; the output halves it.

Parameters

  • x: the input tensor.
  • dim: the dimension to split. Negative counts from the end. Default -1 (the last).

Throws

  • ClikaRT::Error: when dim is out of range or its size is odd, or when the input's dtype is not served (code_name() carries the reason).

Returns: the input shape with dim halved; dtype of x.

auto h = ClikaRT::ops::glu(proj);      // [.., 2d] -> [.., d]

Declared in ClikaRT/compute/ops.h, line 809