---
title: "ClikaRT::ops::moe"
sidebar_label: "moe"
description: "Fused Mixture-of-Experts layer: route, run the top-k experts, combine in one call, with no per-expert dispatch from the caller."
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--function">function</span>

## moe() {#moe}

<div class="api-signature">

[`Tensor`](../Tensor.md) **`moe`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `input`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `router_logits`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `fc1_experts`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `fc2_experts`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`int64_t` `top_k`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `fc1_bias` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `fc2_bias` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `fc3_experts` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `fc3_bias` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `e_score_correction_bias` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `router_weights` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<`[`MoeRouting`](./index.md#MoeRouting)`>` `routing_mode` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<bool>` `renormalize` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<int64_t>` `n_group` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<int64_t>` `topk_group` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `routed_scaling_factor` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `sparse_mixer_eps` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<bool>` `apply_router_weight_on_input` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<`[`Activation`](./index.md#Activation)`>` `activation` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<`[`fused::SwigluFusion`](./fused/index.md#SwigluFusion)`>` `swiglu_fusion` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `swiglu_alpha` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `swiglu_beta` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<double>` `swiglu_limit` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::optional<`[`GeluMode`](./index.md#GeluMode)`>` `gelu_mode` `=` `std::nullopt`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `shared_output` `=` `{}`<br />
)

</div>

Fused Mixture-of-Experts layer: route, run the top-k experts, combine in one call, with no per-expert dispatch from the caller.

Per token, `router_logits [T, E]` select `top_k` experts under `routing_mode`; each selected expert applies its own MLP (`fc1 [E, F·I, H]` → activation → `fc2 [E, H, I]`, with `F` = 2 for a gated activation, else 1, and an optional multiplicative `fc3 [E, I, H]` branch); the expert outputs combine under the routing weights.

`input` is `[T, H]`; the result is `[T, H]` at `input`'s dtype. Per-expert biases ride `fc1_bias` / `fc2_bias` / `fc3_bias`; `e_score_correction_bias` and `n_group` / `topk_group` / `routed_scaling_factor` serve the group-limited routing families; `router_weights` feeds [`MoeRouting::PreComputed`](./index.md#MoeRouting) (caller-supplied combine weights); `sparse_mixer_eps` tunes [`MoeRouting::SparseMixer`](./index.md#MoeRouting). The gated-activation scalars (`swiglu_*`, `gelu_mode`) carry their [`swiglu`](./swiglu.md#swiglu) / [`geglu`](./geglu.md#geglu) meanings; `shared_output [T, H]` folds a shared-expert branch into the final combine. A config argument left `std::nullopt` takes the runtime default (SoftmaxTopK routing, renormalized top-k weights).

**Returns:** the combined expert output `[T, H]`.

**Throws**

- `ClikaRT::Error`: (`INVALID_ARGUMENT`) when the expert-weight geometry disagrees with `input` / `router_logits`, or a routing config value is outside its family's range (`code_name()` carries the reason).

<div class="api-declared">

Declared in `ClikaRT/compute/ops.h`, line 2916

</div>