Skip to main content

ClikaRT::nn::MoeOptions

struct

Header: ClikaRT/nn/moe.h

MoE configuration beyond the required top-k. Every field is optional; the defaults are the runtime's (SoftmaxTopK routing, renormalized top-k weights, SwiGLU with the Interleaved gate‖up layout, erf gelu).

Data members​

routing_mode​

std::optional<ops::MoeRouting> routing_mode

default SoftmaxTopK

Declared in ClikaRT/nn/moe.h, line 51

renormalize​

std::optional<bool> renormalize

÷ sum of top-k (default true)

Declared in ClikaRT/nn/moe.h, line 52

activation​

std::optional<ops::Activation> activation

gated SwiGlu/GeGlu/ReGlu or plain (default SwiGlu)

Declared in ClikaRT/nn/moe.h, line 53

swiglu_fusion​

std::optional<ops::SwigluFusion> swiglu_fusion

gate‖up layout (default Interleaved)

Declared in ClikaRT/nn/moe.h, line 54

gelu_mode​

std::optional<ops::GeluMode> gelu_mode

a gated GeGlu's gelu form (default erf)

Declared in ClikaRT/nn/moe.h, line 55

n_group​

std::optional<std::int64_t> n_group

group-limited routing (DeepSeek)

Declared in ClikaRT/nn/moe.h, line 56

topk_group​

std::optional<std::int64_t> topk_group

Experts drawn per group under group-limited routing.

Declared in ClikaRT/nn/moe.h, line 58

routed_scaling_factor​

std::optional<double> routed_scaling_factor

× the combine weights

Declared in ClikaRT/nn/moe.h, line 59

sparse_mixer_eps​

std::optional<double> sparse_mixer_eps

Sparse-mixer routing epsilon (the sparse-mixer variant).

Declared in ClikaRT/nn/moe.h, line 61

apply_router_weight_on_input​

std::optional<bool> apply_router_weight_on_input

Scale the expert INPUT by the router weight (instead of the output combine).

Declared in ClikaRT/nn/moe.h, line 64

swiglu_alpha​

std::optional<double> swiglu_alpha

clamped-SwiGLU form knobs

Declared in ClikaRT/nn/moe.h, line 65

swiglu_beta​

std::optional<double> swiglu_beta

Clamped-SwiGLU beta (see ops::swiglu).

Declared in ClikaRT/nn/moe.h, line 67

swiglu_limit​

std::optional<double> swiglu_limit

Clamped-SwiGLU symmetric limit; absent = no clamping.

Declared in ClikaRT/nn/moe.h, line 69

expert_output_scale​

std::optional<std::vector<double>> expert_output_scale

Per-expert combine scale ([E], one value per expert): multiplies each SELECTED expert's routing weight after renormalization. For a checkpoint that ships a per-expert output scale beside QUANTIZED expert stacks this is the faithful application point; the payloads stay untouched (a dense checkpoint may instead fold the scale into its down weights; the two are value-identical).

Declared in ClikaRT/nn/moe.h, line 76