ClikaRT::nn::QMoEWoQ

class

Header: ClikaRT/nn/qmoe_woq.h
Inherits: ClikaRT::nn::Module

QMoEWoQ: a bound Mixture-of-Experts block over QUANTIZED expert weights (the weight-only-quantization serving path), exposed as an nn::Module leaf: the activation stays floating point and the stacked expert weights stay quantized at rest for the module's lifetime; each expert weight decodes one block at a time inside the GEMM dot, never a dense copy. This is the module that serves a block-quantized MoE checkpoint (e.g. MXFP4 gpt-oss) at its on-disk footprint; the dense-expert sibling is MoE (exactly the Linear vs QLinearWoQ split).

The lifecycle (uniform across every weight-bearing nn module):

make(experts, hidden, intermediate, top_k, options, ...), the ONE constructor: declares the storage-free expert / bias slots.
set_weights(...) or load_state_dict(...) binds them; a quantized payload lands on its slot as-is (the scheme travels with it, no cast).
forward(x, router_logits) packs on first call (once, thread-safe), then serves; LoadOptions::pack_on_load pays the pack at load time.

The caller computes the per-token router logits itself and hands them to forward beside the activations; routing (MoeRouting), the gated activation, and the gate‖up layout (ops::fused::SwigluFusion) are configuration (MoeOptions, shared with MoE).

Weight-only-quantized mixture-of-experts module: the stacked [E, ...] expert weights rest quantized and decode inside the per-expert kernels, MoE's weight-only sibling (same routing surface and options).

The module owns its weights: construct with make(...), or declare shapes and bind a checkpoint via load_state_dict. The first forward packs the weight (or LoadOptions::pack_on_load packs it at the end of the load); the pack is the one resident copy, and named_parameters() / state_dict() read the weight back from it; to(dtype) restores, casts, and re-packs on the next forward.

Static member functions

make()

static std::shared_ptr<QMoEWoQ> make(
    std::int64_t num_experts,
    std::int64_t hidden_size,
    std::int64_t intermediate_size,
    std::int64_t top_k,
    bool gate_up_bias = false,
    bool down_bias = false,
    DataType dtype = DataType::Float32,
    Device device = Device::cpu(),
    MoeOptions options = {}
)

Defaults: gate_up_bias / down_bias = false (no bias slots), dtype = Float32, device = CPU, options = MoeOptions{} (each field's own default; see the struct). The routing variants ride the optional trailing options; the four counts are the whole required surface.

Throws

ClikaRT::Error: as stated above.

Declared in ClikaRT/nn/qmoe_woq.h, line 73

Member functions

~QMoEWoQ()

~QMoEWoQ() override

Releases the module's packed expert weights.

Declared in ClikaRT/nn/qmoe_woq.h, line 85

set_weights()

void set_weights(
    QTensor gate_up_experts,
    QTensor down_experts,
    OptionalTensor gate_up_bias = {},
    OptionalTensor down_bias = {},
    QTensor gate_experts = {},
    OptionalTensor gate_bias = {},
    OptionalTensor e_score_correction_bias = {}
)

Bind the declared slots positionally: the QUANTIZED gate_up_experts / down_experts stacks (required; e.g. from io::load_gguf, make_quantized, make_quantized_mxfp4), the declared floating-point per-expert biases, the Unfused quantized gate_experts (+ gate_bias), and the aux-free selection bias [E] (bindable even when not declared). Payload logical geometry must match the declaration. Re-binding after a pack drops the pack; the next forward re-packs. Raises ClikaRT::Error on a geometry mismatch or a bias without its declared slot.

Declared in ClikaRT/nn/qmoe_woq.h, line 100

to_impl(StreamOrDevice)

virtual Result<void> to_impl(StreamOrDevice where) override

Move to a placement (the packed experts rebuild on the target). A dtype to is Unsupported: the experts stay quantized at rest; dequantize explicitly if a dense copy is wanted. can_cast_to_impl reports the same refusal without moving a byte (the tree-wide check behind Module::to(dtype)).

Declared in ClikaRT/nn/qmoe_woq.h, line 117

to_impl(DataType)

virtual Result<void> to_impl(DataType dtype) override

Declared in ClikaRT/nn/qmoe_woq.h, line 118

can_cast_to_impl()

virtual Result<void> can_cast_to_impl(DataType dtype) const override

Whether THIS module's own slots can be cast to dtype (children are asked by the tree walk, never here). A failure is the refusal to(dtype) would raise; Module::to(dtype) asks every module of the tree BEFORE it casts any, so a refused cast leaves the tree untouched and names the leaf. Default: a dense module casts. Override in a leaf that pins its dtype.

Declared in ClikaRT/nn/qmoe_woq.h, line 119

forward()

Tensor forward(
    Tensor x,
    Tensor router_logits,
    OptionalTensor shared_output = {}
) const

One fused MoE step: x [T, H] and the caller-computed router_logits [T, E] → [T, H] (dtype mirrors x). The first call packs (once, thread-safe); every later call reuses the pack. Raises ClikaRT::Error on a shape/dtype mismatch or a still-fake slot. An optional shared_output (the shared-expert branch's [T, H] at x's dtype, EXACT shape) folds into the routed combine (the fused routed+shared epilogue; no separate add pass).

Declared in ClikaRT/nn/qmoe_woq.h, line 130

Protected member functions

initialize_impl()

virtual Result<void> initialize_impl() override

The pack hook (Module::initialize_impl): packs now (idempotent, thread-safe). Every declared slot must hold a real (loaded) tensor; a still-fake slot is a clean error. After the pack the expert weight slots read back from the pack, the single resident copy; biases stay bound.

Declared in ClikaRT/nn/qmoe_woq.h, line 147

Static member functions​

make()​

Member functions​

~QMoEWoQ()​

set_weights()​

to_impl(StreamOrDevice)​

to_impl(DataType)​

can_cast_to_impl()​

forward()​

Protected member functions​

initialize_impl()​