---
title: "ClikaRT::nn::QMoEWoQ"
sidebar_label: "QMoEWoQ"
description: "Weight-only-quantized mixture-of-experts module: the stacked [E, ...] expert weights rest quantized and decode inside the per-expert kernels, MoE's weight-only sibling (same routing surface and option…"
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--class">class</span>

**Header:** `ClikaRT/nn/qmoe_woq.h`  
**Inherits:** [`ClikaRT::nn::Module`](./Module.md)

`QMoEWoQ`: a bound Mixture-of-Experts block over QUANTIZED expert weights (the weight-only-quantization serving path), exposed as an `nn::Module` leaf: the stacked expert weights stay quantized at rest for the module's lifetime, and each expert weight decodes one block at a time inside the GEMM dot, never a dense copy. The activation stays floating point, except where a packed block weight runs on an int8 block tier, which rounds it to per-block int8 before the dot (see `fused::QComputeMode`). This is the module that serves a block-quantized MoE checkpoint (e.g. MXFP4 gpt-oss) at its on-disk footprint; the dense-expert sibling is `MoE` (exactly the `Linear` vs `QLinearWoQ` split).

The lifecycle (uniform across every weight-bearing `nn` module):

1. `make(experts, hidden, intermediate, top_k, options, ...)`, the ONE constructor: declares the storage-free expert / bias slots.
2. `set_weights(...)` or `load_state_dict(...)` binds them; a quantized payload lands on its slot as-is (the scheme travels with it, no cast).
3. `forward(x, router_logits)` packs on first call (once, thread-safe), then serves; `LoadOptions::pack_on_load` pays the pack at load time.

The caller computes the per-token router logits itself and hands them to `forward` beside the activations; routing (`MoeRouting`), the gated activation, and the gate‖up layout (`ops::fused::SwigluFusion`) are configuration (`MoeOptions`, shared with `MoE`).

Weight-only-quantized mixture-of-experts module: the stacked `[E, ...]` expert weights rest quantized and decode inside the per-expert kernels, [`MoE`](./MoE.md)'s weight-only sibling (same routing surface and options).

The module owns its weights: construct with `make(...)`, or declare shapes and bind a checkpoint via [`load_state_dict`](./Module.md#load_state_dict). The first forward packs the weight (or [`LoadOptions::pack_on_load`](./LoadOptions.md#pack_on_load) packs it at the end of the load); the pack is the one resident copy, and [`named_parameters()`](./Module.md#named_parameters) / [`state_dict()`](./Module.md#state_dict) read the weight back from it; `to(dtype)` restores, casts, and re-packs on the next forward.

## Static member functions

### make() {#make}

<div class="api-signature">

`static` `std::shared_ptr<QMoEWoQ>` **`make`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::int64_t` `num_experts`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::int64_t` `hidden_size`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::int64_t` `intermediate_size`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`std::int64_t` `top_k`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`bool` `gate_up_bias` `=` `false`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;`bool` `down_bias` `=` `false`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`DataType`](../../index.md#DataType) `dtype` `=` [`DataType::Float32`](../../index.md#DataType),<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Device`](../Device.md) `device` `=` [`Device::cpu`](../Device.md#cpu)`()`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`MoeOptions`](./MoeOptions.md) `options` `=` `{}`<br />
)

</div>

Defaults: `gate_up_bias` / `down_bias` = `false` (no bias slots), [`dtype`](../dtype/index.md) = `Float32`, [`device`](../device-namespace/index.md) = CPU, `options` = [`MoeOptions`](./MoeOptions.md)`{}` (each field's own default; see the struct). The routing variants ride the optional trailing `options`; the four counts are the whole required surface.

**Throws**

- `ClikaRT::Error`: as stated above.

<div class="api-declared">

Declared in `ClikaRT/nn/qmoe_woq.h`, line 75

</div>

## Member functions

### \~QMoEWoQ() {#QMoEWoQ}

<div class="api-signature">

**`~QMoEWoQ`**() `override`

</div>

Releases the module's packed expert weights.

<div class="api-declared">

Declared in `ClikaRT/nn/qmoe_woq.h`, line 87

</div>

### set\_weights() {#set_weights}

<div class="api-signature">

`void` **`set_weights`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`QTensor`](../QTensor.md) `gate_up_experts`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`QTensor`](../QTensor.md) `down_experts`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `gate_up_bias` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `down_bias` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`QTensor`](../QTensor.md) `gate_experts` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `gate_bias` `=` `{}`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `e_score_correction_bias` `=` `{}`<br />
)

</div>

Bind the declared slots positionally: the QUANTIZED `gate_up_experts` / `down_experts` stacks (required; e.g. from [`io::load_gguf`](../io/index.md#load_gguf), [`make_quantized`](../../index.md#make_quantized), [`make_quantized_mxfp4`](../../index.md#make_quantized_mxfp4)), the declared floating-point per-expert biases, the Unfused quantized `gate_experts` (+ `gate_bias`), and the aux-free selection bias `[E]` (bindable even when not declared). Payload logical geometry must match the declaration. Re-binding after a pack drops the pack; the next [`forward`](./QMoEWoQ.md#forward) re-packs. Raises [`ClikaRT::Error`](../Error.md) on a geometry mismatch or a bias without its declared slot.

<div class="api-declared">

Declared in `ClikaRT/nn/qmoe_woq.h`, line 102

</div>

### to\_impl(StreamOrDevice) {#to_impl}

<div class="api-signature">

`virtual` [`Result`](../Result.md)`<void>` **`to_impl`**([`StreamOrDevice`](../StreamOrDevice.md) `where`) `override`

</div>

Move to a placement (the packed experts rebuild on the target). A dtype [`to`](./Module.md#to) is `Unsupported`: the experts stay quantized at rest; dequantize explicitly if a dense copy is wanted. [`can_cast_to_impl`](./QMoEWoQ.md#can_cast_to_impl) reports the same refusal without moving a byte (the tree-wide check behind `Module::to(dtype)`).

<div class="api-declared">

Declared in `ClikaRT/nn/qmoe_woq.h`, line 119

</div>

### to\_impl(DataType) {#to_impl-2}

<div class="api-signature">

`virtual` [`Result`](../Result.md)`<void>` **`to_impl`**([`DataType`](../../index.md#DataType) `dtype`) `override`

</div>

<div class="api-declared">

Declared in `ClikaRT/nn/qmoe_woq.h`, line 120

</div>

### can\_cast\_to\_impl() {#can_cast_to_impl}

<div class="api-signature">

`virtual` [`Result`](../Result.md)`<void>` **`can_cast_to_impl`**([`DataType`](../../index.md#DataType) `dtype`) `const override`

</div>

Whether THIS module's own slots can be cast to [`dtype`](../dtype/index.md) (children are asked by the tree walk, never here). A failure is the refusal `to(dtype)` would raise; `Module::to(dtype)` asks every module of the tree BEFORE it casts any, so a refused cast leaves the tree untouched and names the leaf. Default: a dense module casts. Override in a leaf that pins its dtype.

<div class="api-declared">

Declared in `ClikaRT/nn/qmoe_woq.h`, line 121

</div>

### forward() {#forward}

<div class="api-signature">

[`Tensor`](../Tensor.md) **`forward`**(<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `x`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`Tensor`](../Tensor.md) `router_logits`,<br />
&nbsp;&nbsp;&nbsp;&nbsp;[`OptionalTensor`](../../index.md#OptionalTensor) `shared_output` `=` `{}`<br />
) `const`

</div>

One fused [MoE](./MoE.md) step: `x` `[T, H]` and the caller-computed `router_logits` `[T, E]` → `[T, H]` (dtype mirrors `x`). The first call packs (once, thread-safe); every later call reuses the pack. Raises [`ClikaRT::Error`](../Error.md) on a shape/dtype mismatch or a still-fake slot. An optional `shared_output` (the shared-expert branch's `[T, H]` at `x`'s dtype, EXACT shape) folds into the routed combine (the fused routed+shared epilogue; no separate add pass).

<div class="api-declared">

Declared in `ClikaRT/nn/qmoe_woq.h`, line 132

</div>

## Protected member functions

### initialize\_impl() {#initialize_impl}

<div class="api-signature">

`virtual` [`Result`](../Result.md)`<void>` **`initialize_impl`**() `override`

</div>

The pack hook ([`Module::initialize_impl`](./Module.md#initialize_impl)): packs now (idempotent, thread-safe). Every declared slot must hold a real (loaded) tensor; a still-fake slot is a clean error. After the pack the expert weight slots read back from the pack, the single resident copy; biases stay bound.

<div class="api-declared">

Declared in `ClikaRT/nn/qmoe_woq.h`, line 149

</div>