Skip to main content

QMoEWoQ

Weight-only-quantized mixture-of-experts: the stacked expert weights bind as :class:QTensor and rest quantized; the activation stays floating point.

__init__​

__init__(self, num_experts: 'int', hidden_size: 'int', intermediate_size: 'int', top_k: 'int', *, gate_up_bias: 'bool' = False, down_bias: 'bool' = False, dtype: 'DataType' = DataType.Float32, device: 'Device | None' = None, **options: 'Any') -> 'None'

Declare the expert stacks; same counts, flags, and routing kwargs as :class:MoE (dtype declares the LOGICAL element type).

extra_repr​

extra_repr(self) -> 'str'

One line of per-class detail for :meth:__repr__ (a layer prints its geometry here).

forward​

forward(self, x: 'Tensor', router_logits: 'Tensor', shared_output: 'Tensor | None' = None) -> 'Tensor'

Subclasses define the computation here; call the module itself (m(x)), not forward directly.

set_weights​

set_weights(self, gate_up_experts: 'QTensor', down_experts: 'QTensor', *, gate_up_bias: 'Tensor | None' = None, down_bias: 'Tensor | None' = None, gate_experts: 'QTensor | None' = None, gate_bias: 'Tensor | None' = None, e_score_correction_bias: 'Tensor | None' = None) -> 'None'

Bind the quantized expert stacks (schemes travel with them) and the declared float biases.