ClikaRT::ops::fused
namespace
Enumerations
enum SwigluFusion
enum class SwigluFusion : std::uint8_t
How the gate‖up halves are laid out in a gated MoE FC1 output.
| Enumerator | Value | Description |
|---|---|---|
Unfused | 0 | gate comes from a SEPARATE fc3 matrix |
Interleaved | 1 | one fused [E, 2I, H]; gate = even, up = odd columns |
Concat | 2 | one fused [E, 2I, H]; gate = first half, up = second |
Declared in ClikaRT/compute/fused_op_enums.h, line 14
enum QComputeMode
enum class QComputeMode : std::uint8_t
How a weight-only-quantized contraction (qmatmul_woq / qlinear_woq / the *_woq conv family) contracts a floating-point activation against a quantized weight.
ExactFP (the default) keeps the activation in floating point wherever the weight's format has a float route: an affine, codebook, FP8 or dense-coded weight dequantizes and runs the float dot, bit-faithful to the stored scale / zero-point. A packed block weight (the GGUF Q-series, K-quant and i-quant formats, the MX formats and NVFP4) runs on an int8 block tier instead where the backend has one: the activation rounds to per-block int8 before an integer dot, so the result differs from the float dot by that rounding.
- CPU: every GGUF weight format, MXFP4, MXINT8 and NVFP4; MXFP6 and MXFP8 keep the float dot.
- CUDA and Vulkan: the block formats their integer-dot tier covers, where K is a whole number of 32-element blocks (on Vulkan, also where the device's integer dot is verified); other formats keep the float dot.
- Metal and TPU: no int8 tier; every block weight keeps the float dot. On the runtime's parity tests (uniform activations) the rounding stays below a relative L2 of 0.03 to 0.05 against the float dot, or 0.08 with a bf16 activation. An activation with outlier channels can exceed these bounds.
DynamicInt8 dynamically quantizes the activation to int8 and runs an integer dot, faster on dot-product hardware, at a small accuracy cost (the activation loses precision). A ROUTING HINT, not a shape contract: shapes outside a backend's fused coverage fall back to ExactFP.
| Enumerator | Value | Description |
|---|---|---|
ExactFP | 0 | |
DynamicInt8 | 1 |
Declared in ClikaRT/compute/fused_op_enums.h, line 48
ClikaRT/compute/fused_op_enums.h
#include <ClikaRT/compute/fused_op_enums.h>
Enum parameters of the FUSED operators: how a gated MoE lays out its gate‖up halves, and how a weight-only-quantized contraction is computed. Values are PINNED (ABI): never reorder or remove a value; append new ones at the end.