ClikaRT::quant

namespace

The quantized-checkpoint import taxonomy: the parsed quantization_config facts and the routing enums the split factories consume.

Classes

Name	Description
QuantizationConfig	The checkpoint-level facts a HF quantization_config dict carries: the factory arguments the dict OWNS, normalized across the exporters' spellings. Per-weight facts (the tensors themselves, a bitsandbytes quant_state blocksize, a GPTQ weight's input extent) stay with the state dict; this struct is the dict side only.

Enumerations

enum QuantizedCheckpointMethod

enum class QuantizedCheckpointMethod

Which serving import route a parsed quantization_config resolves to; one enumerator per split factory above.

Enumerator	Description
Gptq	make_quantized_gptq (per weight: qweight/qzeros/scales[/g_idx])
Awq	make_quantized_awq (the AutoAWQ GEMM layout)
Bnb4	make_quantized_bnb4 (bitsandbytes nf4/fp4)

Declared in ClikaRT/compute/q_tensor.h, line 254

Functions

parse_quantization_config()

QuantizationConfig parse_quantization_config(std::string_view json_text)

Declared in ClikaRT/compute/q_tensor.h, line 340

ClikaRT/compute/q_tensor.h

#include <ClikaRT/compute/q_tensor.h>

QTensor: the typed public view of a quantized weight. A quantized checkpoint entry (e.g. a block-quantized tensor from io::load_gguf) rides a plain Tensor whose element data is the packed byte stream; QTensor names what that payload IS: the packed data plus the quantization scheme and the logical element shape it encodes. Build one with quantized_view(tensor) and hand it to the primitives that consume quantized weights (QLinearWoQ).

── ORIENTATION MAP: read this before wiring a new checkpoint format ──────

Each ingestion entry below takes its checkpoint format's NATIVE orientation; that is what makes adoption zero-copy (a unified convention would force a transpose of somebody's checkpoint bytes), so the orientations deliberately DIFFER across entries, and mirroring the nearest sibling is how a new format gets wired wrong. The map, for the entries that take a shape argument (the ones a caller can mis-orient):

entry	2-D shape argument reads as
make_quantized	{in_features, out_features} (k-major file
make_quantized_mxfp4	order; row-contiguous dim FIRST)

make_quantized_nvfp4 | make_quantized_fp8 | {in_features, out_features}; a checkpoint | storing [out, in] transposes BEFORE entry make_quantized_fp8_blocked | {out_features, in_features}; N-MAJOR, the | checkpoint's own order (blocks run along | in_features, the trailing axis) make_quantized_bnb4 | the checkpoint's own [out, in] make_quantized_affine | {out_features, in_features}; N-MAJOR, the | orientation this scheme's checkpoints ship

A stacked expert weight keeps its entry's file order with the expert dim LAST ({K, N_per_expert, E}). The word-split adopts (make_quantized_gptq / _awq / _ct_pack) take no shape argument; the stored word layout IS their contract, and each entry's own doc names the adopted orientation.

THREE AUTHORITIES, one weight: (1) the ingestion entry above fixes how the payload's bytes are LABELED; (2) the module bind (nn::QLinearWoQ / the linear family) states the layout it expects of the labeled weight; (3) the runtime canonicalizes orientation ONCE at first use (the transpose_b fold), or, for a scheme whose canonical layout is n-major, keeps the bound layout as the served form, so a mislabeled weight is refused by the structural gates (loudly, at load), never multiplied in the wrong orientation. When adding a format, pick the entry whose orientation matches the CHECKPOINT's bytes; never re-shape the bytes to fit a different entry.