ClikaRT::quant::QuantizationConfig
struct
Header: ClikaRT/compute/q_tensor.h
The checkpoint-level facts a HF quantization_config dict carries: the factory arguments the dict OWNS, normalized across the exporters' spellings. Per-weight facts (the tensors themselves, a bitsandbytes quant_state blocksize, a GPTQ weight's input extent) stay with the state dict; this struct is the dict side only.
Types
enum ZerosConvention
enum class ZerosConvention
The GPTQ checkpoint zero-point storage convention. The original exporters store each group zero MINUS ONE (the true zero is stored + 1, wrapped at the code width); checkpoints declaring checkpoint_format: "gptq_v2" store true zeros. Name the checkpoint's convention explicitly; guessing it wrong shifts every decoded weight by one scale step, silently.
| Enumerator | Description |
|---|---|
OffsetByOne | stored = true − 1 (the original format) |
Exact | stored = true (gptq_v2) |
Declared in ClikaRT/compute/q_tensor.h, line 272
Data members
method
which checkpoint family the dict declares (routes the parse)
Declared in ClikaRT/compute/q_tensor.h, line 277
bits
std::int64_t bits = 0
Code width (GPTQ/AWQ bits, AWQ's standalone w_bit; 4 for bitsandbytes 4-bit).
Declared in ClikaRT/compute/q_tensor.h, line 280
group_size
std::int64_t group_size = 0
Elements per quantization group along the input axis (GPTQ group_size, AWQ group_size/q_group_size). GPTQ's -1 spelling (one group spanning the whole input axis) passes through verbatim; resolve it against the weight's own input extent at bind time. 0 for bitsandbytes (its per-tensor blocksize rides the checkpoint's quant_state tensors, not the dict).
Declared in ClikaRT/compute/q_tensor.h, line 287
sym
bool sym = false
Symmetric codes (GPTQ sym; AWQ !zero_point; bitsandbytes codebooks are zero-point-free, so true).
Declared in ClikaRT/compute/q_tensor.h, line 290
desc_act
bool desc_act = false
GPTQ act-order (desc_act): the shipped g_idx maps input rows to groups out of order. make_quantized_gptq imports it by sorting rows into contiguous groups and carrying the permutation on the scheme (decode restores original order; serving rides the CPU dequant-compose route), surfaced so a loader can route or report before touching tensors.
Declared in ClikaRT/compute/q_tensor.h, line 297
zeros_convention
ZerosConvention zeros_convention = ZerosConvention::OffsetByOne
The GPTQ zero-point storage convention (ZerosConvention above), resolved from the dict's checkpoint_format: "gptq_v2" ⇒ Exact; "gptq" or absent ⇒ OffsetByOne. Meaningful for Gptq only.
Declared in ClikaRT/compute/q_tensor.h, line 303
lm_head_quantized
bool lm_head_quantized = false
GPTQ lm_head: the head projection is itself quantized (GPTQModel exporters). Default false; the common export keeps the head dense.
Declared in ClikaRT/compute/q_tensor.h, line 306
quant_type
std::string quant_type
bitsandbytes code grid: bnb_4bit_quant_type ("nf4" / "fp4"), the quant_type to pass make_quantized_bnb4. Empty otherwise.
Declared in ClikaRT/compute/q_tensor.h, line 309
double_quant
bool double_quant = false
bitsandbytes bnb_4bit_use_double_quant: the absmax plane itself arrives quantized (pass the nested parts to make_quantized_bnb4).
Declared in ClikaRT/compute/q_tensor.h, line 312
quant_storage
std::string quant_storage
bitsandbytes bnb_4bit_quant_storage: the dtype the packed code buffer is VIEWED as in the checkpoint ("uint8" is the plain form; a wider view is a sharded-training artifact to re-view as bytes before the factory's flat-UInt8 contract).
Declared in ClikaRT/compute/q_tensor.h, line 317
modules_to_not_convert
std::vector<std::string> modules_to_not_convert
Module-name entries the checkpoint left DENSE (AWQ modules_to_not_convert, bitsandbytes llm_int8_skip_modules); a loader must route these to the dense path or the named weights decode as garbage. GPTQ carries no such list (its per-weight signal is the qweight/qzeros/scales tensor presence; see lm_head_quantized).
Declared in ClikaRT/compute/q_tensor.h, line 323