---
title: "ClikaRT::quant::QuantizationConfig"
sidebar_label: "QuantizationConfig"
description: "The checkpoint-level facts a HF quantization_config dict carries: the factory arguments the dict OWNS, normalized across the exporters' spellings."
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--struct">struct</span>

**Header:** `ClikaRT/compute/q_tensor.h`

The checkpoint-level facts a HF `quantization_config` dict carries: the factory arguments the dict OWNS, normalized across the exporters' spellings. Per-weight facts (the tensors themselves, a bitsandbytes `quant_state` blocksize, a GPTQ weight's input extent) stay with the state dict; this struct is the dict side only.

## Types

### enum ZerosConvention {#ZerosConvention}

<div class="api-signature">

`enum class` **`ZerosConvention`**

</div>

The GPTQ checkpoint zero-point storage convention. The original exporters store each group zero MINUS ONE (the true zero is `stored + 1`, wrapped at the code width); checkpoints declaring `checkpoint_format: "gptq_v2"` store true zeros. Name the checkpoint's convention explicitly; guessing it wrong shifts every decoded weight by one scale step, silently.

| Enumerator | Description |
| --- | --- |
| `OffsetByOne` | stored = true − 1 (the original format) |
| `Exact` | stored = true (`gptq_v2`) |

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 309

</div>

## Data members

### method {#method}

<div class="api-signature">

[`QuantizedCheckpointMethod`](./index.md#QuantizedCheckpointMethod) **`method`** `=` [`QuantizedCheckpointMethod::Gptq`](./index.md#QuantizedCheckpointMethod)

</div>

which checkpoint family the dict declares (routes the parse)

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 314

</div>

### bits {#bits}

<div class="api-signature">

`std::int64_t` **`bits`** `= 0`

</div>

Code width (GPTQ/AWQ [`bits`](./QuantizationConfig.md#bits), AWQ's standalone `w_bit`; 4 for bitsandbytes 4-bit).

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 317

</div>

### group\_size {#group_size}

<div class="api-signature">

`std::int64_t` **`group_size`** `= 0`

</div>

Elements per quantization group along the input axis (GPTQ [`group_size`](./QuantizationConfig.md#group_size), AWQ [`group_size`](./QuantizationConfig.md#group_size)/`q_group_size`). GPTQ's `-1` spelling (one group spanning the whole input axis) passes through verbatim; resolve it against the weight's own input extent at bind time. 0 for bitsandbytes (its per-tensor blocksize rides the checkpoint's `quant_state` tensors, not the dict).

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 324

</div>

### sym {#sym}

<div class="api-signature">

`bool` **`sym`** `= false`

</div>

Symmetric codes (GPTQ [`sym`](./QuantizationConfig.md#sym); AWQ `!zero_point`; bitsandbytes codebooks are zero-point-free, so true).

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 327

</div>

### desc\_act {#desc_act}

<div class="api-signature">

`bool` **`desc_act`** `= false`

</div>

GPTQ act-order ([`desc_act`](./QuantizationConfig.md#desc_act)): the shipped `g_idx` maps input rows to groups out of order. [`make_quantized_gptq`](../../index.md#make_quantized_gptq) imports it by sorting rows into contiguous groups and carrying the permutation on the scheme (decode restores original order; serving rides the dequant-compose route where a backend has no act-order tier), surfaced so a loader can route or report before touching tensors.

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 334

</div>

### zeros\_convention {#zeros_convention}

<div class="api-signature">

[`ZerosConvention`](./QuantizationConfig.md#ZerosConvention) **`zeros_convention`** `=` [`ZerosConvention::OffsetByOne`](./QuantizationConfig.md#ZerosConvention)

</div>

The GPTQ zero-point storage convention ([`ZerosConvention`](./QuantizationConfig.md#ZerosConvention) above), resolved from the dict's `checkpoint_format`: `"gptq_v2"` ⇒ `Exact`; `"gptq"` or absent ⇒ `OffsetByOne`. Meaningful for `Gptq` only.

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 340

</div>

### lm\_head\_quantized {#lm_head_quantized}

<div class="api-signature">

`bool` **`lm_head_quantized`** `= false`

</div>

GPTQ `lm_head`: the head projection is itself quantized (GPTQModel exporters). Default false; the common export keeps the head dense.

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 343

</div>

### quant\_type {#quant_type}

<div class="api-signature">

`std::string` **`quant_type`**

</div>

bitsandbytes code grid: `bnb_4bit_quant_type` ("nf4" / "fp4"), the [`quant_type`](./QuantizationConfig.md#quant_type) to pass [`make_quantized_bnb4`](../../index.md#make_quantized_bnb4). Empty otherwise.

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 346

</div>

### double\_quant {#double_quant}

<div class="api-signature">

`bool` **`double_quant`** `= false`

</div>

bitsandbytes `bnb_4bit_use_double_quant`: the absmax plane itself arrives quantized (pass the nested parts to [`make_quantized_bnb4`](../../index.md#make_quantized_bnb4)).

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 349

</div>

### quant\_storage {#quant_storage}

<div class="api-signature">

`std::string` **`quant_storage`**

</div>

bitsandbytes `bnb_4bit_quant_storage`: the dtype the packed code buffer is VIEWED as in the checkpoint ("uint8" is the plain form; a wider view is a sharded-training artifact to re-view as bytes before the factory's flat-UInt8 contract).

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 354

</div>

### modules\_to\_not\_convert {#modules_to_not_convert}

<div class="api-signature">

`std::vector<std::string>` **`modules_to_not_convert`**

</div>

Module-name entries the checkpoint left DENSE (AWQ [`modules_to_not_convert`](./QuantizationConfig.md#modules_to_not_convert), bitsandbytes `llm_int8_skip_modules`); a loader must route these to the dense path or the named weights decode as garbage. GPTQ carries no such list (its per-weight signal is the qweight/qzeros/scales tensor presence; see [`lm_head_quantized`](./QuantizationConfig.md#lm_head_quantized)).

<div class="api-declared">

Declared in `ClikaRT/compute/q_tensor.h`, line 360

</div>