Skip to main content

ClikaRT::nn::QConv

class

Header: ClikaRT/nn/qconv.h
Inherits: ClikaRT::nn::Module

A bound, weight-packing static-quant (QDQ) convolution, the public face of the runtime's quantized conv, exposed as an nn::Module leaf. BOTH operands are quantized: the weight binds once as a QTensor (codes + scheme) and packs into the backend's integer kernel layout on the first forward; the activation arrives ALREADY quantized (from calibration or a preceding quantize; this module never quantizes it). For the weight-only flavor (float activations against a quantized weight) use QConvWoQ instead.

The lifecycle (uniform across every weight-bearing nn module):

  1. make(in, out, kernel, options, ...), the ONE constructor: declares storage-free weight / bias slots under their canonical names.
  2. set_weights(qw[, b]) or load_state_dict(...) binds; a quantized payload lands as-is (its scheme travels with it). Optionally set_output_quantization(...) binds the output requant params.
  3. forward(x) packs ON FIRST CALL (once, thread-safe), then serves the packed form every step. initialize() remains available as an optional warm-up.

Output dtype is conditional on the bound output quantization:

  • set_output_quantization CALLED: the conv product requantizes to the bound scheme and forward returns a QUANTIZED tensor carrying it.
  • NOT called: the dequant path; forward returns a FLOAT tensor at the activation scheme's float target dtype.

Layout: activations are channels-last [N, D1..Dn, C]; the weight's logical shape is OHWI [out_channels, K1..Kn, in_channels/groups] (the quantized codes' element geometry). An integer bias is used as-is in the integer accumulator at a_scale·w_scale (per output channel); a float bias adds in the real domain. Held via std::shared_ptr (an nn::Module leaf); copy/move are pinned by the base.

Statically-quantized convolution module: quantized weight AND activation-quantized compute where the backend serves the scheme, Conv's static-quant sibling (construct with a QTensor weight).

Layouts follow Conv (channels-last, OHWI weight). The module owns its weights: construct with make(...), or declare shapes and bind a checkpoint via load_state_dict. After the first forward (or initialize()) the weight lives ONLY in the backend's packed form (one resident copy); to(dtype) restores, casts, and repacks on the next forward.

Static member functions​

make()​

static std::shared_ptr<QConv> make(
    std::int64_t in_channels,
    std::int64_t out_channels,
    Span<const std::int64_t> kernel,
    Span<const std::int64_t> stride = {},
    Span<const std::int64_t> padding = {},
    Span<const std::int64_t> dilation = {},
    std::int64_t groups = 1,
    bool bias = false,
    ops::PadMode mode = ops::PadMode::Constant,
    std::optional<double> value = std::nullopt,
    std::optional<ops::Activation> activation = std::nullopt,
    DataType dtype = DataType::Float32,
    Device device = Device::cpu()
)

Defaults: stride / padding / dilation = empty (unit stride, no pad, unit dilation), groups = 1, bias = false (no bias slot), mode = PadMode::Constant, value = unset (zero fill), activation = none, dtype = Float32, device = CPU.

Throws

  • ClikaRT::Error: as stated above.

Declared in ClikaRT/nn/qconv.h, line 84

Member functions​

~QConv()​

~QConv() override

Declared in ClikaRT/nn/qconv.h, line 98

set_weights()​

void set_weights(QTensor weight, OptionalTensor bias = {})

Bind the declared slots positionally: weight (a quantized QTensor whose logical shape is OHWI [out, K.., in/groups]) and, when the module was made with one, bias ([out]; an integer bias is used as-is in the integer accumulator at a_scale·w_scale; a float bias adds in the real domain). The quantized payload lands as-is (no dtype cast; quant geometry is scheme-defined; the logical dims are checked). Re-binding after a pack drops the pack. Raises ClikaRT::Error on a geometry mismatch or a bias without a declared bias slot.

Declared in ClikaRT/nn/qconv.h, line 110

set_output_quantization()​

void set_output_quantization(
    Tensor scale,
    OptionalTensor zero_point = {},
    std::int64_t quant_axis = -1,
    std::optional<DataType> out_dtype = std::nullopt
)

Optionally bind the OUTPUT quantization: the conv product requantizes to scale / zero_point (per-tensor affine) and forward returns a quantized tensor whose code dtype is out_dtype (defaulting to the zero-point's dtype when one is bound, else Int8). Calling it again re-binds (and drops any pack); never calling it keeps the float (dequant) output path. Raises ClikaRT::Error on undefined scale.

Declared in ClikaRT/nn/qconv.h, line 123

initialize_impl()​

virtual Result<void> initialize_impl() override

Optional warm-up: run the first-forward pack NOW (idempotent, thread-safe). Every declared slot must hold a real (loaded) tensor; a still-fake slot is a clean error. After the pack the weight slot is released (the packed form is the single resident quantized copy), so named_parameters() enumerates only the unpacked slots; a re-bind re-inserts and the next pack re-releases.

Declared in ClikaRT/nn/qconv.h, line 136

to_impl(StreamOrDevice)​

virtual Result<void> to_impl(StreamOrDevice where) override

Placement move: rebuilds the pack on the target. A dtype cast is Unsupported; the weight stays quantized at rest, and dequantize explicitly if a dense copy is wanted.

Declared in ClikaRT/nn/qconv.h, line 141

to_impl(DataType)​

virtual Result<void> to_impl(DataType dtype) override

Declared in ClikaRT/nn/qconv.h, line 142

forward()​

Tensor forward(QTensor x) const

Apply the bound quantized weight to the QUANTIZED activation: x [N, D1..Dn, in] (a QTensor, quantized by calibration or a preceding quantize) -> [N, O1..On, out] (float, or quantized when set_output_quantization bound a scheme). The first call packs (once, thread-safe); every later call reuses the pack. Raises ClikaRT::Error on a shape mismatch or a still-fake slot.

Declared in ClikaRT/nn/qconv.h, line 151