Skip to main content

ClikaRT::nn::QConvWoQ

class

Header: ClikaRT/nn/qconv_woq.h
Inherits: ClikaRT::nn::Module

A bound, weight-packing weight-only-quantized convolution, the public face of the runtime's WoQ conv, exposed as an nn::Module leaf. The weight arrives quantized (a QTensor: codes + scheme) and STAYS quantized at rest; the first forward packs it into the backend's fused layout where the geometry/scheme admit one (the fused path computes straight from the quantized bytes), or serves the exact dequant fallback otherwise. Build one per layer at load, reuse it every step.

The lifecycle (uniform across every weight-bearing nn module):

  1. make(in, out, kernel, options, ...), the ONE constructor: declares storage-free weight / bias slots under their canonical names.
  2. set_weights(qw[, b]) or load_state_dict(...) binds; a quantized payload lands as-is (its scheme travels with it).
  3. forward(x) packs ON FIRST CALL (once, thread-safe), then serves the packed form every step. initialize() remains available as an optional warm-up.

Layout: activations are channels-last [N, D1..Dn, C]; the weight's logical shape is OHWI [out_channels, K1..Kn, in_channels/groups] (the quantized codes' element geometry). Held via std::shared_ptr (an nn::Module leaf); copy/move are pinned by the base.

Weight-only-quantized convolution module: the weight rests quantized (a QTensor) and decodes inside the kernels; compute runs at the activation dtype, Conv's weight-only sibling.

Layouts follow Conv (channels-last, OHWI weight). The module owns its weights: construct with make(...), or declare shapes and bind a checkpoint via load_state_dict. After the first forward (or initialize()) the weight lives ONLY in the backend's packed form (one resident copy); to(dtype) restores, casts, and repacks on the next forward.

Static member functions​

make()​

static std::shared_ptr<QConvWoQ> make(
    std::int64_t in_channels,
    std::int64_t out_channels,
    Span<const std::int64_t> kernel,
    Span<const std::int64_t> stride = {},
    Span<const std::int64_t> padding = {},
    Span<const std::int64_t> dilation = {},
    std::int64_t groups = 1,
    bool bias = false,
    ops::PadMode mode = ops::PadMode::Constant,
    std::optional<double> value = std::nullopt,
    std::optional<ops::Activation> activation = std::nullopt,
    DataType dtype = DataType::Float32,
    Device device = Device::cpu()
)

Defaults: stride / padding / dilation = empty (unit stride, no pad, unit dilation), groups = 1, bias = false (no bias slot), mode = PadMode::Constant, value = unset (zero fill), activation = none, dtype = Float32, device = CPU.

Throws

  • ClikaRT::Error: as stated above.

Declared in ClikaRT/nn/qconv_woq.h, line 72

Member functions​

~QConvWoQ()​

~QConvWoQ() override

Declared in ClikaRT/nn/qconv_woq.h, line 86

set_weights()​

void set_weights(QTensor weight, OptionalTensor bias = {})

Bind the declared slots positionally: weight (a quantized QTensor whose logical shape is OHWI [out, K.., in/groups]) and, when the module was made with one, bias ([out]). The quantized payload lands as-is (no dtype cast; quant geometry is scheme-defined; the logical dims are checked). Re-binding after a pack drops the pack. Raises ClikaRT::Error on a geometry mismatch or a bias without a declared bias slot.

Declared in ClikaRT/nn/qconv_woq.h, line 96

initialize_impl()​

virtual Result<void> initialize_impl() override

Optional warm-up: run the first-forward pack NOW (idempotent, thread-safe). Every declared slot must hold a real (loaded) tensor; a still-fake slot is a clean error. Where the geometry/scheme admit the fused pack, the packed form becomes the single resident quantized copy and the weight slot is released; a geometry the fused path does not serve keeps the quantized weight bound for the exact dequant fallback.

Declared in ClikaRT/nn/qconv_woq.h, line 106

to_impl(StreamOrDevice)​

virtual Result<void> to_impl(StreamOrDevice where) override

Placement move: rebuilds the pack on the target. A dtype cast is Unsupported; the weight stays quantized at rest, and dequantize explicitly if a dense copy is wanted.

Declared in ClikaRT/nn/qconv_woq.h, line 111

to_impl(DataType)​

virtual Result<void> to_impl(DataType dtype) override

Declared in ClikaRT/nn/qconv_woq.h, line 112

forward()​

Tensor forward(Tensor x) const

Apply the bound quantized weight: x [N, D1..Dn, in] -> [N, O1..On, out] (then bias + activation if configured). The first call packs (once, thread-safe); every later call reuses the pack. Raises ClikaRT::Error on a shape/dtype mismatch or a still-fake slot.

Declared in ClikaRT/nn/qconv_woq.h, line 120