Skip to main content

ClikaRT::nn::QConvWoQ

class

Header: ClikaRT/nn/qconv_woq.h
Inherits: ClikaRT::nn::Module

A bound, weight-packing weight-only-quantized convolution, the public face of the runtime's WoQ conv, exposed as an nn::Module leaf. The weight arrives quantized (a QTensor: codes + scheme) and STAYS quantized at rest; the first forward packs it into the backend's fused layout where the geometry/scheme admit one (the fused path computes straight from the quantized bytes), or serves the exact dequant fallback otherwise. Build one per layer at load, reuse it every step.

The lifecycle (uniform across every weight-bearing nn module):

  1. make(in, out, kernel, options, ...), the ONE constructor: declares storage-free weight / bias slots under their canonical names.
  2. set_weights(qw[, b]) or load_state_dict(...) binds; a quantized payload lands as-is (its scheme travels with it).
  3. forward(x) packs ON FIRST CALL (once, thread-safe), then serves the packed form every step; LoadOptions::pack_on_load pays the pack at load time.

Layout: activations are channels-last [N, D1..Dn, C]; the weight's logical shape is OHWI [out_channels, K1..Kn, in_channels/groups] (the quantized codes' element geometry). Held via std::shared_ptr (an nn::Module leaf); copy/move are pinned by the base.

Weight-only-quantized convolution module: the weight rests quantized (a QTensor) and decodes inside the kernels; compute runs at the activation dtype, Conv's weight-only sibling.

Layouts follow Conv (channels-last, OHWI weight). The module owns its weights: construct with make(...), or declare shapes and bind a checkpoint via load_state_dict. The first forward packs the weight (or LoadOptions::pack_on_load packs it at the end of the load); the pack is the one resident copy, and named_parameters() / state_dict() read the weight back from it; to(dtype) restores, casts, and re-packs on the next forward.

Static member functions​

make()​

static std::shared_ptr<QConvWoQ> make(
    std::int64_t in_channels,
    std::int64_t out_channels,
    Span<const std::int64_t> kernel,
    Span<const std::int64_t> stride = {},
    Span<const std::int64_t> padding = {},
    Span<const std::int64_t> dilation = {},
    std::int64_t groups = 1,
    bool bias = false,
    ops::PadMode mode = ops::PadMode::Constant,
    std::optional<double> value = std::nullopt,
    std::optional<ops::Activation> activation = std::nullopt,
    DataType dtype = DataType::Float32,
    Device device = Device::cpu()
)

Defaults: stride / padding / dilation = empty (unit stride, no pad, unit dilation), groups = 1, bias = false (no bias slot), mode = PadMode::Constant, value = unset (zero fill), activation = none, dtype = Float32, device = CPU.

Throws

  • ClikaRT::Error: as stated above.

Declared in ClikaRT/nn/qconv_woq.h, line 72

Member functions​

~QConvWoQ()​

~QConvWoQ() override

Declared in ClikaRT/nn/qconv_woq.h, line 86

set_weights()​

void set_weights(QTensor weight, OptionalTensor bias = {})

Bind the declared slots positionally: weight (a quantized QTensor whose logical shape is OHWI [out, K.., in/groups]) and, when the module was made with one, bias ([out]). The quantized payload lands as-is (no dtype cast; quant geometry is scheme-defined; the logical dims are checked). Re-binding after a pack drops the pack. Raises ClikaRT::Error on a geometry mismatch or a bias without a declared bias slot.

Declared in ClikaRT/nn/qconv_woq.h, line 96

to_impl(StreamOrDevice)​

virtual Result<void> to_impl(StreamOrDevice where) override

Placement move: rebuilds the pack on the target. A dtype cast is Unsupported; the weight stays quantized at rest, and dequantize explicitly if a dense copy is wanted. can_cast_to_impl reports the same refusal without moving a byte (the tree-wide check behind Module::to(dtype)).

Declared in ClikaRT/nn/qconv_woq.h, line 105

to_impl(DataType)​

virtual Result<void> to_impl(DataType dtype) override

Declared in ClikaRT/nn/qconv_woq.h, line 106

can_cast_to_impl()​

virtual Result<void> can_cast_to_impl(DataType dtype) const override

Whether THIS module's own slots can be cast to dtype (children are asked by the tree walk, never here). A failure is the refusal to(dtype) would raise; Module::to(dtype) asks every module of the tree BEFORE it casts any, so a refused cast leaves the tree untouched and names the leaf. Default: a dense module casts. Override in a leaf that pins its dtype.

Declared in ClikaRT/nn/qconv_woq.h, line 107

forward()​

Tensor forward(Tensor x) const

Apply the bound quantized weight: x [N, D1..Dn, in] -> [N, O1..On, out] (then bias + activation if configured). The first call packs (once, thread-safe); every later call reuses the pack. Raises ClikaRT::Error on a shape/dtype mismatch or a still-fake slot.

Declared in ClikaRT/nn/qconv_woq.h, line 115

Protected member functions​

initialize_impl()​

virtual Result<void> initialize_impl() override

The pack hook (Module::initialize_impl): packs now (idempotent, thread-safe). Every declared slot must hold a real (loaded) tensor; a still-fake slot is a clean error. Where the geometry/scheme admit the fused pack, the packed form becomes the single resident quantized copy and the weight slot reads back from it; a geometry the fused path does not serve keeps the quantized weight bound for the exact dequant fallback.

Declared in ClikaRT/nn/qconv_woq.h, line 129