Skip to main content

ClikaRT::nn::QLinearWoQ

class

Header: ClikaRT/nn/qlinear_woq.h
Inherits: ClikaRT::nn::Linear

QLinearWoQ: the weight-only-quantization PIN of the Linear family: a Linear whose weight MUST be quantized-at-rest. The activation stays floating point and the weight stays quantized for the module's lifetime; it is never materialized dense at load. The first forward packs the quantized payload once into the backend's kernel layout (the packed form is then the single resident copy); every later forward runs the quantized-weight matmul off it. This is the path that serves a block-quantized checkpoint at its on-disk footprint.

A plain Linear already SERVES a quantized payload that binds onto its declared slot (the base routes at pack time; see linear.h); this subclass exists to make the posture a CONTRACT: a dense payload is refused with a clean error, at set_weights and at the pack gate alike, instead of being silently dense-packed. kind() reports WeightQuantized constantly. Construct through Linear::make (which returns this class for a quantized weight) or pin explicitly through the factories here; the lifecycle, movement, and forward are the family's (linear.h).

The weight's orientation is the PAYLOAD's own, read by the family's one quantized-orientation law (linear.h): a grouped scheme is oriented by its group axis and a plain code tensor by its per-channel scale axis (one scale per output feature); past those, the declared counts decide a non-square shape, a load_gguf block stream reads in the loader's [in_features, out_features] order, and a per-tensor code tensor reads the dense [out, in] from the tensor alone, a square one included; a payload whose factory fixes its orientation (make_quantized_fp8: [in, out]) carries that statement and is read by it first. The registry slot declares [in_features, out_features] (the loader order), and a payload in either logical order binds onto it; the pack serves the resolved orientation. A dtype to is Unsupported: the weight stays quantized at rest; dequantize explicitly for a dense copy.

Weight-only-quantized linear module: the weight rests quantized (a QTensor) and decodes inside the matmul kernels; compute runs at the activation dtype, Linear's weight-only sibling (same forward surface).

The module owns its weights: construct with make(...), or declare shapes and bind a checkpoint via load_state_dict. The first forward packs the weight (or LoadOptions::pack_on_load packs it at the end of the load); the pack is the one resident copy, and named_parameters() / state_dict() read the weight back from it; to(dtype) restores, casts, and re-packs on the next forward.

Static member functions​

make(Tensor, OptionalTensor, LinearOptions)​

static std::shared_ptr<QLinearWoQ> make(
    Tensor weight,
    OptionalTensor bias = {},
    LinearOptions options = {}
)

Construct FROM a quantized weight (a static factory, so a bad config is a clean error): geometry (the payload's own orientation, by the law in linear.h), dtype and device all read off the payload; declared AND bound in one call (an optional float bias [out] binds beside it; a bias slot exists exactly when bias is passed). The QTensor overload is the typed convenience over the same factory. Raises ClikaRT::Error on a dense payload; that is this class's whole contract.

Declared in ClikaRT/nn/qlinear_woq.h, line 66

make(QTensor, OptionalTensor, LinearOptions)​

static std::shared_ptr<QLinearWoQ> make(
    QTensor weight,
    OptionalTensor bias = {},
    LinearOptions options = {}
)

Construct from a quantized weight in either logical order (its own orientation, by the law in linear.h; + optional bias and the shared LinearOptions). Raises when the scheme cannot be served, or when the payload's evidence contradicts a declared geometry.

Declared in ClikaRT/nn/qlinear_woq.h, line 75

make(int64_t, int64_t, LinearOptions)​

static std::shared_ptr<QLinearWoQ> make(
    std::int64_t in_features,
    std::int64_t out_features,
    LinearOptions options = {}
)

Construct FROM a quantized weight (a static factory, so a bad config is a clean error): geometry (the payload's own orientation, by the law in linear.h), dtype and device all read off the payload; declared AND bound in one call (an optional float bias [out] binds beside it; a bias slot exists exactly when bias is passed). The QTensor overload is the typed convenience over the same factory. Raises ClikaRT::Error on a dense payload; that is this class's whole contract.

Declared in ClikaRT/nn/qlinear_woq.h, line 92

Member functions​

~QLinearWoQ()​

~QLinearWoQ() override

Releases the module's packed weights.

Declared in ClikaRT/nn/qlinear_woq.h, line 99

kind()​

virtual LinearKind kind() const noexcept override

WeightQuantized, constantly; this class IS the posture pin.

Declared in ClikaRT/nn/qlinear_woq.h, line 102

set_weights(QTensor, OptionalTensor)​

void set_weights(QTensor weight, OptionalTensor bias = {})

Bind the declared slots positionally: the QUANTIZED weight (its logical shape in either orientation, read by the orientation law in linear.h and validated against the declared feature counts) and, when the module was declared with one, a floating-point bias. A DENSE weight is refused, the pin this class exists for. Re-binding after a pack drops the pack. The QTensor overload is the typed convenience (e.g. a load_gguf entry through quantized_view). Raises ClikaRT::Error on a dense payload, a geometry mismatch, or a bias without a declared bias slot.

Declared in ClikaRT/nn/qlinear_woq.h, line 115

set_weights(Tensor, OptionalTensor)​

void set_weights(Tensor weight, OptionalTensor bias = {})

Bind the declared slots positionally: the QUANTIZED weight (its logical shape in either orientation, read by the orientation law in linear.h and validated against the declared feature counts) and, when the module was declared with one, a floating-point bias. A DENSE weight is refused, the pin this class exists for. Re-binding after a pack drops the pack. The QTensor overload is the typed convenience (e.g. a load_gguf entry through quantized_view). Raises ClikaRT::Error on a dense payload, a geometry mismatch, or a bias without a declared bias slot.

Declared in ClikaRT/nn/qlinear_woq.h, line 114

Protected member functions​

initialize_impl()​

virtual Result<void> initialize_impl() override

The pack hook (linear.h), plus the pin at the pack gate: a DENSE payload that reached the slot by name (load_state_dict binds without set_weights) is refused here rather than dense-packed.

Declared in ClikaRT/nn/qlinear_woq.h, line 126