Skip to main content

ClikaRT::nn::QLinearWoQ

class

Header: ClikaRT/nn/qlinear_woq.h
Inherits: ClikaRT::nn::Linear

QLinearWoQ: the weight-only-quantization PIN of the Linear family: a Linear whose weight MUST be quantized-at-rest. The activation stays floating point and the weight stays quantized for the module's lifetime; it is never materialized dense at load. The first forward packs the quantized payload once into the backend's kernel layout (the packed form is then the single resident copy); every later forward runs the quantized-weight matmul off it. This is the path that serves a block-quantized checkpoint at its on-disk footprint.

A plain Linear already SERVES a quantized payload that binds onto its declared slot (the base routes at pack time; see linear.h); this subclass exists to make the posture a CONTRACT: a dense payload is refused with a clean error, at set_weights and at the pack gate alike, instead of being silently dense-packed. kind() reports WeightQuantized constantly. Construct through Linear::make (which returns this class for a quantized weight) or pin explicitly through the factories here; the lifecycle, movement, and forward are the family's (linear.h).

The weight's canonical logical orientation is [in_features, out_features], exactly what io::load_gguf produces for a block-quantized matrix. A payload in the transposed [out, in] logical order is accepted too, validated against the declared feature counts (the ambiguous square case reads as the canonical [in, out]); the pack resolves the orientation. A dtype to is Unsupported: the weight stays quantized at rest; dequantize explicitly for a dense copy.

Weight-only-quantized linear module: the weight rests quantized (a QTensor) and decodes inside the matmul kernels; compute runs at the activation dtype, Linear's weight-only sibling (same forward surface).

The module owns its weights: construct with make(...), or declare shapes and bind a checkpoint via load_state_dict. After the first forward (or initialize()) the weight lives ONLY in the backend's packed form (one resident copy); to(dtype) restores, casts, and repacks on the next forward.

Static member functions​

make(Tensor, OptionalTensor, LinearOptions)​

static std::shared_ptr<QLinearWoQ> make(
    Tensor weight,
    OptionalTensor bias = {},
    LinearOptions options = {}
)

Construct FROM a quantized weight (a static factory, so a bad config is a clean error): geometry (the payload's logical shape, read in the canonical [in, out] loader order), dtype and device all read off the payload; declared AND bound in one call (an optional float bias [out] binds beside it; a bias slot exists exactly when bias is passed). The QTensor overload is the typed convenience over the same factory. Raises ClikaRT::Error on a dense payload; that is this class's whole contract.

Declared in ClikaRT/nn/qlinear_woq.h, line 60

make(QTensor, OptionalTensor, LinearOptions)​

static std::shared_ptr<QLinearWoQ> make(
    QTensor weight,
    OptionalTensor bias = {},
    LinearOptions options = {}
)

Construct from a quantized weight [out, in] (+ optional bias and the shared LinearOptions). Raises when the scheme cannot be served.

Declared in ClikaRT/nn/qlinear_woq.h, line 67

make(int64_t, int64_t, LinearOptions)​

static std::shared_ptr<QLinearWoQ> make(
    std::int64_t in_features,
    std::int64_t out_features,
    LinearOptions options = {}
)

Construct FROM a quantized weight (a static factory, so a bad config is a clean error): geometry (the payload's logical shape, read in the canonical [in, out] loader order), dtype and device all read off the payload; declared AND bound in one call (an optional float bias [out] binds beside it; a bias slot exists exactly when bias is passed). The QTensor overload is the typed convenience over the same factory. Raises ClikaRT::Error on a dense payload; that is this class's whole contract.

Declared in ClikaRT/nn/qlinear_woq.h, line 84

Member functions​

~QLinearWoQ()​

~QLinearWoQ() override

Releases the module's packed weights.

Declared in ClikaRT/nn/qlinear_woq.h, line 91

kind()​

virtual LinearKind kind() const noexcept override

WeightQuantized, constantly; this class IS the posture pin.

Declared in ClikaRT/nn/qlinear_woq.h, line 94

set_weights(QTensor, OptionalTensor)​

void set_weights(QTensor weight, OptionalTensor bias = {})

Bind the declared slots positionally: the QUANTIZED weight (its logical shape in either orientation, validated against the declared feature counts) and, when the module was declared with one, a floating-point bias. A DENSE weight is refused, the pin this class exists for. Re-binding after a pack drops the pack. The QTensor overload is the typed convenience (e.g. a load_gguf entry through quantized_view). Raises ClikaRT::Error on a dense payload, a geometry mismatch, or a bias without a declared bias slot.

Declared in ClikaRT/nn/qlinear_woq.h, line 106

initialize_impl()​

virtual Result<void> initialize_impl() override

The family warm-up (linear.h), plus the pin at the pack gate: a DENSE payload that reached the slot by name (load_state_dict binds without set_weights) is refused here rather than dense-packed.

Declared in ClikaRT/nn/qlinear_woq.h, line 113

set_weights(Tensor, OptionalTensor)​

void set_weights(Tensor weight, OptionalTensor bias = {})

Bind the declared slots positionally: the QUANTIZED weight (its logical shape in either orientation, validated against the declared feature counts) and, when the module was declared with one, a floating-point bias. A DENSE weight is refused, the pin this class exists for. Re-binding after a pack drops the pack. The QTensor overload is the typed convenience (e.g. a load_gguf entry through quantized_view). Raises ClikaRT::Error on a dense payload, a geometry mismatch, or a bias without a declared bias slot.

Declared in ClikaRT/nn/qlinear_woq.h, line 105