ClikaRT::nn::Module

class

Base class of every ClikaRT module: an ownership tree of parameters, buffers, and child modules.

Lifecycle: construct (register tensors and children) -> load_state_dict or set_weights binds a checkpoint onto the registered names -> to(...) places or casts the tree -> forward. The first forward packs each weight into its backend form, once and thread-safe (LoadOptions::pack_on_load pays that pack at the end of the load instead); after the pack exactly ONE resident copy of each weight exists, and named_parameters() / state_dict() stay total before and after it: a packed weight reads back from its pack as a fresh owning tensor, at its registered name and position. Held by std::shared_ptr (module trees share children).

Member functions

~Module()

virtual ~Module()

Virtual base destructor; derived modules release their packed weights.

Declared in ClikaRT/nn/module.h, line 147

Module(Module)

Module(const Module&) =delete

Declared in ClikaRT/nn/module.h, line 149

operator=(Module)

Module& operator=(const Module&) =delete

Declared in ClikaRT/nn/module.h, line 150

Module(Module)

Module(Module&&) =delete

Declared in ClikaRT/nn/module.h, line 151

operator=(Module)

Module& operator=(Module&&) =delete

Declared in ClikaRT/nn/module.h, line 152

register_parameter(string_view, Tensor)

Tensor register_parameter(std::string_view name, Tensor tensor)

Declared in ClikaRT/nn/module.h, line 161

register_buffer(string_view, Tensor)

Tensor register_buffer(std::string_view name, Tensor tensor)

Declared in ClikaRT/nn/module.h, line 164

register_parameter(string_view, FakeTensor)

Tensor register_parameter(std::string_view name, const FakeTensor& spec)

Declare a storage-free SLOT from a metadata spec (shape / dtype / placement): the slot enumerates under its dotted name, so load_state_dict matches it and binds the checkpoint tensor onto the spec's placement + dtype, and reads as Tensor::is_fake() until bound. A module built this way holds no weight memory until loading. The returned handle is the registered slot; re-registering the same name (e.g. with a real tensor) replaces it, keeping its position.

Declared in ClikaRT/nn/module.h, line 172

register_buffer(string_view, FakeTensor)

Tensor register_buffer(std::string_view name, const FakeTensor& spec)

Declare a storage-free buffer slot under name from the metadata spec; a later bind fills it (the buffer twin of the parameter overload above).

Declared in ClikaRT/nn/module.h, line 176

register_buffer(string_view, Tensor, bool)

Tensor register_buffer(
    std::string_view name,
    Tensor tensor,
    bool persistent
)

register_buffer with a persistence choice. A NON-persistent buffer (persistent == false) enumerates through named_buffers() and moves and casts with the tree, but stays OUT of state_dict(), and load_state_dict neither expects it nor binds it (a table the module derives itself, a cache). The two-argument forms register persistent buffers.

Declared in ClikaRT/nn/module.h, line 183

register_buffer(string_view, FakeTensor, bool)

Tensor register_buffer(
    std::string_view name,
    const FakeTensor& spec,
    bool persistent
)

Declared in ClikaRT/nn/module.h, line 184

register_module()

void register_module(std::string_view name, std::shared_ptr<Module> module)

Attach a child module under name (the tree edge name.child_key names its tensors).

Declared in ClikaRT/nn/module.h, line 187

release_parameter()

void release_parameter(std::string_view name)

Erase THIS module's parameter slot name (no dots) from the registry: a slot bound from a checkpoint's own mapping hands its pages back to the file first, then the entry goes. An absent name is a no-op success; a bound, live slot is released the same way (the caller owns the one-copy law: call it only once every reader of the raw values has finished).

Declared in ClikaRT/nn/module.h, line 202

release_buffer()

void release_buffer(std::string_view name)

The buffer twin of release_parameter: erase THIS module's buffer slot name (no dots), pages first. Same absent-name and live-slot contract.

Declared in ClikaRT/nn/module.h, line 206

named_parameters()

std::vector<std::pair<std::string, Tensor>> named_parameters(bool recurse = true) const

All parameters as (dotted-name, tensor) pairs; recurse walks children. A tensor shared under several names appears once, under the first. A weight a layer has packed for serving reads back from the pack as a fresh owning tensor, at its own name and position, so the list is total before and after the pack.

Declared in ClikaRT/nn/module.h, line 217

named_buffers()

std::vector<std::pair<std::string, Tensor>> named_buffers(bool recurse = true) const

All buffers as (dotted-name, tensor) pairs, persistent and non-persistent alike; recurse walks children.

Declared in ClikaRT/nn/module.h, line 220

named_children()

std::vector<std::pair<std::string, std::shared_ptr<Module>>> named_children() const

Direct children as (name, module) pairs.

Declared in ClikaRT/nn/module.h, line 222

parameters()

std::vector<Tensor> parameters(bool recurse = true) const

All parameter tensors (names dropped); recurse walks children.

Declared in ClikaRT/nn/module.h, line 225

buffers()

std::vector<Tensor> buffers(bool recurse = true) const

All buffer tensors (names dropped); recurse walks children.

Declared in ClikaRT/nn/module.h, line 228

state_dict()

NamedTensors state_dict(bool recurse = true) const

The module's checkpoint view: every named parameter and every PERSISTENT named buffer under its dotted name, the load_state_dict dual: a dict taken here loads back exactly, before or after a pack (a packed weight reads back from its pack as a fresh owning tensor; still-bound entries share storage with the module). A slot released after a fold (release_parameter / release_buffer) is not in the dict: its values live in the form that consumed them.

Declared in ClikaRT/nn/module.h, line 236

get_parameter()

Tensor get_parameter(std::string_view target) const

ONE parameter by its dotted name ("layers.0.weight"): the bound tensor (shares storage with the module) or, for a weight the layer has packed, a fresh owning read-back of that one slot from the pack.

Throws

ClikaRT::Error: when no parameter carries the name.

Declared in ClikaRT/nn/module.h, line 243

get_buffer()

Tensor get_buffer(std::string_view target) const

ONE buffer by its dotted name, persistent or not.

Throws

ClikaRT::Error: when no buffer carries the name.

Declared in ClikaRT/nn/module.h, line 247

parameter_spec()

FakeTensor parameter_spec(std::string_view target) const

The dtype, shape, placement and quantized flag of the parameter slot at dotted target, read with NO reconstruction: a packed weight answers with the geometry its carrier recorded and the dtype and quantized flag the weight had when it packed (never the carrier's own dtype, which is what a re-bind accepts), an unpacked one with its own metadata. The read a dtype probe wants after a load with pack_on_load (get_parameter reads the whole weight back for it).

Throws

ClikaRT::Error: when a submodule on the way or the slot does not exist.

Declared in ClikaRT/nn/module.h, line 258

to(StreamOrDevice)

void to(StreamOrDevice where)

Move the tree to a placement: a Device, or a Stream so the tree computes on that stream and lands its outputs there (the lane-placement call: build/load once, to(lane) once, forward every step). Children move first; a packed leaf (Linear / QLinearWoQ / MoE / QMoEWoQ) rebuilds its pack on the target, and a still-fake slot re-declares its metadata there.

Throws

ClikaRT::Error: when the tree cannot move as asked.

Declared in ClikaRT/nn/module.h, line 277

can_cast_to_impl()

virtual Result<void> can_cast_to_impl(DataType dtype) const

Whether THIS module's own slots can be cast to dtype (children are asked by the tree walk, never here). A failure is the refusal to(dtype) would raise; Module::to(dtype) asks every module of the tree BEFORE it casts any, so a refused cast leaves the tree untouched and names the leaf. Default: a dense module casts. Override in a leaf that pins its dtype.

Declared in ClikaRT/nn/module.h, line 284

to(DataType)

void to(DataType dtype)

Cast the tree's parameters/buffers to dtype (children first), ATOMICALLY: every module of the tree is asked can_cast_to_impl before any slot is cast, so a refusal (a quantized leaf, a shared weight) leaves the whole tree at its prior dtypes and names the refusing module. A dense packed leaf restores its raw weights, casts, and re-packs on the next forward; quantized leaves refuse dtype moves.

Throws

ClikaRT::Error: when the cast cannot be served (a quantized leaf, e.g.).

Declared in ClikaRT/nn/module.h, line 292

to(StreamOrDevice, DataType)

void to(StreamOrDevice where, DataType dtype)

Cast, then place: to(dtype) followed by to(where), in the order that re-packs ONCE. The cast restores a packed dense weight from its pack, casts it and drops the pack; the move then carries the unpacked slots to where, and the next forward (or a later load with pack_on_load) packs once, on the target.

Throws

ClikaRT::Error: when either half cannot be served (a quantized leaf refuses the cast).

Declared in ClikaRT/nn/module.h, line 300

load_state_dict(NamedTensors, LoadOptions)

void load_state_dict(NamedTensors checkpoint, LoadOptions options = {})

Bind every parameter/buffer whose dotted name is present in checkpoint (placed on the slot's declared device + dtype). With options.strict, raises ClikaRT::Error if any declared name is missing from the checkpoint or any checkpoint key is unexpected, minus the per-name allowances in options, the same completeness verdict the container overload below applies. The dict is taken by value: pass it with std::move for the one-copy load (this call drops its own handle before the pack_on_load pack runs, so no weight is held twice); an lvalue keeps the caller's own pin on every bound tensor until the caller drops it. Note: a bare braced second argument (load_state_dict(ckpt, {…})) is ambiguous against the strictness-only overload below; spell LoadOptions{…} (or a named local) explicitly.

Declared in ClikaRT/nn/module.h, line 317

load_state_dict(NamedTensors, bool)

void load_state_dict(NamedTensors checkpoint, bool strict)

Declared in ClikaRT/nn/module.h, line 323

load_state_dict(io::TensorsContainer, LoadOptions)

void load_state_dict(io::TensorsContainer& checkpoint, LoadOptions options = {})

Declared in ClikaRT/nn/module.h, line 339

output_shapes()

virtual std::vector<FakeTensor> output_shapes(ClikaRT::Span<const FakeTensor>) const

Shape rule: read input metadata (each input a FakeTensor: SymInt dims, dtype, placement, quantized flag) → output FakeTensors. Compose dimensions with SymInt / SymFloat; declare a data-dependent extent with SymInt::unknown(...). Outputs are placed on the op's stream (derived from the inputs), so the output FakeTensor's placement is informational. An exception thrown here surfaces as the dispatch call's error (a ClikaRT::Error keeps its Status; any other exception reports Status::Internal). Default: an error (a composite module never calls dispatch, so it need not override this).

Declared in ClikaRT/nn/module.h, line 351

compute()

virtual void compute(ClikaRT::Span<const Tensor>, ClikaRT::Span<Tensor>) const

Kernel: outputs arrive pre-allocated on the op's device (sharing storage). Read inputs and write outputs through the pointers that match where the kernel runs. device_const_data_ptr() / device_mutable_data_ptr() are for a kernel on the op's device: no host wait, and the pointer is ordered on the op's stream. const_data_ptr() / mutable_data_ptr() are for a plain host loop over CPU-resident tensors: they wait for the data and refuse a device tensor. The runtime binds the op's stream (read it with this->stream()) and makes its device current before calling this, so a GPU kernel launches on the right device.

This is the hand-written-kernel hook. compute() runs on whatever device its tensors live on; branch on this->stream().device() and launch directly:

Stream s = this->stream();                                     // the op's stream
auto cu = static_cast<cudaStream_t>(s.native_handle());        // the queue
const float* q = static_cast<const float*>(inputs[0].device_const_data_ptr());
float*       o = static_cast<float*>(outputs[0].device_mutable_data_ptr());
my_attention_kernel<<<grid, block, smem, cu>>>(q, ..., o);     // your kernel

Size the launch from get_device_properties(this->stream().device()) (SM count, shared-mem budget); take stream-ordered scratch from this->stream().allocate(shape, dtype). An exception thrown here does not escape: it surfaces as the operation's error at the caller's synchronization point (a ClikaRT::Error keeps its Status; any other exception reports Status::Internal), and the process keeps running. Default: an error.

Declared in ClikaRT/nn/module.h, line 386

stream()

Stream stream() const

The stream the runtime is currently dispatching this module on; valid ONLY inside compute() (and anything it calls). It is the queue a hand-written kernel launches on (stream().native_handle()) and the device the op runs on (stream().device()). Calling it outside a compute() invocation raises ClikaRT::Error.

Declared in ClikaRT/nn/module.h, line 399

dispatch()

std::vector<Tensor> dispatch(
ClikaRT::Span<const Tensor> inputs,
Device device = Device::cpu()
) const

Dispatch THIS module's (output_shapes, compute) through the runtime: eager runs compute; tracing records the op from output_shapes alone. device (default CPU) picks where a leaf with no tensor inputs runs.

Throws

ClikaRT::Error: when the dispatch is rejected (e.g. output_shapes reports an error).

Declared in ClikaRT/nn/module.h, line 407

Protected static member functions

initialize_child()

static Result<void> initialize_child(Module& child)

Runs one child's pack hook now, through the entry point that child captured at construction. A parent that consumed some of its children (their weights fused into one leaf it built) packs the leaves it keeps from its own initialize_impl override, one call per kept child, and skips the consumed ones; the base walk would reach a consumed child and refuse. Reachable from a derived class only: a load with LoadOptions::pack_on_load and a leaf's first forward remain the public routes to the pack.

Declared in ClikaRT/nn/module.h, line 442

Protected member functions

Module()

Module()

Constructs the module and captures the hook entry points compiled into the translation unit this constructor runs in (the unit that defines the most derived constructor). The runtime reaches every overridable hook (output_shapes, compute, to_impl, can_cast_to_impl, initialize_impl, apply_weight_residency) through those entry points, so an exception an override raises is caught by the same C++ runtime that raised it and arrives in the library as a status; no exception unwinds across the library boundary.

Declared in ClikaRT/nn/module.h, line 419

Module(ModuleHooks)

explicit Module(const ModuleHooks& hooks)

The constructor Module() delegates to, with the hook entry points.

Declared in ClikaRT/nn/module.h, line 421

initialize_impl()

virtual Result<void> initialize_impl()

The one-time weight pack, the hook LoadOptions::pack_on_load runs from the root and a packing leaf's first forward runs on itself. The base packs nothing of its own and recurses into its children (registration order), so one call on a root packs the whole tree; a leaf with a packable form overrides this with its pack (once, thread-safe, idempotent); a composite that fuses or folds at pack time overrides it, does that work, and calls the base for the children it did not consume. A weight slot still unbound (Tensor::is_fake()) is an error here.

Declared in ClikaRT/nn/module.h, line 432

apply_weight_residency()

virtual void apply_weight_residency(
    WeightResidency residency,
    std::uint64_t& stamped,
    std::uint64_t& class_pinned
)

The load-scope residency stamp (LoadOptions::weight_residency routes here before the checkpoint binds). The base recurses into children in registration order; a module that admits a residency choice overrides this to adopt residency (counting itself in stamped) or to keep a class-pinned posture (counting itself in class_pinned), and then calls the base so its own children are still walked. A subclass with no residency of its own needs no override.

Declared in ClikaRT/nn/module.h, line 451

Member functions​

~Module()​

Module(Module)​

operator=(Module)​

Module(Module)​

operator=(Module)​

register_parameter(string_view, Tensor)​

register_buffer(string_view, Tensor)​

register_parameter(string_view, FakeTensor)​

register_buffer(string_view, FakeTensor)​

register_buffer(string_view, Tensor, bool)​

register_buffer(string_view, FakeTensor, bool)​

register_module()​

release_parameter()​

release_buffer()​

named_parameters()​

named_buffers()​

named_children()​

parameters()​

buffers()​

state_dict()​

get_parameter()​

get_buffer()​

parameter_spec()​

to(StreamOrDevice)​

can_cast_to_impl()​

to(DataType)​

to(StreamOrDevice, DataType)​

load_state_dict(NamedTensors, LoadOptions)​

load_state_dict(NamedTensors, bool)​

load_state_dict(io::TensorsContainer, LoadOptions)​

output_shapes()​

compute()​

stream()​

dispatch()​

Protected static member functions​

initialize_child()​

Protected member functions​

Module()​

Module(ModuleHooks)​

initialize_impl()​

apply_weight_residency()​

Member functions

~Module()

Module(Module)

operator=(Module)

Module(Module)

operator=(Module)

register_parameter(string_view, Tensor)

register_buffer(string_view, Tensor)

register_parameter(string_view, FakeTensor)

register_buffer(string_view, FakeTensor)

register_buffer(string_view, Tensor, bool)

register_buffer(string_view, FakeTensor, bool)

register_module()

release_parameter()

release_buffer()

named_parameters()

named_buffers()

named_children()

parameters()

buffers()

state_dict()

get_parameter()

get_buffer()

parameter_spec()

to(StreamOrDevice)

can_cast_to_impl()

to(DataType)

to(StreamOrDevice, DataType)

load_state_dict(NamedTensors, LoadOptions)

load_state_dict(NamedTensors, bool)

load_state_dict(io::TensorsContainer, LoadOptions)

output_shapes()

compute()

stream()

dispatch()

Protected static member functions

initialize_child()

Protected member functions

Module()

Module(ModuleHooks)

initialize_impl()

apply_weight_residency()