Skip to main content

ClikaRT::runtime::Model

class

Header: ClikaRT/runtime/model.h
Inherited by: ClikaRT::runtime::FunctionModel

The serving-model contract: subclass this to run under Executor / Pipeline.

A model declares its I/O (schema), its phase list (phases), and implements the phase bodies (Phase_RunOnce for one-shot phases, Phase_Iterate for continuous-batched stepping); stateful models allocate per-session state in make_state and reclaim it in evict. A model that reports its capacity (capacity_units, project_units) lets the scheduler admit against it (SchedulerConfig::capacity_policy). Overrides MAY raise ClikaRT::Error; the runtime catches caller-side and fails only that session.

Member functions​

~Model()​

virtual ~Model()

Virtual base destructor.

Declared in ClikaRT/runtime/model.h, line 49

schema()​

virtual const ModelSchema& schema() const =0

The model's I/O contract. Return a reference to model-owned storage.

Declared in ClikaRT/runtime/model.h, line 52

phases()​

virtual ClikaRT::Span<const PhaseSpec> phases() const =0

The model's phases, in order. Borrowed (model-owned storage).

Declared in ClikaRT/runtime/model.h, line 54

make_state()​

virtual ModelState* make_state(Stream stream) const

Allocate per-session state (stateful models only). Return a heap-allocated ModelState subclass (the runtime takes ownership and frees it after evict); the default returns nullptr (stateless). MAY raise on failure.

Declared in ClikaRT/runtime/model.h, line 59

Phase_RunOnce()​

virtual void Phase_RunOnce(const PhaseSpec& phase, PhaseContext& ctx) =0

Run a RunOnce phase: read ctx.inputs, write ctx.outputs. MAY raise.

Declared in ClikaRT/runtime/model.h, line 62

Phase_Iterate()​

virtual StepBatchOut Phase_Iterate(const PhaseSpec&, StepContext&)

Step an Iterate phase over the batch in ctx.slots; return one SessionStepResult per slot (in order). The default raises Unsupported (a RunOnce-only model need not override it). MAY raise.

Declared in ClikaRT/runtime/model.h, line 67

evict()​

virtual void evict(ModelState& state) noexcept

Reclaim a session's state before the runtime destroys it (free slots, caches). Default: no-op. Must not throw.

Declared in ClikaRT/runtime/model.h, line 77

capacity_units()​

virtual std::int64_t capacity_units() const noexcept

The capacity this model has free right now, in its own units. The default, -1, reports no capacity: every capacity policy then admits under SchedulerConfig::max_active_sessions alone, which is also all that CapacityPolicy::SlotCap ever does. Read once per admission round. Must not throw.

Declared in ClikaRT/runtime/model.h, line 89

project_units()​

virtual UnitProjection project_units(const NamedTensors& inputs) const noexcept

A request's need in the same units, read from its inputs before it is admitted (and again before a session set aside with Disposition::Preempted is re-admitted, its resume payload merged in): total through the output cap, first_step for what the admitting step commits (the prompt, or its first chunk). The default reports both unknown (-1), and the request is admitted under the session cap alone. Must not throw.

Declared in ClikaRT/runtime/model.h, line 98