ClikaRT::graph

namespace

Classes

Name	Description
CaptureOptions	How ClikaRT::compile captures. The zero-options default runs the standard graph-optimization pipeline and finalizes packed weights, with shape specialization gated behind specialize (the shape guard and the re-capture loop keep an unspecialized capture honest across shapes).
CompiledFunction	The self-capturing callable ClikaRT::compile returns. Move-only.
CompileOptions	How io::OnnxModel::compile builds the executable graph.
DynamicAxis	One axis declared dynamic ahead of capture, the zero-re-capture road for an axis known to vary call to call (a batch size, a sequence length). input selects by input name (a signature name, or the positional default "input_0", "input_1", …); empty applies to every input that has the axis. A non-empty name also names the dimension: the same name on two axes declares the same size, exactly as spec::TensorSpec::dim_names.
KVCacheLayerInfo	Per-attention-layer KV-cache descriptor: which graph inputs carry the layer's past keys/values in, which graph outputs carry the present keys/values out (any may be empty, a layer with no cache IO), plus the layer's attention properties. A serving loop drives its cache handling off these; generate_kv_input_specs derives compile-time shape specs from them.
ModelGraph	An executable model graph. Move-only.
TraceOptions	How a trace FINISHES the captured graph. The defaults reproduce the plain trace(...) behavior exactly, so existing calls are unaffected; each knob exists for a caller that drives the finishing itself (a compile step, a graph-tooling flow that wants the raw capture).

Enumerations

enum AttentionMaskKind

enum class AttentionMaskKind : std::uint8_t

The attention masking a layer applies; a sliding-window layer's cache is evictable past the window.

Enumerator	Description
Causal
SlidingWindowCausal

Declared in ClikaRT/graph/model_graph.h, line 37

Type aliases

using TraceFunction

using TraceFunction = std::function<std::vector<Tensor>(const std::vector<Tensor> &)>

A traceable model function: positional inputs → positional outputs. An nn::Module forward qualifies through a thin lambda.

Declared in ClikaRT/graph/trace.h, line 69

Functions

compile(TraceFunction, Span< spec::TensorSpec>, CaptureOptions)

CompiledFunction compile(
    TraceFunction fn,
    ClikaRT::Span<const spec::TensorSpec> signature = {},
    CaptureOptions options = {}
)

Wrap fn (any callable of the graph::trace shape: positional tensors in, positional tensors out) in a CompiledFunction. signature follows graph::trace's vocabulary: one spec::TensorSpec per input, dynamic dims as kDynamicDim, shareable by dim_names; empty derives every spec from the first call's tensors. Options whose values the captured road cannot honor raise a readable error here (see CaptureOptions); a malformed signature surfaces at the first capture instead, warned once, and the function keeps serving eagerly.

Declared in ClikaRT/graph/compile.h, line 223

compile(TraceFunction, CaptureOptions)

CompiledFunction compile(TraceFunction fn, CaptureOptions options)

Options-only form of compile; the signature derives from the first call.

Declared in ClikaRT/graph/compile.h, line 230

generate_kv_input_specs()

spec::InputSpecs generate_kv_input_specs(
    const ModelGraph& graph,
    const spec::KVSpecOptions& options = {}
)

Declared in ClikaRT/graph/model_graph.h, line 153

trace(TraceFunction, Span< spec::TensorSpec>, string_view, Span< string>, …)

ModelGraph trace(
    TraceFunction fn,
    ClikaRT::Span<const spec::TensorSpec> signature,
    std::string_view graph_name = "traced_model",
    ClikaRT::Span<const std::string> output_names = {},
    const TraceOptions& options = {}
)

Trace fn against a SIGNATURE: one spec::TensorSpec per input, the library's one IO-slot vocabulary: name, dtype, dims (dynamic dims as kDynamicDim, shareable by dim_names). The road for models whose feeds are a chore to hand-build (many inputs, KV caches): the stand-ins are synthesized from the specs. output_names (optional, positional) names the function's returned outputs: one name per output, or none for the "output_0", "output_1", … defaults; surfaced in-place state outputs name themselves (see the authoring convention above) and are appended after. options drives how the capture is finished: optimization, device placement, serving finalization (see TraceOptions; the defaults are the full standard finishing).

Declared in ClikaRT/graph/trace.h, line 185

trace(TraceFunction, vector<Tensor>, Span< string>, string_view, …)

ModelGraph trace(
    TraceFunction fn,
    const std::vector<Tensor>& tracing_inputs,
    ClikaRT::Span<const std::string> input_names = {},
    std::string_view graph_name = "traced_model",
    ClikaRT::Span<const std::string> output_names = {},
    const TraceOptions& options = {}
)

Trace fn against EXAMPLE TENSORS: their shapes and dtypes matter, their values do not (the trace runs on data-free stand-ins mirroring them). A thin reduction to the signature road: each example becomes its spec (spec::TensorSpec::from_tensors), so each input's shape traces as-is and the graph is specialized to it; declare dynamic dims through the signature road instead. input_names (optional, positional) names the graph inputs; absent names default to "input_0", "input_1", …. output_names names the returned outputs the same way (defaults "output_0", …); surfaced in-place state outputs are appended after. options drives the finishing, exactly as on the signature road.

Declared in ClikaRT/graph/trace.h, line 204

ClikaRT/graph/compile.h

#include <ClikaRT/graph/compile.h>

ClikaRT::compile: wrap an eager model function in a self-capturing CompiledFunction: the first call serves the eager result and records the function into an executable ModelGraph; later calls whose input shapes match the capture run the graph instead. The wrapped function behaves EXACTLY like calling the function directly (same values, same failures) in every state; the graph is a serving substitution, never a semantic one.

The serving contract (fail-soft)

A CompiledFunction is a small state machine:

• Pending (fresh / after reset): a call serves the eager result, then captures synchronously, the same recording graph::trace performs, against the declared signature widened by the shapes the call actually presented. A capture failure is warned once and the function latches Fallback; the call's result is unaffected.
• Compiled: a call first passes a cheap input-shape guard (dtype, rank, and every concrete captured axis must match; a dynamic captured axis matches any extent). On a match the captured graph serves the call. On drift the eager result serves, the drifted axes widen to dynamic, and the function re-captures; shapes only ever widen, so the loop converges; past kMaxRecaptures re-captures it latches Fallback instead. A graph run that fails is warned once, the call is served eagerly, and the function latches Fallback.
• Fallback: every call serves eagerly. The function never re-arms itself; reset() is the explicit re-arm.

There is no staleness detection: a function whose captured behavior depends on external state (a C++ flag, a mutated closure tensor) keeps serving the captured behavior until an explicit reset() re-captures.

Calls are serialized: one call runs at a time per CompiledFunction (a concurrent caller waits). For concurrent serving, take_graph() the captured ModelGraph and share that.

Frictionless by default

auto step = ClikaRT::compile([&](const std::vector<Tensor>& in) {
    return std::vector<Tensor>{ops::relu(in[0])};
});
auto y0 = step({x});   // eager + capture
auto y1 = step({x});   // served by the captured graph

With no signature, the capture derives one spec per input from the first call's tensors (shapes and dtypes; values are never read). Passing a signature (spec::TensorSpec, the same vocabulary graph::trace takes) names the inputs and declares dynamic dims up front; a declared spec that disagrees with an observed extent widens that axis to dynamic.

Every fallible operation returns its value directly and raises ClikaRT::Error on failure; wrap in CLIKART_TRY(...) to inspect a Result instead.

ClikaRT/graph/model_graph.h

#include <ClikaRT/graph/model_graph.h>

ModelGraph: an executable model graph with named, typed inputs and outputs, the operator graph between them, and the model's weights, ready to run.

A ModelGraph is the GENERIC executable form, not tied to any one source. Today it is produced by io::OnnxModel::compile (an ONNX model made executable) and by ClikaRT::trace (an eager model function recorded into a graph); future tooling that optimizes or quantizes a model operates on this same type.

Every fallible operation returns its value directly and raises ClikaRT::Error on failure. Wrap a call in CLIKART_TRY(...) to inspect a Result instead of catching.