ClikaRT::graph

namespace

Classes

Name	Description
Attr	One operator attribute. name is the configuration setting ("dim", "activation", "padding_template"). An enumerated selector carries its readable value in the std::string arm ("Reflect") and the selector's type in enum_type ("IPadMode"); enum_type is empty for every other attribute.
BenchOptions	The knobs of one ModelGraph::bench call. Every field has a default.
BenchReport	One benchmark's measurement.
BoundTensor	One tensor bound into an operator (a traced module's weight; a compiled ONNX graph's initializer, which is also a constant input). name is the operator's own slot for it ("b": a matmul's second operand; "bias"); tensor is a shared handle to the bound bytes, no copy.
CaptureOptions	How ClikaRT::compile captures. The zero-options default runs the standard graph-optimization pipeline and finalizes packed weights, with shape specialization gated behind specialize (the shape guard and the re-capture loop keep an unspecialized capture honest across shapes).
CompiledFunction	The self-capturing callable ClikaRT::compile returns. Move-only.
CompileOptions	How io::OnnxModel::compile builds the executable graph.
DimDomain	The extents one input dimension admits.
DynamicAxis	One axis declared dynamic ahead of capture, the zero-re-capture road for an axis known to vary call to call (a batch size, a sequence length). input selects by input name (a signature name, or the positional default "input_0", "input_1", …); empty applies to every input that has the axis. A non-empty name also names the dimension: the same name on two axes declares the same size, exactly as spec::TensorSpec::dim_names.
InputShape	A concrete shape for one named input.
KVCacheLayerInfo	Per-attention-layer KV-cache descriptor: which graph inputs carry the layer's past keys/values in, which graph outputs carry the present keys/values out (any may be empty, a layer with no cache IO), plus the layer's attention properties. A serving loop drives its cache handling off these; generate_kv_input_specs derives compile-time shape specs from them.
Match	One occurrence of a Pattern in a graph: the matched node per pattern node, in add_node order. An optional pattern node absent from this occurrence reads as std::nullopt; every other entry holds a node. The nodes are borrowed views (node.h): valid while the searched graph is alive and unchanged.
ModelGraph	An executable model graph. Move-only.
Node	One node of a compiled graph: an operator, or a graph input.
Pattern	A graph of operator constraints to find in a ModelGraph. Move-only.
ShapeDomain	One input's admitted shapes: its name, its element type, one DimDomain per dimension, and the axis a benchmark counts items along.
TraceOptions	How a trace FINISHES the captured graph. The defaults reproduce the plain trace(...) behavior exactly, so existing calls are unaffected; each knob exists for a caller that drives the finishing itself (a compile step, a graph-tooling flow that wants the raw capture).
Value	A tensor on a graph edge: an operator's output on one port, a graph input, or a constant (a weight bound into the graph). One value per (producer, port): the same tensor consumed by two operators is one value.

Enumerations

enum FeedFill

enum class FeedFill : std::int32_t

How a benchmark fills the input tensors it synthesizes.

Enumerator	Description
Zeros	every element is zero
Uniform	Uniform random values in [0, 1) from a fixed seed, so two runs feed the same bytes. Applies to floating-point inputs; integer and bool inputs (token ids, masks) are zero-filled.

Declared in ClikaRT/graph/bench.h, line 84

enum AttentionMaskKind

enum class AttentionMaskKind : std::uint8_t

The attention masking a layer applies; a sliding-window layer's cache is evictable past the window.

Enumerator	Description
Causal
SlidingWindowCausal

Declared in ClikaRT/graph/model_graph.h, line 45

enum NodeKind

enum class NodeKind : std::uint8_t

What a node is.

Enumerator	Description
Op	An operator; Node::op_code() names it.
Input	A graph input: the named entry point a caller's tensor binds to at run. Its one output value is that input; op_code() is OpCode::Placeholder.

Declared in ClikaRT/graph/node.h, line 44

enum OpCode

enum class OpCode : std::int32_t

The operator vocabulary. Values are unspecified (see the file comment): compare and persist names.

Enumerator	Description
Unknown

Declared in ClikaRT/graph/op_code.h, line 36

Type aliases

using AttrValue

using AttrValue = std::variant<std::monostate, bool, std::int64_t, double, std::string, std::vector<std::int64_t>, std::vector<double>, std::vector<std::optional<std::int64_t>>, std::vector<std::optional<double>>, DataType, Scalar>

One attribute value. std::monostate means "not statically known": an unset optional setting, or a value the operator reads from a runtime operand instead of its configuration. A shape template keeps its per-slot knowledge: std::vector<std::optional<std::int64_t>> (and the double twin) holds the literal of every static slot and std::nullopt for every slot fed at run time.

Declared in ClikaRT/graph/node.h, line 59

using NodePredicate

using NodePredicate = std::function<bool(const Node &)>

A test a graph node must pass to match a pattern node: true accepts it. It runs for every candidate node the search visits.

Declared in ClikaRT/graph/pattern.h, line 59

using TraceFunction

using TraceFunction = std::function<std::vector<Tensor>(const std::vector<Tensor> &)>

A traceable model function: positional inputs → positional outputs. An nn::Module forward qualifies through a thin lambda.

Declared in ClikaRT/graph/trace.h, line 69

Functions

to_table(BenchReport)

tables::Table to_table(const BenchReport& report)

One row for the report, the shapes and outputs rendered as name:d0xd1x... strings (entries joined with ;), the device as API:index, the fill as zeros / uniform. The columns, in order: shapes, outputs, device, fill, warmup, iterations, threads, outputs_reused, outputs_on_device, first_call_ms, p50_ms, p95_ms, p99_ms, mean_ms, min_ms, max_ms, stddev_ms, batch_axis, items_per_iteration, throughput_items_per_s, peak_rss_bytes, device_memory_valid, device_active_bytes, device_reserved_bytes, device_peak_active_bytes, device_free_bytes, device_total_bytes. The same columns as the multi-report form, so a sweep's table appends row by row.

Declared in ClikaRT/graph/bench.h, line 171

to_table(vector<BenchReport>)

tables::Table to_table(const std::vector<BenchReport>& reports)

One row per report, in order; the columns of the single-report form.

Declared in ClikaRT/graph/bench.h, line 175

iterations_table()

tables::Table iterations_table(const BenchReport& report)

One row per measured run, columns iteration (0-based) and ms.

Declared in ClikaRT/graph/bench.h, line 179

to_json()

json::Json to_json(const BenchReport& report)

Every field of the report as a JSON object under the table's column names: the shapes and outputs as arrays of {name, dims}, iteration_ms as an array, the device as API:index, the memory counters as the nested device_memory object with device::MemoryStats's field names.

Declared in ClikaRT/graph/bench.h, line 186

compile(TraceFunction, Span< spec::TensorSpec>, CaptureOptions)

CompiledFunction compile(
    TraceFunction fn,
    ClikaRT::Span<const spec::TensorSpec> signature = {},
    CaptureOptions options = {}
)

Wrap fn (any callable of the graph::trace shape: positional tensors in, positional tensors out) in a CompiledFunction. signature follows graph::trace's vocabulary: one spec::TensorSpec per input, dynamic dims as kDynamicDim, shareable by dim_names; empty derives every spec from the first call's tensors. Options whose values the captured road cannot honor raise a readable error here (see CaptureOptions); a malformed signature surfaces at the first capture instead, warned once, and the function keeps serving eagerly.

Declared in ClikaRT/graph/compile.h, line 223

compile(TraceFunction, CaptureOptions)

CompiledFunction compile(TraceFunction fn, CaptureOptions options)

Options-only form of compile; the signature derives from the first call.

Declared in ClikaRT/graph/compile.h, line 230

generate_kv_input_specs()

spec::InputSpecs generate_kv_input_specs(
    const ModelGraph& graph,
    const spec::KVSpecOptions& options = {}
)

Declared in ClikaRT/graph/model_graph.h, line 375

op_code_count()

constexpr std::size_t op_code_count() noexcept

The number of operators in the vocabulary (Unknown not counted).

Declared in ClikaRT/graph/op_code.h, line 44

op_code_name()

constexpr std::string_view op_code_name(OpCode code) noexcept

The enumerator's name: "MatMul", "Custom", "Reserved0"; "Unknown" for OpCode::Unknown and for any value that names no operator.

Declared in ClikaRT/graph/op_code.h, line 51

parse_op_code()

constexpr std::optional<OpCode> parse_op_code(std::string_view name) noexcept

The enumerator spelled name (exact, case-sensitive): OpCode::Unknown for "Unknown", std::nullopt when no operator has that name.

Declared in ClikaRT/graph/op_code.h, line 66

trace(TraceFunction, Span< spec::TensorSpec>, string_view, Span< string>, …)

ModelGraph trace(
    TraceFunction fn,
    ClikaRT::Span<const spec::TensorSpec> signature,
    std::string_view graph_name = "traced_model",
    ClikaRT::Span<const std::string> output_names = {},
    const TraceOptions& options = {}
)

Trace fn against a SIGNATURE: one spec::TensorSpec per input, the library's one IO-slot vocabulary: name, dtype, dims (dynamic dims as kDynamicDim, shareable by dim_names). The road for models whose feeds are a chore to hand-build (many inputs, KV caches): the stand-ins are synthesized from the specs. output_names (optional, positional) names the function's returned outputs: one name per output, or none for the "output_0", "output_1", … defaults; surfaced in-place state outputs name themselves (see the authoring convention above) and are appended after. options drives how the capture is finished: optimization, device placement, serving finalization (see TraceOptions; the defaults are the full standard finishing).

Declared in ClikaRT/graph/trace.h, line 185

trace(TraceFunction, vector<Tensor>, Span< string>, string_view, …)

ModelGraph trace(
    TraceFunction fn,
    const std::vector<Tensor>& tracing_inputs,
    ClikaRT::Span<const std::string> input_names = {},
    std::string_view graph_name = "traced_model",
    ClikaRT::Span<const std::string> output_names = {},
    const TraceOptions& options = {}
)

Trace fn against EXAMPLE TENSORS: their shapes and dtypes matter, their values do not (the trace runs on data-free stand-ins mirroring them). A thin reduction to the signature road: each example becomes its spec (spec::TensorSpec::from_tensors), so each input's shape traces as-is and the graph is specialized to it; declare dynamic dims through the signature road instead. input_names (optional, positional) names the graph inputs; absent names default to "input_0", "input_1", …. output_names names the returned outputs the same way (defaults "output_0", …); surfaced in-place state outputs are appended after. options drives the finishing, exactly as on the signature road.

Declared in ClikaRT/graph/trace.h, line 204

ClikaRT/graph/bench.h

#include <ClikaRT/graph/bench.h>

The vocabulary of ModelGraph::bench and the shape checks beside it: the extents a compiled graph admits per input (ShapeDomain, DimDomain), the shapes a benchmark feeds (InputShape), its knobs (BenchOptions), its measurement (BenchReport), and the table / JSON renderings of a report.

Every fallible operation returns its value directly and raises ClikaRT::Error on failure. Wrap a call in CLIKART_TRY(...) to inspect a Result instead of catching.

ClikaRT/graph/compile.h

#include <ClikaRT/graph/compile.h>

ClikaRT::compile: wrap an eager model function in a self-capturing CompiledFunction: the first call serves the eager result and records the function into an executable ModelGraph; later calls whose input shapes match the capture run the graph instead. The wrapped function behaves EXACTLY like calling the function directly (same values, same failures) in every state; the graph is a serving substitution, never a semantic one.

The serving contract (fail-soft)

A CompiledFunction is a small state machine:

• Pending (fresh / after reset): a call serves the eager result, then captures synchronously, the same recording graph::trace performs, against the declared signature widened by the shapes the call actually presented. A capture failure is warned once and the function latches Fallback; the call's result is unaffected.
• Compiled: a call first passes a cheap input-shape guard (dtype, rank, and every concrete captured axis must match; a dynamic captured axis matches any extent). On a match the captured graph serves the call. On drift the eager result serves, the drifted axes widen to dynamic, and the function re-captures; shapes only ever widen, so the loop converges; past kMaxRecaptures re-captures it latches Fallback instead. A graph run that fails is warned once, the call is served eagerly, and the function latches Fallback.
• Fallback: every call serves eagerly. The function never re-arms itself; reset() is the explicit re-arm.

There is no staleness detection: a function whose captured behavior depends on external state (a C++ flag, a mutated closure tensor) keeps serving the captured behavior until an explicit reset() re-captures.

Calls are serialized: one call runs at a time per CompiledFunction (a concurrent caller waits). For concurrent serving, take_graph() the captured ModelGraph and share that.

Frictionless by default

auto step = ClikaRT::compile([&](const std::vector<Tensor>& in) {
    return std::vector<Tensor>{ops::relu(in[0])};
});
auto y0 = step({x});   // eager + capture
auto y1 = step({x});   // served by the captured graph

With no signature, the capture derives one spec per input from the first call's tensors (shapes and dtypes; values are never read). Passing a signature (spec::TensorSpec, the same vocabulary graph::trace takes) names the inputs and declares dynamic dims up front; a declared spec that disagrees with an observed extent widens that axis to dynamic.

Every fallible operation returns its value directly and raises ClikaRT::Error on failure; wrap in CLIKART_TRY(...) to inspect a Result instead.

ClikaRT/graph/model_graph.h

#include <ClikaRT/graph/model_graph.h>

ModelGraph: an executable model graph with named, typed inputs and outputs, the operator graph between them, and the model's weights, ready to run.

A ModelGraph is the GENERIC executable form, not tied to any one source. Today it is produced by io::OnnxModel::compile (an ONNX model made executable) and by ClikaRT::trace (an eager model function recorded into a graph); future tooling that optimizes or quantizes a model operates on this same type.

Every fallible operation returns its value directly and raises ClikaRT::Error on failure. Wrap a call in CLIKART_TRY(...) to inspect a Result instead of catching.

ClikaRT/graph/node.h

#include <ClikaRT/graph/node.h>

Node and Value: read-only views over a ModelGraph's operators and the tensors that flow between them, for graph tooling (an exporter, a quantizer, an audit) that needs the structure of a compiled graph and each operator's configuration.

Both are BORROWED views, like std::string_view over a string: cheap to copy, valid while the ModelGraph that produced them is alive and unchanged, and invalid once that graph is destroyed, moved from, or finalized again. Obtain them from ModelGraph::nodes(), node(name), find_nodes(code) and constants(), and walk from there through inputs() / outputs() / producer() / predecessors() / successors(); bound_tensors() reads the weights a traced module bound into an operator. Every accessor is const; the fallible ones return a Result instead of raising.

ClikaRT/graph/op_code.h

#include <ClikaRT/graph/op_code.h>

OpCode: the operator vocabulary of a compiled graph. Every operator the runtime executes has one enumerator; graph::Node::op_code() reports it and ModelGraph::find_nodes(OpCode) selects by it.

The vocabulary is part of the public API: every name here is stable and readable in every build, fused and runtime-only operators included, and a name a program persists keeps its meaning across releases. The numeric values are NOT part of the contract: they follow the runtime's operator table and may change between releases. Persist and compare NAMES (op_code_name, parse_op_code), never integers.

Three kinds of enumerator carry a special meaning:

• Custom is every user-defined operator (an nn::Module that overrides output_shapes and compute); the operator's own name is Node::custom_name().
• Placeholder is a graph input: an Input node whose single output is the tensor a caller binds at run.
• Reserved<N> is a slot held for a future operator; no node reports one. Unknown is the sentinel for a value that names no operator.

Macros

#define CLIKART_DETAIL_OP_CODE_ENUMERATOR

#define CLIKART_DETAIL_OP_CODE_ENUMERATOR(name) name,

Declared in ClikaRT/graph/op_code.h, line 33

#define CLIKART_DETAIL_OP_CODE_COUNT_ONE

#define CLIKART_DETAIL_OP_CODE_COUNT_ONE(name) +1

Declared in ClikaRT/graph/op_code.h, line 42

#define CLIKART_DETAIL_OP_CODE_NAME

#define CLIKART_DETAIL_OP_CODE_NAME(name) case OpCode::name: \ return #name;

Declared in ClikaRT/graph/op_code.h, line 53

#define CLIKART_DETAIL_OP_CODE_PARSE

#define CLIKART_DETAIL_OP_CODE_PARSE(row) if (name == #row) return OpCode::row;

Declared in ClikaRT/graph/op_code.h, line 67

ClikaRT/graph/pattern.h

#include <ClikaRT/graph/pattern.h>

Pattern: a small graph of operator constraints that ModelGraph::find_pattern locates inside a compiled graph. Read-only: every occurrence comes back as the matched Nodes, one per pattern node, for an audit that counts unfused pairs, an exporter deciding how a group of operators lowers, or a quantizer choosing where a fused kernel lands.

Build a pattern once and match it against any number of graphs. add_node declares what a graph node must be: one operator, one of several, or a predicate you write over the Node view. add_edge declares which pattern node feeds which, on any port or on the ports you name. The first node added is the root the search starts from. Pattern::chain builds the common straight line (MatMul -> Relu) in one call.

Matching rules: occurrences never overlap (a graph node belongs to at most one occurrence per search: occurrences are claimed in match order, the richest reading of an optional node first, and a later occurrence that shares a graph node with an earlier one is dropped); a matched node may have inputs and consumers the pattern does not name (a MatMul that also feeds a second reader is still a MatMul -> Relu occurrence); an optional pattern node may be absent from an occurrence, and then its Match entry is std::nullopt and its predecessors are read as feeding its successors (the search adds that bypass itself; a pattern spells only the full line).