Skip to main content

ClikaRT::graph

namespace

Namespaces​

NameDescription
ClikaRT::graph::transformsOne factory per built-in transform, named as the transform is: transforms::remove_double_permute() returns the handle whose name() is "remove_double_permute".

Classes​

NameDescription
AttrOne operator attribute. name is the configuration setting ("dim", "activation", "pad"). An enumerated selector carries its readable value in the std::string arm ("Reflect") and the selector's type in enum_type ("IPadMode"); enum_type is empty for every other attribute.
BenchOptionsThe knobs of one ModelGraph::bench call. Every field has a default.
BenchReportOne benchmark's measurement.
BoundTensorOne tensor bound into an operator (a traced module's weight; a compiled ONNX graph's initializer, which is also a constant input). name is the operator's own slot for it ("b": a matmul's second operand; "bias"); tensor is a shared handle to the bound bytes, no copy.
CaptureOptionsHow ClikaRT::compile captures. The zero-options default runs the standard graph-optimization pipeline and finalizes packed weights, with shape specialization gated behind specialize (the shape guard and the re-capture loop keep an unspecialized capture honest across shapes).
CompiledFunctionThe self-capturing callable ClikaRT::compile returns. Move-only.
CompileOptionsHow io::OnnxModel::compile builds the graph. The compile returns the graph as built: optimize() runs the graph optimizer when wanted (its DefaultsOptions hold the specialization and channels-last choices), and finalize() places the graph on the weights device and readies it to run.
ConnectionOne output of a part feeding one input of another: from names the output, to the input.
ContainedPathOptionsPathOptions with its edge filter contained. You never build this; the path queries (ModelGraph::has_path, shortest_path, all_simple_paths and the rest) build one from your PathOptions, containing its filter in your translation unit.
ContainedWalkOptionsWalkOptions with its callables contained. You never build this; the walks (ModelGraph::walk, visit, ancestors, descendants) build one from your WalkOptions, containing each callable in your translation unit.
DefaultsOptionsChoices the optimizer's transforms read. Each default keeps the graph's input and output contract and serves every shape the graph admits.
DimDomainThe extents one input dimension admits.
DynamicAxisOne axis declared dynamic ahead of capture, the zero-re-capture road for an axis known to vary call to call (a batch size, a sequence length). input selects by input name (a signature name, or the positional default "input_0", "input_1", …); empty applies to every input that has the axis. A non-empty name also names the dimension: the same name on two axes declares the same size, exactly as spec::TensorSpec::dim_names.
EdgeOne read of a value: the value src() produces on output port src_port(), read by dst() on input port dst_port(); Mul(x, x) reads x over two edges. Edges join nodes only: a constant or a state operand is a Value (Node::inputs(), Node::input(port)), not an edge, and returning a value from the graph adds none. The view finds its edge again by its two nodes and ports after an edit; once no edge has them, the view answers empty and check() says so.
GraphPartOne graph to merge, and the name the merged graph knows it by. The name prefixes the graph's operators in the merged graph, and its inputs and outputs where MergeOptions::io_names calls for it; it is not empty and holds no / and no ..
InputShapeA concrete shape for one named input.
KVCacheLayerInfoPer-attention-layer KV-cache descriptor: which graph inputs carry the layer's past keys/values in, which graph outputs carry the present keys/values out (any may be empty, a layer with no cache IO), plus the layer's attention properties. A serving loop drives its cache handling off these; generate_kv_input_specs derives compile-time shape specs from them.
MatchOne occurrence of a Pattern in a graph: the matched node per pattern node, in add_node order. An optional pattern node absent from this occurrence reads as std::nullopt; every other entry holds a node. The nodes are views (node.h): they find their nodes again by name after an edit, and once the searched graph is gone they refuse as views whose graph is gone.
MergedGraphA merged graph, and every name the merge changed (Rename, model_graph.h).
MergeOptionsHow merge names and places what it joins.
ModelGraphAn executable model graph. Move-only.
NodeOne node of a compiled graph: an operator, or a graph input. The view finds its node again by name after an edit; once no node has that name (it was removed or renamed), the view answers empty and check() says so.
NodeMapOne T per node (a weight, a score, a depth), iterating in the order the entries were added. Keyed by Node views, which compare by name, so an entry still finds its node after the graph is edited.
OnnxExportHandlerA handler for one operator, in one export (OnnxExportOptions::handlers): it replaces the library's ONNX form for the nodes of code. The operators it writes are its own choice: the export writes them as they are, and io::OnnxModel::compile refuses one this library does not read. Scope: the library's own operators; a handler for OpCode::Custom, the one code every user-defined operator shares, is refused.
OnnxExportOptionsHow ModelGraph::export_onnx and io::OnnxModel::from_graph write a graph as an ONNX model.
OnnxExportWriterWrites one node's ONNX form, for the handler the export calls with it. It is valid for that call only: a copy kept past it refuses every call (FAILED_PRECONDITION, naming the node).
OptimizeOptionsHow ModelGraph::optimize runs.
OptimizeReportWhat one ModelGraph::optimize run did.
PathOptionsThe bounds and filters of a path query (ModelGraph::find_paths_to, has_path, shortest_path, all_simple_paths, all_simple_edge_paths, count_paths). A path never repeats a node.
PatternA graph of operator constraints to find in a ModelGraph. Move-only.
PortRefAn input or an output of a part: the part's name, and the input's or the output's own name in that part.
RenameOne name a merge changed (graph::merge, compose.h): in part part, the node, input or output named from is named to in the merged graph.
RewriteMatchOne occurrence a rule rewrites, read on the graph as the rule's earlier replacements in the same iteration left it.
ShapeDomainOne input's admitted shapes: its name, its element type, one DimDomain per dimension, and the axis a benchmark counts items along.
SplitGraphA graph cut into parts, in an order every connection runs forward in, and the connections that join them. merge(parts, connections) gives back a graph with the source's inputs and outputs, by name, and its operators named <part>/<name>.
TransformA graph transform: one of ours, from its factory in transforms, or one you write (from_function). Cheap to copy.
TransformReportWhat one transform in the list did during an optimize run.
ValueA tensor on a graph edge: an operator's output on one port, a graph input, or a constant (a weight bound into the graph). One value per (producer, port): the same tensor consumed by two operators is one value, and every view of it compares equal and hashes alike (std::hash<Value>), whichever node listed it. The view finds its value again by that key after an edit; once no value has it, the view answers empty and check() says so.
ValueCutA cut through a graph's values (ModelGraph::min_value_cut): the values the nodes before the cut make and the nodes after it read, a graph output made before the cut included, in nodes() order of their producers, then port. A view and the tensor it re-describes share their bytes, so both may be listed while bytes counts them once.
WalkOptionsThe bounds and filters of a walk. A walk is breadth-first from its start: each node is reached once, at its fewest edges from the start.
WeightedPathA path with the edges it takes and its summed cost: edges[i] joins nodes[i] to nodes[i + 1], naming the parallel edge taken. Empty, at cost 0, when no path runs.

Enumerations​

enum FeedFill​

enum class FeedFill : std::int32_t

How a benchmark fills the input tensors it synthesizes.

EnumeratorDescription
Zerosevery element is zero
UniformUniform random values in [0, 1) from a fixed seed, so two runs feed the same bytes. Applies to floating-point inputs; integer and bool inputs (token ids, masks) are zero-filled.

Declared in ClikaRT/graph/bench.h, line 84

enum IoNames​

enum class IoNames : std::uint8_t

How the merged graph names the inputs and outputs no connection joins.

EnumeratorDescription
PrefixOnCollisionA name two parts both use becomes <part>/<name> in every part that uses it; every other name stays as it is.
PrefixAlwaysEvery name becomes <part>/<name>.
KeepEvery name stays as it is, and a name two parts both use refuses the merge.

Declared in ClikaRT/graph/compose.h, line 54

enum CrossDevice​

enum class CrossDevice : std::uint8_t

What a connection does when its output and its input sit on two devices.

EnumeratorDescription
MoveThe merged graph moves the value onto the input's device.
RefuseThe merge refuses, naming the connection.

Declared in ClikaRT/graph/compose.h, line 66

enum AttentionMaskKind​

enum class AttentionMaskKind : std::uint8_t

The attention masking a layer applies; a sliding-window layer's cache is evictable past the window.

EnumeratorDescription
Causal
SlidingWindowCausal

Declared in ClikaRT/graph/model_graph.h, line 55

enum NameKind​

enum class NameKind : std::uint8_t

What a changed name names.

EnumeratorDescription
Node
Input
Output

Declared in ClikaRT/graph/model_graph.h, line 87

enum NodeKind​

enum class NodeKind : std::uint8_t

What a node is.

EnumeratorDescription
OpAn operator; Node::op_code() names it.
InputA graph input: the named entry point a caller's tensor binds to at run. Its one output value is that input; op_code() is OpCode::Placeholder.

Declared in ClikaRT/graph/node.h, line 58

enum OpCode​

enum class OpCode : std::int32_t

The operator vocabulary. Values are unspecified (see the file comment): compare and persist names.

EnumeratorDescription
Unknown

Declared in ClikaRT/graph/op_code.h, line 36

enum NameMatch​

enum class NameMatch : std::uint8_t

How a name pattern must match a node's name.

EnumeratorDescription
Wholethe pattern matches the whole name
Anywherethe pattern matches some part of the name

Declared in ClikaRT/graph/query.h, line 29

enum WalkDirection​

enum class WalkDirection : std::uint8_t

Which way a walk crosses edges.

EnumeratorDescription
Forwardto the nodes that read a node's outputs (Node::out_edges())
Backwardto the nodes that produce a node's inputs (Node::in_edges())

Declared in ClikaRT/graph/query.h, line 35

enum VisitAction​

enum class VisitAction : std::uint8_t

A visitor's answer for the node it was shown.

EnumeratorDescription
Continuego on past the node
Prunedo not go past the node: nothing is reached through it
Stopend the walk

Declared in ClikaRT/graph/query.h, line 41

enum TopologicalOrder​

enum class TopologicalOrder : std::uint8_t

Which topological order ModelGraph::topological_order returns. Every kind lists each node once, after every node whose output it reads; the kinds differ in how they order the nodes that could go either way.

EnumeratorDescription
DeterministicThe Input nodes first, in input order; ties among the rest go to the node the graph created first. A trace creates its operators as it records them from the outputs, each after its producers in input-port order, so a program read in the order it was written lists as written, and an edit keeps the untouched nodes in place: the order nodes() lists.
StructuralRead off the structure alone: from the graph outputs in output order, back through each node's inputs in input-port order, so renaming nodes never changes it. The nodes no graph output depends on come last, in the Deterministic order.
MinMemoryAn order that keeps fewer bytes alive at its peak, counting each value as freed after its last reader: the lowest peak a bounded search over valid orders finds. The Deterministic order when the search finds none lower or declines the graph.

Declared in ClikaRT/graph/query.h, line 50

enum TransformKind​

enum class TransformKind : std::int32_t

Which transform a handle names: one enumerator per built-in transform, and User for a transform the program defines. Values are unspecified (see the file comment): compare and persist names.

EnumeratorDescription
User

Declared in ClikaRT/graph/transform.h, line 50

Type aliases​

using PartitionFunction​

using PartitionFunction = std::function<std::string(const Node &)>

The name of the part a node goes to: not empty, and holding no / and no .. split asks it once for each operator, in nodes() order, on the thread that called split. Inside it:

  • the graph's reads and queries run as usual;
  • a run, an edit, optimize(), finalize(), to() or a merge of the graph is refused (FailedPrecondition), since split holds it;
  • moving from the graph, or assigning over it, leaves both graphs as they were and logs the reason at Error: a move cannot fail, and split is still working on the graph.

Declared in ClikaRT/graph/compose.h, line 165

using AttrValue​

using AttrValue = std::variant<std::monostate, bool, std::int64_t, double, std::string, std::vector<std::int64_t>, std::vector<double>, std::vector<std::optional<std::int64_t>>, std::vector<std::optional<double>>, DataType, Scalar>

One attribute value. std::monostate means "not statically known": an unset optional setting, or a value the operator reads from a runtime operand instead of its configuration. A shape template keeps its per-slot knowledge: std::vector<std::optional<std::int64_t>> (and the double twin) holds the literal of every static slot and std::nullopt for every slot fed at run time.

Declared in ClikaRT/graph/node.h, line 73

using NodePredicate​

using NodePredicate = std::function<bool(const Node &)>

A test a graph node must pass: true accepts it. It runs for every candidate node the search or the walk visits.

Declared in ClikaRT/graph/node_predicate.h, line 20

using OnnxExportHandlerFn​

using OnnxExportHandlerFn = std::function<Result<std::vector<std::string>>(const Node &node, OnnxExportWriter &writer)>

Writes node's ONNX form through writer, and returns the ONNX names that become the node's outputs, one per node.outputs() entry, in that order.

Declared in ClikaRT/graph/onnx_export.h, line 75

using EdgePredicate​

using EdgePredicate = std::function<bool(const Edge &)>

A test an edge must pass for a walk to cross it: true crosses.

Declared in ClikaRT/graph/query.h, line 71

using NodeVisitor​

using NodeVisitor = std::function<VisitAction(const Node &)>

Called for every node a walk reaches, in walk order.

Declared in ClikaRT/graph/query.h, line 74

using NodeCost​

using NodeCost = std::function<double(const Node &)>

A node's cost (ModelGraph::critical_path).

Declared in ClikaRT/graph/query.h, line 127

using EdgeCost​

using EdgeCost = std::function<double(const Edge &)>

An edge's cost (ModelGraph::shortest_path, critical_path).

Declared in ClikaRT/graph/query.h, line 130

using RewriteFunction​

using RewriteFunction = std::function<std::vector<Tensor>(const RewriteMatch &, const std::vector<Tensor> &)>

A rule's replacement: given the occurrence and one tensor per match.inputs, the new values, one per match.outputs and in their order. The tensors are traced as ModelGraph::replace traces its function: each with its value's dtype, device and shape, and a constant scalar as the literal it holds.

Declared in ClikaRT/graph/rewrite.h, line 52

using RewriteCondition​

using RewriteCondition = std::function<bool(const RewriteMatch &)>

A rule's condition: whether to replace this occurrence. It reads the graph through the match's views and does not edit it.

Declared in ClikaRT/graph/rewrite.h, line 56

using TraceFunction​

using TraceFunction = std::function<std::vector<Tensor>(const std::vector<Tensor> &)>

A traceable model function: positional inputs → positional outputs. An nn::Module forward qualifies through a thin lambda.

Declared in ClikaRT/graph/trace_function.h, line 22

using TransformFunction​

using TransformFunction = std::function<Result<void>(ModelGraph &)>

A transform you write: it edits the graph it is handed through ModelGraph's edits (model_graph.h) and returns a failure to stop the run. optimize calls it on its own thread, once per iteration, at its place in the list.

Declared in ClikaRT/graph/transform.h, line 60

Variables​

kDefaultMaxNumIterations​

std::int64_t kDefaultMaxNumIterations = 100

The iteration cap OptimizeOptions starts from. One iteration runs the whole transform list once; the optimizer iterates because one rewrite can expose another, and stops at the first iteration that changes nothing.

Declared in ClikaRT/graph/optimize.h, line 20

kMaxOptionalPatternNodes​

std::size_t kMaxOptionalPatternNodes = 10

The most nodes one Pattern may mark optional. The search tries every combination of present and absent optional nodes, so each optional node doubles its work, and ModelGraph::find_pattern refuses a pattern that marks more as Status::InvalidArgument.

Declared in ClikaRT/graph/pattern.h, line 62

kMaxPathsWithoutLimit​

std::size_t kMaxPathsWithoutLimit = 10000

The most paths a listing returns when PathOptions::limit is empty (ModelGraph::find_paths_to, all_simple_paths, all_simple_edge_paths). A listing that finds more fails with Status::InvalidArgument naming this cap; it never returns a cut list. Paths multiply at every branch and join (a stack of residual blocks doubles them per block), so a longer list is a runaway: take the first ones with limit, or count them with ModelGraph::count_paths.

Declared in ClikaRT/graph/query.h, line 106

Functions​

to_table(BenchReport)​

tables::Table to_table(const BenchReport& report)

One row for the report, the shapes and outputs rendered as name:d0xd1x... strings (entries joined with ;), the device as API:index, the fill as zeros / uniform. The columns, in order: shapes, outputs, device, fill, warmup, iterations, threads, outputs_reused, outputs_on_device, first_call_ms, p50_ms, p95_ms, p99_ms, mean_ms, min_ms, max_ms, stddev_ms, batch_axis, items_per_iteration, throughput_items_per_s, peak_rss_bytes, device_memory_valid, device_active_bytes, device_reserved_bytes, device_peak_active_bytes, device_free_bytes, device_total_bytes. The same columns as the multi-report form, so a sweep's table appends row by row.

Declared in ClikaRT/graph/bench.h, line 171

to_table(vector<BenchReport>)​

tables::Table to_table(const std::vector<BenchReport>& reports)

One row per report, in order; the columns of the single-report form.

Declared in ClikaRT/graph/bench.h, line 175

iterations_table()​

tables::Table iterations_table(const BenchReport& report)

One row per measured run, columns iteration (0-based) and ms.

Declared in ClikaRT/graph/bench.h, line 179

to_json()​

json::Json to_json(const BenchReport& report)

Every field of the report as a JSON object under the table's column names: the shapes and outputs as arrays of {name, dims}, iteration_ms as an array, the device as API:index, the memory counters as the nested device_memory object with device::MemoryStats's field names.

Declared in ClikaRT/graph/bench.h, line 186

compile(TraceFunction, Span< spec::TensorSpec>, CaptureOptions)​

CompiledFunction compile(
    TraceFunction fn,
    ClikaRT::Span<const spec::TensorSpec> signature = {},
    CaptureOptions options = {}
)

Wrap fn (any callable of the graph::trace shape: positional tensors in, positional tensors out) in a CompiledFunction. signature follows graph::trace's vocabulary: one spec::TensorSpec per input, dynamic dims as kDynamicDim, shareable by dim_names; empty derives every spec from the first call's tensors. Options whose values the captured road cannot honor raise a readable error here (see CaptureOptions); a malformed signature surfaces at the first capture instead, warned once, and the function keeps serving eagerly.

Declared in ClikaRT/graph/compile.h, line 238

compile(TraceFunction, CaptureOptions)​

CompiledFunction compile(TraceFunction fn, CaptureOptions options)

Options-only form of compile; the signature derives from the first call.

Declared in ClikaRT/graph/compile.h, line 245

merge()​

MergedGraph merge(
    std::vector<GraphPart>& parts,
    ClikaRT::Span<const Connection> connections = {},
    const MergeOptions& options = {}
)

Merge parts into one graph. Each connection feeds an output of one part into an input of another, and parts no connection joins sit side by side.

  • Names: every operator of part p is named p/<name>. The inputs and outputs no connection joins are the merged graph's, named as options.io_names says; a connected input or output is none of the merged graph's. renames lists every name that changed.
  • Order: the merged graph's inputs are each part's remaining inputs, parts in the order given, and its outputs likewise.
  • Shapes: a connection's output and input have one dtype and one rank, and each size both of them fix is the same. A dynamic size that meets a fixed one takes that size, in every graph that shares the dynamic size. Two dynamic sizes stay separate: a Warning names them, and each run must give both the same size; give either one a fixed size to tie them.
  • Devices: a part's values sit on the device it was sent to (to(), or a compile for a device), else where they are computed. A connection between two devices moves the value onto the input's device, or refuses under CrossDevice::Refuse. When the parts do not all share one device, each part not yet on its device is placed there before the merge builds, and the merged graph keeps every part on its own device; when they share one, the merged graph is placed there by finalize(). A placement that fails puts every part back where it was and raises its error. If putting a part back fails too, that part is left on two devices, and it runs again only after a to() succeeds; the error says so.
  • KV caches: every part's KV cache layers are the merged graph's, under the merged names (kv_cache_info()).
  • Weights: the merged graph holds every part's weights, one copy of each.

On success every part's graph is consumed (left empty), and a view of it (node.h) then refuses as a view whose graph is gone. On a refusal, or on a failure inside the merge, every part is left as it was, its graph and its device, except that a dynamic size a connection already fixed keeps that size, which is the size the merge would give it. Merge before finalize() and attach_kv_cache(): the merged graph is ready for optimize() and finalize().

Refuses InvalidArgument: no parts; a part name that is empty, used twice, or holds a / or a .; an empty (moved-from) part graph; a connection naming a part, an output or an input that does not exist; an input fed twice; an output that is a state buffer its part writes in place; connections that feed parts round in a loop; a dtype, a rank or a fixed size the two ends of a connection disagree on; a connection between two devices under CrossDevice::Refuse; under IoNames::PrefixOnCollision, a merged name two inputs, or two outputs, would both take (the names IoNames::PrefixAlways gives never meet); and, under IoNames::Keep, a name two parts both use. Refuses FailedPrecondition: a part that is finalized, holds a KV cache, was left on two devices by a failed to(), or is busy with another call (a run, an edit, optimize(), finalize(), to(), another merge, a split or an extract).

Declared in ClikaRT/graph/compose.h, line 141

extract()​

ModelGraph extract(
    ModelGraph& graph,
    ClikaRT::Span<const Value> inputs,
    ClikaRT::Span<const Value> outputs
)

The operators of graph that compute outputs from inputs, as a graph of their own.

  • Inputs: in inputs' order, each value once. A graph input keeps its name, and any other value becomes a new input named after it (its name, else <node>_<port>).
  • Outputs: in outputs' order, each keeping its graph output name when it has one, else named after its value.
  • KV caches: the layers whose attention it holds come with it, their cache inputs among inputs and their cache outputs among outputs.
  • Weights: it holds its operators' weights, one copy of each, and the weights of the operators it leaves out are freed. The exception is weights stored inside an ONNX file rather than as external data: they share one buffer, which the extracted graph keeps in memory, as graph did.

On success graph is consumed (left empty), and a view of it (node.h) then refuses as a view whose graph is gone; on a refusal it is left as it was. Refuses InvalidArgument: no output; a value of another graph, a constant or a state value, a value an edit removed, or a value of a graph that is gone; operators that read a graph input inputs does not list; and a KV layer whose attention it holds without its cache inputs among inputs and its cache outputs among outputs. Refuses FailedPrecondition: a graph that is finalized, was left on two devices by a failed to(), or is busy with another call (a run, an edit, optimize(), finalize(), to(), a merge, a split or another extract).

Declared in ClikaRT/graph/compose.h, line 229

split(ModelGraph, ValueCut)​

SplitGraph split(ModelGraph& graph, const ValueCut& cut)

Split graph in two at cut, a min_value_cut() with an operator pinned on each side, say (an unpinned cut can fall at the graph's inputs or outputs, which this refuses): the part before holds the operators that produce the cut values and every operator they depend on, and the part after every other operator. The values after reads from before cross through connections. These are the cut values after reads and the graph inputs both parts read, so merge(parts, connections) joins the two parts back. The rest is the partitioning split below.

Also refuses InvalidArgument: a value of another graph, a constant or a state value, a value an edit removed, or a value of a graph that is gone; a cut that is not closed: an operator before it hands a value that is no cut value to an operator after it; and a cut with no operator on one side.

Declared in ClikaRT/graph/compose.h, line 248

split(ModelGraph, PartitionFunction)​

SplitGraph split(ModelGraph& graph, PartitionFunction partition)

Split graph into parts, partition naming each operator's part.

  • Parts: in an order every connection runs forward in, ties going to the part whose first operator comes first.
  • Values across parts: a value read in a part other than its producer's is an output there and an input of each reading part, under one name (its name, else <node>_<port>), and a connection joins them.
  • Inputs: a graph input goes with the first part that reads it, and every later reader receives it through a connection, so a merge gives one input back. The graph's own inputs and outputs stay with their parts, under their names.
  • KV caches: each layer goes with the part of its attention.
  • Weights: each part holds its operators' weights, one copy of each. A constant two parts read is shared, and parts placed on one device hold one copy of it there.

On success graph is consumed (left empty), and a view of it (node.h), the nodes partition saw included, then refuses as a view whose graph is gone; on a refusal it is left as it was. A partition that fails makes split refuse with its failure. Refuses InvalidArgument: an empty partition function; a part name that is empty or holds a / or a .; two nodes that run one operator and would land in two parts; a weight one operator holds, or a state buffer, that another part would also use; parts that feed each other round in a loop; a KV layer across two parts; and an output that returns a graph input or a constant with no operator computing it. Refuses FailedPrecondition: a graph that is finalized, was left on two devices by a failed to(), or is busy with another call (a run, an edit, optimize(), finalize(), to(), a merge, an extract or another split).

Declared in ClikaRT/graph/compose.h, line 281

generate_kv_input_specs()​

spec::InputSpecs generate_kv_input_specs(
    const ModelGraph& graph,
    const spec::KVSpecOptions& options = {}
)

Declared in ClikaRT/graph/model_graph.h, line 1178

op_code_count()​

constexpr std::size_t op_code_count() noexcept

The number of operators in the vocabulary (Unknown not counted).

Declared in ClikaRT/graph/op_code.h, line 44

op_code_name()​

constexpr std::string_view op_code_name(OpCode code) noexcept

The enumerator's name: "MatMul", "Custom", "Reserved0"; "Unknown" for OpCode::Unknown and for any value that names no operator.

Declared in ClikaRT/graph/op_code.h, line 51

parse_op_code()​

constexpr std::optional<OpCode> parse_op_code(std::string_view name) noexcept

The enumerator spelled name (exact, case-sensitive): OpCode::Unknown for "Unknown", std::nullopt when no operator has that name.

Declared in ClikaRT/graph/op_code.h, line 66

trace(TraceFunction, Span< spec::TensorSpec>, string_view, Span< string>)​

ModelGraph trace(
    TraceFunction fn,
    ClikaRT::Span<const spec::TensorSpec> signature,
    std::string_view graph_name = "traced_model",
    ClikaRT::Span<const std::string> output_names = {}
)

Trace fn against a SIGNATURE: one spec::TensorSpec per input, the library's one IO-slot vocabulary: name, dtype, dims (dynamic dims as kDynamicDim, shareable by dim_names). The road for models whose feeds are a chore to hand-build (many inputs, KV caches): the stand-ins are synthesized from the specs. output_names (optional, positional) names the function's returned outputs: one name per output, or none for the "output_0", "output_1", … defaults; surfaced in-place state outputs name themselves (see the authoring convention above) and are appended after. The graph comes back as recorded: call optimize() on it when wanted, to() to choose where it runs, then finalize(), and it runs.

Declared in ClikaRT/graph/trace.h, line 132

trace(TraceFunction, vector<Tensor>, Span< string>, string_view, …)​

ModelGraph trace(
    TraceFunction fn,
    const std::vector<Tensor>& tracing_inputs,
    ClikaRT::Span<const std::string> input_names = {},
    std::string_view graph_name = "traced_model",
    ClikaRT::Span<const std::string> output_names = {}
)

Trace fn against EXAMPLE TENSORS: their shapes and dtypes matter, their values do not (the trace runs on data-free stand-ins mirroring them). A thin reduction to the signature road: each example becomes its spec (spec::TensorSpec::from_tensors), so each input's shape traces as-is and the graph is specialized to it; declare dynamic dims through the signature road instead. input_names (optional, positional) names the graph inputs; absent names default to "input_0", "input_1", …. output_names names the returned outputs the same way (defaults "output_0", …); surfaced in-place state outputs are appended after. The graph comes back as recorded, exactly as on the signature road.

Declared in ClikaRT/graph/trace.h, line 149

ClikaRT/graph/bench.h​

#include <ClikaRT/graph/bench.h>

The vocabulary of ModelGraph::bench and the shape checks beside it: the extents a compiled graph admits per input (ShapeDomain, DimDomain), the shapes a benchmark feeds (InputShape), its knobs (BenchOptions), its measurement (BenchReport), and the table / JSON renderings of a report.

Every fallible operation returns its value directly and raises ClikaRT::Error on failure. Wrap a call in CLIKART_TRY(...) to inspect a Result instead of catching.

ClikaRT/graph/compile.h​

#include <ClikaRT/graph/compile.h>

ClikaRT::compile: wrap an eager model function in a self-capturing CompiledFunction: the first call serves the eager result and records the function into an executable ModelGraph; later calls whose input shapes match the capture run the graph instead. The wrapped function behaves EXACTLY like calling the function directly (same values, same failures) in every state; the graph is a serving substitution, never a semantic one.

​

The serving contract (fail-soft)​

A CompiledFunction is a small state machine:

  • Pending (fresh / after reset): a call serves the eager result, then captures synchronously, the same recording graph::trace performs, against the declared signature widened by the shapes the call actually presented. A capture failure is warned once and the function latches Fallback; the call's result is unaffected.
  • Compiled: a call first passes a cheap input-shape guard (dtype, rank, and every concrete captured axis must match; a dynamic captured axis matches any extent). On a match the captured graph serves the call. On drift the eager result serves, the drifted axes widen to dynamic, and the function re-captures; shapes only ever widen, so the loop converges; past kMaxRecaptures re-captures it latches Fallback instead. A graph run that fails is warned once, the call is served eagerly, and the function latches Fallback.
  • Fallback: every call serves eagerly. The function never re-arms itself; reset() is the explicit re-arm.

A call made while a trace is being recorded on its thread (inside a function graph::trace, ModelGraph::insert or replace traces) runs the function inline, so that trace records it, in every state: it neither captures, serves the graph nor latches, and the state stays as it was.

There is no staleness detection: a function whose captured behavior depends on external state (a C++ flag, a mutated closure tensor) keeps serving the captured behavior until an explicit reset() re-captures. State the function writes in place is not captured behavior: a closure KV cache an attention appends to, or a recurrence state an update advances, is written in place by every graph-served call, as by the function itself.

Calls are serialized: one call runs at a time per CompiledFunction (a concurrent caller waits). For concurrent serving, take_graph() the captured ModelGraph and share that, unless the function writes state in place: every run of such a graph advances that one state, so it runs one call at a time, and a run started while another is in flight refuses (FAILED_PRECONDITION). Two graphs that capture the same state are not serialized against each other.

​

Frictionless by default​

auto step = ClikaRT::compile([&](const std::vector<Tensor>& in) {
    return std::vector<Tensor>{ops::relu(in[0])};
});
auto y0 = step({x});   // eager + capture
auto y1 = step({x});   // served by the captured graph

With no signature, the capture derives one spec per input from the first call's tensors (shapes and dtypes; values are never read). Passing a signature (spec::TensorSpec, the same vocabulary graph::trace takes) names the inputs and declares dynamic dims up front; a declared spec that disagrees with an observed extent widens that axis to dynamic.

Every fallible operation returns its value directly and raises ClikaRT::Error on failure; wrap in CLIKART_TRY(...) to inspect a Result instead.

ClikaRT/graph/compose.h​

#include <ClikaRT/graph/compose.h>

One graph from several, and several from one: merge joins model graphs into one graph that runs them all, the outputs of one feeding the inputs of another, or side by side; split cuts a graph into parts that merge joins back; and extract takes the operators that compute some values from others.

Every fallible operation returns its value directly and raises ClikaRT::Error on failure. Wrap a call in CLIKART_TRY(...) to inspect a Result instead of catching.

ClikaRT/graph/model_graph.h​

#include <ClikaRT/graph/model_graph.h>

ModelGraph: an executable model graph with named, typed inputs and outputs, the operator graph between them, and the model's weights, ready to run.

A ModelGraph is the GENERIC executable form, not tied to any one source. Today it is produced by io::OnnxModel::compile (an ONNX model made executable) and by ClikaRT::trace (an eager model function recorded into a graph); future tooling that optimizes or quantizes a model operates on this same type.

Every fallible operation returns its value directly and raises ClikaRT::Error on failure. Wrap a call in CLIKART_TRY(...) to inspect a Result instead of catching.

ClikaRT/graph/node.h​

#include <ClikaRT/graph/node.h>

Node, Value and Edge: read-only views over a ModelGraph's operators, the tensors that flow between them and the edges that carry them (several edges may join two nodes), for graph tooling (an exporter, a quantizer, an audit) that needs the structure of a compiled graph and each operator's configuration. NodeMap<T> holds one T per node.

All three are views, like std::string_view over a string: small to copy, and they follow their graph to the ModelGraph it moves to. Each carries its key: a node's name, a value's producer and output port, an edge's two nodes and ports. After the graph is edited, a view finds its node, value or edge again by that key before it answers. When the key is gone (the node was removed or renamed), every accessor answers empty, check() fails with Status::InvalidArgument naming the key, and a ModelGraph query given the view fails the same way (one with no Result answers empty). A view whose graph no longer exists answers the same way, and its refusals say the graph is gone: a merge, a split or an extract consumed it, or the ModelGraph holding it was destroyed or assigned another graph. Obtain them from ModelGraph::nodes(), node(name), find_nodes(code) and constants(), and walk from there through inputs() / outputs() / producer() / predecessors() / successors(); bound_tensors() reads the weights a traced module bound into an operator. Every accessor is const; the fallible ones return a Result instead of raising.

ClikaRT/graph/node_predicate.h​

#include <ClikaRT/graph/node_predicate.h>

NodePredicate: a test you write over a Node, run by the graph searches (Pattern::add_node, ModelGraph::find_nodes) and by the walks (WalkOptions::stop_at).

ClikaRT/graph/onnx_export.h​

#include <ClikaRT/graph/onnx_export.h>

Handlers that write chosen operators your way when ModelGraph::export_onnx or io::OnnxModel::from_graph writes a graph as ONNX: an OnnxExportHandler replaces the library's ONNX form for one operator, in one export, and writes each of its nodes through an OnnxExportWriter.

ClikaRT/graph/op_code.h​

#include <ClikaRT/graph/op_code.h>

OpCode: the operator vocabulary of a compiled graph. Every operator the runtime executes has one enumerator; graph::Node::op_code() reports it and ModelGraph::find_nodes(OpCode) selects by it.

The vocabulary is part of the public API: every name here is stable and readable in every build, fused and runtime-only operators included, and a name a program persists keeps its meaning across releases. The numeric values are NOT part of the contract: they follow the runtime's operator table and may change between releases. Persist and compare NAMES (op_code_name, parse_op_code), never integers.

Three kinds of enumerator carry a special meaning:

  • Custom is every user-defined operator (an nn::Module that overrides output_shapes and compute); the operator's own name is Node::custom_name().
  • Placeholder is a graph input: an Input node whose single output is the tensor a caller binds at run.
  • Reserved<N> is a slot held for a future operator; no node reports one. Unknown is the sentinel for a value that names no operator.

Macros​

#define CLIKART_DETAIL_OP_CODE_ENUMERATOR​

#define CLIKART_DETAIL_OP_CODE_ENUMERATOR(name) name,

Declared in ClikaRT/graph/op_code.h, line 33

#define CLIKART_DETAIL_OP_CODE_COUNT_ONE​

#define CLIKART_DETAIL_OP_CODE_COUNT_ONE(name) +1

Declared in ClikaRT/graph/op_code.h, line 42

#define CLIKART_DETAIL_OP_CODE_NAME​

#define CLIKART_DETAIL_OP_CODE_NAME(name) case OpCode::name: \ return #name;

Declared in ClikaRT/graph/op_code.h, line 53

#define CLIKART_DETAIL_OP_CODE_PARSE​

#define CLIKART_DETAIL_OP_CODE_PARSE(row) if (name == #row) return OpCode::row;

Declared in ClikaRT/graph/op_code.h, line 67

ClikaRT/graph/pattern.h​

#include <ClikaRT/graph/pattern.h>

Pattern: a small graph of operator constraints that ModelGraph::find_pattern locates inside a compiled graph. Read-only: every occurrence comes back as the matched Nodes, one per pattern node, for an audit that counts unfused pairs, an exporter deciding how a group of operators lowers, or a quantizer choosing where a fused kernel lands.

Build a pattern once and match it against any number of graphs. add_node declares what a graph node must be: one operator, one of several, or a predicate you write over the Node view. add_edge declares which pattern node feeds which, on any port or on the ports you name. The first node added is the root the search starts from. Pattern::chain builds the common straight line (MatMul -> Relu) in one call.

Matching rules: occurrences never overlap (a graph node belongs to at most one occurrence per search: occurrences are claimed in match order, the richest reading of an optional node first, and a later occurrence that shares a graph node with an earlier one is dropped); a matched node may have inputs and consumers the pattern does not name (a MatMul that also feeds a second reader is still a MatMul -> Relu occurrence); an optional pattern node may be absent from an occurrence, and then its Match entry is std::nullopt and its predecessors are read as feeding its successors (the search adds that bypass itself; a pattern spells only the full line).

ClikaRT/graph/query.h​

#include <ClikaRT/graph/query.h>

The words of the ModelGraph queries: how a name search matches, which way a walk crosses edges, what a visitor answers, WalkOptions, the bounds and filters of a walk (ModelGraph::walk, visit, ancestors, descendants), PathOptions and the listing cap of the path queries, the costs and WeightedPath of the weighted ones, ValueCut, and which topological order ModelGraph::topological_order returns. A walk or a path follows the edges of node.h: they join nodes, so a constant or a state operand is never crossed.

ClikaRT/graph/trace.h​

#include <ClikaRT/graph/trace.h>

ClikaRT::trace: record an eager model function into an executable ModelGraph.

Any callable written against the public ops/nn surface (positional tensors in, positional tensors out) can be traced: the function runs ONCE on symbolic stand-in tensors, every op it calls records into a graph instead of computing, and the result is the graph as recorded, the same kind of ModelGraph that io::OnnxModel::compile produces: not yet optimized and not yet finalized. optimize() runs the graph optimizer when wanted, finalize() readies the graph to run, and graph-level tooling (queries, edits, quantization; calibration inputs are consumed at that stage) works on it before that, like on any other ModelGraph.

​

The authoring convention (what makes a function traceable)​

  • Public ops/nn only. Every tensor-producing step goes through ClikaRT::ops::* / ClikaRT::nn::*; that is what records. Module weights (real bound tensors) become the graph's weights automatically.
  • Inputs are the signature. A tensor the function reads is one of its inputs or holds data (a captured weight or constant; a plain number such as the 0.5 of y * 0.5 records as a constant on y's device). A data-free tensor from elsewhere, such as another trace's stand-in, fails the trace when it is recorded, naming where it is read.
  • No data-dependent control flow. The trace records ONE execution path, so a branch on host data picks its arm at trace time and that arm is what the graph replays forever. Reading a tensor's VALUE during tracing (item, item_as_vec, host data access) fails the trace loudly; the stand-ins carry no data by design, precisely so a value dependence cannot bake in silently. Branching on plain C++ values (configuration, flags) is fine: the taken arm is the model.
  • Dynamic dims are declared, not defaulted. A dimension that should stay flexible at run time must be dynamic in the tracing signature (spec::TensorSpec::kDynamicDim, optionally named via dim_names; the same name on two inputs declares the same size, exactly as in io::OnnxModel). A concrete signature dim traces a graph specialized to it.
  • State is IO, explicit or surfaced. Anything that persists across calls (a KV cache, a state buffer) either enters as an input and leaves as a returned output, or is a captured live tensor the function writes through an in-place op; the trace then SURFACES each written buffer as an additional named graph output automatically. An attention layer's in-place present planes surface as present.<layer>.key / present.<layer>.value (layer-ordered; enumerated by ModelGraph::kv_cache_info), other written buffers as state_out_<k>. An operation that writes its state in place by contract (an attention layer's present planes, a recurrence update's state) writes the captured buffer itself on every run, as the function does. For any other write, a plain run returns the output in fresh storage; to advance those live buffers in place (the serving-loop cache semantics), bind each surfaced output to its own live tensor through run's bound-output form. Either way the graph computes from its inputs and the captured buffers' current contents. The trace itself writes nothing: when it returns, each captured tensor the function wrote holds what it held before, and every handle the function wrote it through reads it as before.

Every fallible operation returns its value directly and raises ClikaRT::Error on failure; wrap in CLIKART_TRY(...) to inspect a Result instead.

ClikaRT/graph/trace_function.h​

#include <ClikaRT/graph/trace_function.h>

TraceFunction: a function over the public ops that a trace records, the kind graph::trace captures into a graph and ModelGraph::insert and replace splice into one. Its throw-containment lives beside it, so every call that traces one catches a throw in your translation unit.