ClikaRT::graph::ModelGraph
class
Header: ClikaRT/graph/model_graph.h
An executable model graph. Move-only.
Member functions
ModelGraph(ModelGraph)
ModelGraph(ModelGraph&&) noexcept
Move-only: transfers the graph. Inside one of the graph's transforms the move leaves both graphs as they were (optimize).
Declared in ClikaRT/graph/model_graph.h, line 141
operator=(ModelGraph)
ModelGraph& operator=(ModelGraph&&) noexcept
Move-assign: transfers the graph, with the same exception.
Declared in ClikaRT/graph/model_graph.h, line 143
ModelGraph(ModelGraph)
ModelGraph(const ModelGraph&) =delete
Declared in ClikaRT/graph/model_graph.h, line 144
operator=(ModelGraph)
ModelGraph& operator=(const ModelGraph&) =delete
Declared in ClikaRT/graph/model_graph.h, line 145
~ModelGraph()
~ModelGraph()
Releases the graph.
Declared in ClikaRT/graph/model_graph.h, line 147
empty()
bool empty() const noexcept
True for a graph that holds none: default-constructed, moved from, or consumed by a merge, a split or an extract (every part of graph::merge, the second of merge, and the graph graph::split or graph::extract is given).
Declared in ClikaRT/graph/model_graph.h, line 153
input_names()
std::vector<std::string> input_names() const
Runtime input names, in the model's declaration order, the order the positional run binds. Weights are not inputs; they ride the graph.
Declared in ClikaRT/graph/model_graph.h, line 158
output_names()
std::vector<std::string> output_names() const
Output names, declaration order, the order every run returns.
Declared in ClikaRT/graph/model_graph.h, line 160
inputs()
std::vector<spec::TensorSpec> inputs() const
Input / output slots as (name, dtype, dims) specs, same order as input_names() / output_names(). A dimension the graph left dynamic reads as spec::TensorSpec::kDynamicDim (-1).
Declared in ClikaRT/graph/model_graph.h, line 165
outputs()
std::vector<spec::TensorSpec> outputs() const
The produced edges as specs, the output twin of inputs().
Declared in ClikaRT/graph/model_graph.h, line 167
run(vector<Tensor>)
Run with positional inputs (one per input_names() entry, that order). Returns the outputs in output_names() order.
Declared in ClikaRT/graph/model_graph.h, line 185
run(NamedTensors)
std::vector<Tensor> run(const NamedTensors& inputs) const
Run with positional inputs (one per input_names() entry, that order). Returns the outputs in output_names() order.
Declared in ClikaRT/graph/model_graph.h, line 192
run(vector<Tensor>, vector<Tensor>)
Run with positional inputs (one per input_names() entry, that order). Returns the outputs in output_names() order.
Declared in ClikaRT/graph/model_graph.h, line 202
run(NamedTensors, NamedTensors)
std::vector<Tensor> run(const NamedTensors& inputs, const NamedTensors& bound_outputs) const
Run with positional inputs (one per input_names() entry, that order). Returns the outputs in output_names() order.
Declared in ClikaRT/graph/model_graph.h, line 213
attach_kv_cache(nn::KVCacheConfig)
void attach_kv_cache(const nn::KVCacheConfig& config)
Build ONE nn::KVCache on the graph's device from kv_cache_info() and bind it to every self-attention layer. Fields of config left at 0 are taken from the graph: num_layers (the self-attention layer count), num_kv_heads and head_dim (each layer's own values fill its row, so a stack with mixed geometry binds per layer); a layer's sliding window becomes its row's window; kv_dtype is taken from the graph's KV planes (the config's value is not consulted). mode, paged, max_seqs, max_tokens_per_seq and the quantization fields come from config as given. The attentions write the cache planes in place, so each continuous row without a window is preallocated at max_tokens_per_seq and its planes never move; a windowed row is a fixed ring of its window, and a paged row takes blocks from the pool. After this call the past inputs leave input_names(); the present outputs stay in output_names() and run(inputs, step) returns each layer's cache plane at its present position, no copy. Every self-attention layer reports KVCacheLayerInfo::bound_to_cache, and the graph runs through run(inputs, const nn::KVCache::StepIndices&).
The present K/V of a bound layer ARE the cache planes: read them through kv_cache()->keys(l) / values(l) (a continuous cache's [max_seqs, kv_heads, max_tokens_per_seq, head_dim] plane; a paged cache's block pool, addressed through paged_block_table()). A graph with an interior consumer of a present output (a graph node reading it) refuses to attach (Unsupported): the in-place append needs the plane bound at the output port.
Refuses InvalidArgument on a graph with no KV layers, one already attached, or a config the cache cannot build (the cache's own message); Unsupported on a graph with a cross-attention layer; FAILED_PRECONDITION on a graph that is not finalized, while a run is in flight, while optimize, finalize or an edit runs on it, and inside one of its transforms ("Transforms you write"). One cache, one graph run at a time: the caller serializes run calls on the graph (the cache's prepare_step already requires one driving thread).
Declared in ClikaRT/graph/model_graph.h, line 255
attach_kv_cache(shared_ptr<nn::KVCache>)
void attach_kv_cache(std::shared_ptr<nn::KVCache> cache)
Build ONE nn::KVCache on the graph's device from kv_cache_info() and bind it to every self-attention layer. Fields of config left at 0 are taken from the graph: num_layers (the self-attention layer count), num_kv_heads and head_dim (each layer's own values fill its row, so a stack with mixed geometry binds per layer); a layer's sliding window becomes its row's window; kv_dtype is taken from the graph's KV planes (the config's value is not consulted). mode, paged, max_seqs, max_tokens_per_seq and the quantization fields come from config as given. The attentions write the cache planes in place, so each continuous row without a window is preallocated at max_tokens_per_seq and its planes never move; a windowed row is a fixed ring of its window, and a paged row takes blocks from the pool. After this call the past inputs leave input_names(); the present outputs stay in output_names() and run(inputs, step) returns each layer's cache plane at its present position, no copy. Every self-attention layer reports KVCacheLayerInfo::bound_to_cache, and the graph runs through run(inputs, const nn::KVCache::StepIndices&).
The present K/V of a bound layer ARE the cache planes: read them through kv_cache()->keys(l) / values(l) (a continuous cache's [max_seqs, kv_heads, max_tokens_per_seq, head_dim] plane; a paged cache's block pool, addressed through paged_block_table()). A graph with an interior consumer of a present output (a graph node reading it) refuses to attach (Unsupported): the in-place append needs the plane bound at the output port.
Refuses InvalidArgument on a graph with no KV layers, one already attached, or a config the cache cannot build (the cache's own message); Unsupported on a graph with a cross-attention layer; FAILED_PRECONDITION on a graph that is not finalized, while a run is in flight, while optimize, finalize or an edit runs on it, and inside one of its transforms ("Transforms you write"). One cache, one graph run at a time: the caller serializes run calls on the graph (the cache's prepare_step already requires one driving thread).
Declared in ClikaRT/graph/model_graph.h, line 274
kv_cache()
std::shared_ptr<nn::KVCache> kv_cache() const
The attached cache; null when none is attached. Owned or injected, it is the one nn::KVCache the serving loop drives.
Declared in ClikaRT/graph/model_graph.h, line 280
run(vector<Tensor>, nn::KVCache::StepIndices)
std::vector<Tensor> run(std::vector<Tensor> inputs, const nn::KVCache::StepIndices& step) const
Run with positional inputs (one per input_names() entry, that order). Returns the outputs in output_names() order.
Declared in ClikaRT/graph/model_graph.h, line 294
run(NamedTensors, nn::KVCache::StepIndices)
std::vector<Tensor> run(const NamedTensors& inputs, const nn::KVCache::StepIndices& step) const
Run with positional inputs (one per input_names() entry, that order). Returns the outputs in output_names() order.
Declared in ClikaRT/graph/model_graph.h, line 301
kv_cache_info()
std::vector<KVCacheLayerInfo> kv_cache_info() const
Per-attention-layer KV-cache descriptors, layer order. Empty = the graph carries no KV cache. After attach_kv_cache, every self-attention layer reports bound_to_cache with empty past_*_input (they left the input list; the present names stay), and generate_kv_input_specs contributes nothing for those layers.
Declared in ClikaRT/graph/model_graph.h, line 312
nodes()
std::vector<Node> nodes() const
Every node in topological order: each node after the producers of its inputs, the Input nodes first. Fails when the graph holds a cycle (the error's code_name() is INVALID_GRAPH).
Declared in ClikaRT/graph/model_graph.h, line 328
node()
std::optional<Node> node(std::string_view name) const
The node named name, or std::nullopt.
Declared in ClikaRT/graph/model_graph.h, line 331
find_nodes(OpCode)
Every node whose op_code() is code, in name order.
Declared in ClikaRT/graph/model_graph.h, line 334
find_pattern()
Every occurrence of pattern (pattern.h): one Match per occurrence, the matched node per pattern node in the pattern's add_node order, occurrences never overlapping. Fails when a pattern edge names a node that was never added or a node was added with an empty predicate (Status::InvalidArgument), and with a predicate's own failure when one throws.
Declared in ClikaRT/graph/model_graph.h, line 343
constants()
std::vector<Value> constants() const
The graph's edge constants: every tensor bound onto an edge with bytes fixed for the graph's lifetime (an ONNX initializer, a constant a trace captured), one Value each, in the graph's own order; Value::constant() reads the bytes. A compiled ONNX initializer is also its operator's bound tensor (Node::bound_tensors(), the same bytes); a traced module's weights are bound into the operators alone and are not edge constants. A constant add_constant made is listed before anything reads it.
Declared in ClikaRT/graph/model_graph.h, line 355
is_finalized()
bool is_finalized() const
True once finalize() ran: the operators are finalized for serving, weights packed into their kernel layouts and constants absorbed. compile and trace return a graph that is not finalized. Only a finalized graph runs; the edits below, optimize() and other graph rewrites belong before this point.
Declared in ClikaRT/graph/model_graph.h, line 362
replace_all_uses_with()
Every read of old_value reads replacement instead: each operator input it feeds, and each graph output it returns (the output keeps its name). Nothing reads old_value afterwards, so its view is gone, unless it is a graph input's value. Refuses replacement being old_value, a type mismatch, a replacement computed from one of old_value's readers (it would form a cycle), a state buffer's reads, a read an operator serves from a weight it holds itself, and a constant taking a graph output.
Declared in ClikaRT/graph/model_graph.h, line 402
remove_node()
void remove_node(const Node& node)
Remove node, an operator nothing reads, with its reads of other values (a constant only it read goes with it). Refuses an Input node, a node that reads a state buffer, and a node whose outputs are still read (FAILED_PRECONDITION, naming the first reader: rewire or remove that reader first).
Declared in ClikaRT/graph/model_graph.h, line 412
bypass_node()
void bypass_node(
const Node& node,
int input_port = 0,
int output_port = 0
)
Every read of node's output output_port reads the value on its input input_port instead, and node is removed once nothing reads any of its outputs. Refuses an Input node, an input port nothing feeds, an output port nothing reads, a type mismatch between the two values, a node that reads a state buffer, and a constant taking a graph output.
Declared in ClikaRT/graph/model_graph.h, line 420
set_input()
node's input port reads value. The edit replaces the value the port reads, so the port already reads one: an input the operator was built without stays empty. Refuses an Input node (its one input is the tensor run binds), a port nothing feeds, a state buffer's read, a port the operator serves from a weight it holds itself, a type mismatch with the value the port reads, and a value computed from node's own outputs (it would form a cycle).
Declared in ClikaRT/graph/model_graph.h, line 432
set_constant_input()
node's input port reads a new constant holding value's bytes: the graph keeps a handle to them, no copy, and finalize() places them with the graph. The constant replaces the value the port reads, so the port already reads one, of the same type. Refuses an Input node, a port nothing feeds, a state buffer's read, a port the operator serves from a weight it holds itself, a tensor with no data (a traced or a meta tensor), and a type mismatch with the value the port reads.
Declared in ClikaRT/graph/model_graph.h, line 444
add_constant()
A new constant holding value's bytes (a handle, no copy) that nothing reads yet; set_input and replace_all_uses_with take the value it returns. constants() lists it. Once an operator reads it, it lives while something reads it, like every constant; finalize() drops it if nothing reads it by then. Refuses a tensor with no data (a traced or a meta tensor).
Declared in ClikaRT/graph/model_graph.h, line 455
add_node()
Node add_node(
OpCode code,
ClikaRT::Span<const std::optional<Value>> inputs,
ClikaRT::Span<const Attr> attributes = {}
)
A new operator of code, built the way its public op builds it (the op's own checks apply). inputs holds its values, one entry per input port in the op's order: std::nullopt omits an optional operand, and trailing entries may be left off; a constant enters as a value (add_constant). attributes holds its settings under the names Node::attributes() reports: an integer also serves a number, an enumerated setting takes its enumerator's name, and a setting left out takes the op's default. A setting the op derives from its inputs instead of taking it (MaskedFill's fill_is_exact_zero, true only for a literal zero) takes the value the op builds when left out, and when given must be that value; where it asks for one, a constant scalar input goes in as the scalar it holds. A template setting (a list that Node::attributes() reports with std::nullopt in each slot fed at run time, such as a shape, Pad's pad or Slice's bounds) keeps its literal slots, and each run-time slot reads the next input past the op call's own inputs, in the order the op's parameters list them; a lone bound with no value (Select's index) reads the next input the same way. A setting the op takes as a plain list (Unflatten's sizes) takes a template only when every slot is literal. A scalar operand the op records as a setting when it is a literal (Clamp's min, SoftcapLogits' cap) reads its input when one is given and the setting of its name otherwise; given both, the setting must be what the op builds for that input. Full's fill_value is its setting alone and takes no input. Nothing reads its outputs yet, so wire them with set_input, replace_all_uses_with or add_output; finalize() drops a node nothing reads by then, with what only it read. The node is named <OpCode>_<N>, as a trace names it (rename_node renames it). Refuses a value of another graph or one no longer held, a state buffer, a missing required input or setting, an input or a setting the op does not take, a setting of another kind, a slot fed at run time with no input for it, a derived setting or one beside its input that the op builds otherwise, the op's own refusal of the arguments, and a code no single public op call builds (an operator with several outputs, one whose public op call takes a setting the operator does not record, one whose op call takes a quantized tensor object). Build those from the public ops that compute them.
Declared in ClikaRT/graph/model_graph.h, line 496
insert()
std::vector<Value> insert(TraceFunction fn, ClikaRT::Span<const Value> inputs)
New operators computed from inputs: the ones fn records when it is traced over one tensor per input, each with that value's dtype, device and shape (a dynamic dim keeps the graph's own symbol). A constant scalar input goes into fn as the literal it holds, so a value fn reads from it (a range's end, a count) keeps the extent it gives. Returns the values fn returns, in its order, which nothing reads yet: wire them as add_node's outputs are wired. A tensor holding data that fn makes or reads becomes a constant of the graph, a value fn returns unchanged comes back as the input it is, and the new nodes take fresh names. fn computes values: it does not write one of its inputs, or a tensor that holds data, in place, and it does not edit this graph or call optimize(). Refuses a value of another graph or one no longer held, a state buffer, a value with no recorded type, an empty fn, fn's own failure or throw (with its status and code), such an in-place write, an output of fn that is undefined or holds data, and a read of a tensor without data that is none of its inputs.
Declared in ClikaRT/graph/model_graph.h, line 518
replace()
std::vector<Value> replace(
ClikaRT::Span<const Value> old_outputs,
TraceFunction fn,
ClikaRT::Span<const Value> inputs
)
insert(fn, inputs), then every read of old_outputs[i] reads new value i, the graph's returns included (they keep their names), and the operators that computed old_outputs from inputs go, with every producer only they read. An operator an input is computed from stays, since the new values read that input; when it makes an old output (an input read from another of its outputs), that output's reads move all the same. Returns the new values. As in insert, a constant scalar input goes into fn as the literal it holds, so a value fn reads from it keeps its extent. Refuses what insert refuses, and: no old outputs, one listed twice, or one that is a constant, a state buffer or a graph input's value; an old output that is also one of inputs; an old output not computed from inputs; a value those operators make that another node reads or the graph returns (naming it and its reader); a KV cache layer's attention among them (kv_cache_info()); an input computed from a reader of an old output (a cyclic graph); a count of new values other than the old outputs'; a new value whose type differs from its old output's; and a constant a graph output would return.
Declared in ClikaRT/graph/model_graph.h, line 543
rename_node()
void rename_node(const Node& node, std::string name)
Name node name; its outputs keep their names. Views of the old name are gone afterwards: find the node again with node(name). Refuses an Input node (rename it with rename_input), an empty name, and a name another node carries.
Declared in ClikaRT/graph/model_graph.h, line 552
add_input()
Node add_input(const spec::TensorSpec& spec)
A new graph input named spec.name, last in input_names(), with the dtype and dims of spec: a dynamic dim takes its size at run, and dims that share a dim_names entry share that size. Returns its Input node, whose value (output(0)) the other edits take. Refuses an empty name, a name a node carries already, and a dim that is neither positive nor spec::TensorSpec::kDynamicDim.
Declared in ClikaRT/graph/model_graph.h, line 563
add_output()
void add_output(const Value& value, std::string name)
Return value as a new graph output named name, last in output_names(); a value the graph returns already may be returned under a second name. Refuses an empty name, a name another output carries, and a constant or a state buffer (a graph output never returns one directly).
Declared in ClikaRT/graph/model_graph.h, line 571
remove_output()
void remove_output(std::string_view name)
Stop returning the graph output name; the node that computes it stays. Refuses a name no output carries, and an output a KV cache layer returns its cache as (FAILED_PRECONDITION, naming the layer).
Declared in ClikaRT/graph/model_graph.h, line 579
rename_input()
void rename_input(std::string_view name, std::string new_name)
Rename the graph input name, and its Input node, new_name. Views of the old name are gone afterwards. Refuses a name no input carries, an input a KV cache layer reads its cache from (FAILED_PRECONDITION, naming the layer), an empty name, and a name a node carries already.
Declared in ClikaRT/graph/model_graph.h, line 586
rename_output()
void rename_output(std::string_view name, std::string new_name)
Rename the graph output name new_name. Refuses a name no output carries, an output a KV cache layer returns its cache as (FAILED_PRECONDITION, naming the layer), an empty name, and a name another output carries.
Declared in ClikaRT/graph/model_graph.h, line 595
optimize()
OptimizeReport optimize(const OptimizeOptions& options = OptimizeOptions())
Run the graph optimizer: options.transforms, or this graph's default list when it is std::nullopt (default_transforms(options.defaults)), iterated until an iteration changes nothing or options.max_num_iterations have run. The defaults resolve as the build that produced the graph resolves them (optimize.h). Returns what the run did, a row per list entry included. One of the runtime's transforms that fails is undone and skipped, and its row says so; the run goes on. A transform you wrote that fails ends the run: optimize returns its failure and leaves the graph as it found it ("Transforms you write"). Refuses FAILED_PRECONDITION on a finalized graph, while a run is in flight, while another optimize, finalize or an edit runs on this graph, inside one of its transforms, and while a trace is being recorded on this thread (inside a function graph::trace, insert or replace traces); InvalidArgument when max_num_iterations is outside 1 to 2147483647, when the list places a transform ahead of one it must follow (naming both), when the list holds a transform that options.defaults switch off for this graph (naming the choice to set), and when it holds two different transforms of one name (naming both).
Transforms you write (Transform::from_function) run beside the runtime's, at their places in the list, each on the thread that called optimize. One counts as changing the graph when one of its edits did; a refused edit, and one that changes nothing, change nothing. Inside one:
- the edits above run as part of this
optimize; an edit of this graph from another thread is refused; - the graph's inputs and outputs (their names in order, and each one's dtype, rank and dims) change only through the I/O edits; a change made any other way fails the transform;
optimize,finalize,toandattach_kv_cacheon this graph are refused (FAILED_PRECONDITION, naming the transform);- moving from this graph, or assigning over it, leaves both graphs as they were and logs the reason at Error: a move cannot fail, and
optimizeis still working on the graph; - calls on other graphs,
graph::traceincluded, run as usual. A transform that fails, or throws, ends the run:optimizereturns its status and code as it gave them, with the message "optimize: transforms[i] (name) failed: " and its own, and the graph is asoptimizefound it, the changes of every transform in the run undone.
Declared in ClikaRT/graph/model_graph.h, line 644
default_transforms()
std::vector<Transform> default_transforms(const DefaultsOptions& defaults = DefaultsOptions()) const
The transform list optimize() runs on this graph under defaults when OptimizeOptions::transforms is std::nullopt, in the order it runs them: a starting point to edit and hand back as OptimizeOptions::transforms. It follows how the graph was built, so it can differ from the graph-free transforms::defaults(defaults). Empty on a moved-from graph.
Declared in ClikaRT/graph/model_graph.h, line 654
finalize()
void finalize()
Finalize the graph for serving: place it on its device (CompileOptions::weights, or the one to() named; a graph with neither keeps its operators where they are), pack the weights into their kernel layouts, absorb the constants the operators hold, and plan the execution order. A finalized graph runs and refuses optimize; calling finalize again succeeds and changes nothing. Refuses FAILED_PRECONDITION while another optimize, finalize or an edit runs on this graph, and inside one of its transforms (optimize). Refuses DEVICE_MISMATCH when a graph input sits on a device that an operator reading it cannot take it from. A traced graph builds its inputs on the host, so a traced graph whose operators run on an accelerator needs to() first. The refusal names the input and the operator and leaves the graph unfinalized: to(stream), then finalize(), serves it.
Declared in ClikaRT/graph/model_graph.h, line 671
to()
void to(StreamOrDevice where)
Move the graph to a placement: a Device, or a Stream so the graph computes on that stream (build once, to(lane) once, run every step). A Device means the stream StreamOrDevice(device).resolve() names: the calling thread's current stream on that device. Before finalize() it records where finalizing places the graph and moves nothing; device() reports the new device at once. After finalize() it moves the finalized graph, which then computes on the new stream: an operator that changes device re-packs its weights there once, one already on that device keeps its packed weights, the execution order stays the one finalizing chose, and no copy of a moved weight stays behind. A graph traced from a module computes through the module's own operators, so moving the graph moves the module's weights too, an operator the optimizer rebuilt included; a module that already serves on another device stays there. Refuses FAILED_PRECONDITION while a KV cache is attached (call to before attach_kv_cache), while a run is in flight, while optimize, finalize, attach_kv_cache or an edit runs on the graph, and inside one of its transforms (optimize); a device that cannot lease a stream refuses before anything moves. A move that fails part-way leaves the graph finalized and serving where it was.
Declared in ClikaRT/graph/model_graph.h, line 695
device()
Device device() const
The device the graph serves on: the one CompileOptions::weights or to named. A graph with neither serves where its operators sit; when they sit on several devices, this is the device of the first one in nodes() order. A graph with no operators reports the CPU.
Declared in ClikaRT/graph/model_graph.h, line 701
merge()
std::vector<Rename> merge(
ModelGraph& second,
const std::vector<std::pair<std::string, std::string>>& io_map = {}
)
Merge second into this graph, as io::OnnxModel::merge merges two models. Each io_map entry (this_output, second_input) feeds one of this graph's outputs into one of second's inputs; the connected outputs and inputs leave the graph's lists, and everything else stays. The inputs are this graph's, then second's unconnected ones, and the outputs are second's, then this graph's unconnected ones. This graph keeps every name; a node, input or output name of second that this graph already uses takes the first free <name>_<n>. Returns every name that changed, each under the part second (Rename).
The rest is graph::merge's (compose.h), the two graphs being the parts this and second: the shapes, the devices (a connection between two devices moves its value, as under CrossDevice::Move), the KV caches, the weights and every refusal. On success second is consumed (left empty) and this graph is the merged one: a view of second then refuses as a view whose graph is gone (node.h), and a view of this graph finds its node again by key. On a refusal, or on a failure inside the merge, both are left as they were. Also refuses InvalidArgument for a merge of a graph with itself, and FailedPrecondition inside one of this graph's transforms (optimize).
Declared in ClikaRT/graph/model_graph.h, line 725
find_nodes_by_name()
std::vector<Node> find_nodes_by_name(
const regex::Regex& pattern,
NameMatch match = NameMatch::Whole
) const
Every node whose name pattern matches, in name order: the whole name (NameMatch::Whole) or some part of it (NameMatch::Anywhere).
Declared in ClikaRT/graph/model_graph.h, line 743
find_nodes(NodePredicate)
std::vector<Node> find_nodes(NodePredicate predicate) const
Every node predicate accepts, in name order. The predicate runs throw-contained: a throw ends the search, which fails with it. Refuses an empty predicate (Status::InvalidArgument).
Declared in ClikaRT/graph/model_graph.h, line 749
edges()
std::vector<Edge> edges() const
Every edge, in (source name, source port, destination name, destination port) order.
Declared in ClikaRT/graph/model_graph.h, line 755
edges_between()
The edges from src to dst, in (source port, destination port) order. Empty when no edge runs from one to the other, a node of another graph included.
Declared in ClikaRT/graph/model_graph.h, line 760
walk()
std::vector<Node> walk(const Node& start, WalkOptions options = WalkOptions()) const
The nodes a walk from start reaches, in walk order: breadth-first, the start first, each node once at its fewest edges from the start, and a node's neighbors in out_edges() order (in_edges() walking backward). Fails on options.max_nodes and with a callable's own failure.
Declared in ClikaRT/graph/model_graph.h, line 767
visit()
void visit(
const Node& start,
NodeVisitor visitor,
WalkOptions options = WalkOptions()
) const
The same walk, showing visitor each node it reaches, the start included: VisitAction::Prune does not go past that node, Stop ends the walk, which then succeeds. A throw from the visitor ends the walk, which fails with it. Refuses an empty visitor (Status::InvalidArgument).
Declared in ClikaRT/graph/model_graph.h, line 777
layers()
std::vector<std::vector<Node>> layers() const
The topological generations: the nodes with no in-edge first (Node::in_degree() 0), then each node one layer after its latest producer; every layer in nodes() order. Fails like nodes() on a cycle.
Declared in ClikaRT/graph/model_graph.h, line 787
ancestors()
std::vector<Node> ancestors(const Node& node, WalkOptions options = WalkOptions()) const
Every node with a path to node, node left out, nearest first: a backward walk, so options.direction is not read.
Declared in ClikaRT/graph/model_graph.h, line 792
descendants()
std::vector<Node> descendants(const Node& node, WalkOptions options = WalkOptions()) const
Every node with a path from node, node left out, nearest first: a forward walk, so options.direction is not read.
Declared in ClikaRT/graph/model_graph.h, line 799
find_paths_to()
std::vector<std::vector<Node>> find_paths_to(
const Node& start,
NodePredicate target,
PathOptions options = PathOptions()
) const
Every simple path from start that ends at the first node target accepts: a path never passes an accepted node, and an accepted start is the one-node path. Depth-first, each node's next nodes in out_edges() order; parallel edges give one path. The target and the edge filter each run once per node or edge. Fails past kMaxPathsWithoutLimit paths when options.limit is empty, and with a callable's own failure. Refuses an empty target (Status::InvalidArgument).
Declared in ClikaRT/graph/model_graph.h, line 814
has_path()
bool has_path(
const Node& src,
const Node& dst,
PathOptions options = PathOptions()
) const
True when a path runs from src to dst within options; a node reaches itself.
Declared in ClikaRT/graph/model_graph.h, line 822
shortest_path(Node, Node, PathOptions)
std::vector<Node> shortest_path(
const Node& src,
const Node& dst,
PathOptions options = PathOptions()
) const
A path from src to dst with the fewest edges within options, as nodes: {src} when the two are one node, empty when no path runs. One answer among equally short paths, the same on every call.
Declared in ClikaRT/graph/model_graph.h, line 832
weighted_shortest_path_impl()
Result<WeightedPath> weighted_shortest_path_impl(
const Node& src,
const Node& dst,
detail::ContainedEdgeCost edge_cost,
const ContainedPathOptions& options
) const
The cheapest path from src to dst within options, edge_cost summed over the edges it takes (among parallel edges, the cheapest). Every edge cost must be finite and at least 0; an empty edge_cost counts 0. Ties go to fewer edges; under a cutoff, the cheapest path of at most that many edges. Empty when no path runs, {src} at cost 0 when the two are one node. edge_cost runs at most once per edge. A negative, NaN or infinite cost fails naming the edge (Status::InvalidArgument); a throw fails with it.
Declared in ClikaRT/graph/model_graph.h, line 844
shortest_path(Node, Node, EdgeCost, PathOptions)
WeightedPath shortest_path(
const Node& src,
const Node& dst,
EdgeCost edge_cost,
PathOptions options = PathOptions()
) const
A path from src to dst with the fewest edges within options, as nodes: {src} when the two are one node, empty when no path runs. One answer among equally short paths, the same on every call.
Declared in ClikaRT/graph/model_graph.h, line 847
all_simple_paths()
std::vector<std::vector<Node>> all_simple_paths(
const Node& src,
const Node& dst,
PathOptions options = PathOptions()
) const
Every simple path from src to dst, as nodes; parallel edges give one path (networkx's all_simple_paths over the graph with parallel edges merged). src == dst gives the one-node path. The order, callables and cap of find_paths_to.
Declared in ClikaRT/graph/model_graph.h, line 859
all_simple_edge_paths()
std::vector<std::vector<Edge>> all_simple_edge_paths(
const Node& src,
const Node& dst,
PathOptions options = PathOptions()
) const
Every simple path from src to dst, as edges; each parallel edge gives its own path (networkx's all_simple_edge_paths). src == dst gives one empty path. Depth-first in out_edges() order, with the callables and cap of find_paths_to.
Declared in ClikaRT/graph/model_graph.h, line 870
count_paths()
std::uint64_t count_paths(
const Node& src,
const Node& dst,
PathOptions options = PathOptions()
) const
How many paths all_simple_paths lists, counted without listing them, so no cap applies and options.limit is not read. Fails when the count passes the range of std::uint64_t, and like nodes() on a cycle.
Declared in ClikaRT/graph/model_graph.h, line 879
critical_path()
WeightedPath critical_path(NodeCost node_cost, EdgeCost edge_cost = EdgeCost()) const
The critical path: the path of greatest cost, node_cost summed over its nodes and edge_cost over its edges, each finite and of any sign; an empty cost counts 0, so two empty costs give the path with the most edges. A path may start and end at any node: a prefix whose sum falls below 0 is dropped and the path starts again at the next node, as networkx's dag_longest_path does. Ties go to more edges, then to the path found first. Empty on a graph with no nodes. Each cost runs once per node or edge. A NaN or infinite cost fails naming the node or edge (Status::InvalidArgument), a throw fails with it, and a cyclic graph fails like nodes().
Declared in ClikaRT/graph/model_graph.h, line 894
topological_order()
std::vector<Node> topological_order(TopologicalOrder order = TopologicalOrder::Deterministic) const
Every node in the topological order order names (query.h): each node once, after every node whose output it reads. Fails like nodes() on a cycle.
Declared in ClikaRT/graph/model_graph.h, line 903
random_topological_order()
std::vector<Node> random_topological_order(std::uint64_t seed) const
A topological order drawn at random from the valid ones: each node once, after every node whose output it reads. One seed draws one order of a given graph, on every platform. Fails like nodes() on a cycle.
Declared in ClikaRT/graph/model_graph.h, line 912
topological_generations()
std::vector<std::vector<Node>> topological_generations() const
layers() under networkx's name: the same generations from the same call.
Declared in ClikaRT/graph/model_graph.h, line 918
depths()
NodeMap<std::size_t> depths() const
Each node's generation: the most edges on a path that ends at it, so depths()[n] is the index of n's layer in layers(). One entry per node, in nodes() order. Fails like nodes() on a cycle.
Declared in ClikaRT/graph/model_graph.h, line 926
is_topological_order()
bool is_topological_order(ClikaRT::Span<const Node> order) const
True when order is a topological order of this graph: every node exactly once, and every edge running forward. A node of another graph, a repeated or a missing node, or an edge running backward makes it false.
Declared in ClikaRT/graph/model_graph.h, line 932
find_cycle()
std::vector<Edge> find_cycle() const
One cycle as its edges, in order: each edge enters the node the next one leaves, and the last enters the node the first one leaves. Empty when the graph has none, as every traced or compiled graph.
Declared in ClikaRT/graph/model_graph.h, line 937
would_create_cycle()
True when an edge from src to dst would close a cycle: dst reaches src, or the two are one node.
Declared in ClikaRT/graph/model_graph.h, line 942
weakly_connected_components()
std::vector<std::vector<Node>> weakly_connected_components() const
The weakly connected components: the groups of nodes joined by edges taken either way, each in name order, the groups ordered by their first node's name.
Declared in ClikaRT/graph/model_graph.h, line 949
dominators()
Each node's immediate dominator: the last node before it that every path from the graph inputs to it crosses. Paths start at the graph inputs, so a constant or weight operand opens none. A node no other node dominates (an input, or a node where two inputs' paths meet) maps to itself, and a node on no path from an input has no entry. Entries in nodes() order. Fails like nodes() on a cycle.
Declared in ClikaRT/graph/model_graph.h, line 958
post_dominators()
Each node's immediate post-dominator: the first node after it that every path from it to the graph outputs crosses. A node no other node post-dominates maps to itself, and a node on no path to an output has no entry. Entries in nodes() order. Fails like nodes() on a cycle.
Declared in ClikaRT/graph/model_graph.h, line 965
split_points()
std::vector<Node> split_points() const
The nodes every path from the graph inputs to the graph outputs crosses, from the inputs to the outputs: where the graph splits into two stages at one node. Empty when no node is on every such path. Fails like nodes() on a cycle.
Declared in ClikaRT/graph/model_graph.h, line 972
lowest_common_ancestors()
std::vector<Node> lowest_common_ancestors(ClikaRT::Span<const Node> nodes) const
The lowest common ancestors of nodes: the nodes every one of them descends from (each counting as its own ancestor) that no other such node descends from, in nodes() order. networkx's lowest_common_ancestor returns one of them. At least one node.
Declared in ClikaRT/graph/model_graph.h, line 979
merge_point()
std::optional<Node> merge_point(ClikaRT::Span<const Node> nodes) const
Where the paths from nodes merge: the nearest node every path from them to the graph outputs crosses, each counting for itself. Empty when their paths meet only at the outputs. A node on no path to an output is not counted. At least one node.
Declared in ClikaRT/graph/model_graph.h, line 988
region()
The single-entry, single-exit region from entry to exit: the nodes entry dominates and exit post-dominates, both ends included, in nodes() order; a node alone is a region. Fails (Status::InvalidArgument) unless entry dominates exit and exit post-dominates entry, naming the condition that fails.
Declared in ClikaRT/graph/model_graph.h, line 998
between()
Every node on a path from src to dst, both ends included, in nodes() order; empty when no path runs.
Declared in ClikaRT/graph/model_graph.h, line 1005
producer_cone()
std::vector<Node> producer_cone(ClikaRT::Span<const Node> nodes) const
The producer cone of nodes: every node whose output they read, directly or through others, in nodes() order. A node of nodes counts only when another one reads it, so the cone of an input is empty.
Declared in ClikaRT/graph/model_graph.h, line 1014
consumer_cone()
std::vector<Node> consumer_cone(ClikaRT::Span<const Node> nodes) const
The consumer cone of nodes: every node that reads their outputs, directly or through others, in nodes() order; the cone of a node no other node reads is empty.
Declared in ClikaRT/graph/model_graph.h, line 1022
is_convex()
bool is_convex(ClikaRT::Span<const Node> nodes) const
True when nodes is convex: no path leaves the set and comes back into it, so the set can be cut out and replaced as one piece.
Declared in ClikaRT/graph/model_graph.h, line 1029
min_value_cut()
ValueCut min_value_cut(
ClikaRT::Span<const Node> before = {},
ClikaRT::Span<const Node> after = {}
) const
The cheapest cut through the graph's values: a split of the nodes in two, every node before the cut with its producers before it, that hands the fewest bytes from the part before to the part after (ValueCut). The graph inputs and the nodes of before run before the cut and the nodes of after after it; a graph output made before the cut is one of the values. A value weighs its tensor's bytes: a view adds nothing to the tensor it re-describes, and a constant or a state buffer is never one of the values. Of the cheapest cuts this is the one nearest the inputs: the least set of nodes before the cut. Fails (Status::InvalidArgument) when no cut exists, naming the node of after that is a graph input, is in before or feeds a node of before, and (Status::Unsupported) when a value has no fixed size in bytes (a dynamic dim), since the cut weighs every value.
Declared in ClikaRT/graph/model_graph.h, line 1047
visualize()
void visualize(std::string_view path) const
Write the compiled graph to path as an ONNX model for viewing in Netron or any ONNX graph viewer. The file holds the executable topology as compile left it: the runtime's fused and absorbed operators under their own op types, every edge as a tensor wire carrying its shape and dtype, and the graph's inputs and outputs under their names. Structure only. No weights are written, so the file is a picture of the graph, not a runnable model. path is created or overwritten.
Declared in ClikaRT/graph/model_graph.h, line 1062
export_onnx()
void export_onnx(std::string_view path, const OnnxExportOptions& options = {}) const
Write the graph to path as a runnable ONNX model. The inputs and outputs keep their names, dtypes and dims, and a dynamic dimension is written under its name, so two inputs that share a dimension still share it when the model is read back. The weights become the model's initializers, in the file or beside it as options says. They are read as the file is written, one at a time, so the export holds no second copy of them. io::OnnxModel::from_graph builds the same model in memory without writing it; io::OnnxModel::open and compile read the file back.
Refused, with nothing written:
- a graph that rewrites a cache buffer in place: one given a cache by
attach_kv_cache, or one traced over a function that writes its cache in place. Export beforeattach_kv_cache, or trace the step with the cache as an input and its update as an output; - a graph holding an operator that has no ONNX form at the chosen opset, naming the node and the operator;
- a call while
optimize,finalize,attach_kv_cache,toor an edit runs on the graph (FAILED_PRECONDITION, naming the call).
Declared in ClikaRT/graph/model_graph.h, line 1087
shape_domains()
std::vector<ShapeDomain> shape_domains() const
The shapes each input admits, input_names() order. A dimension with a fixed size is Fixed; a dynamic dimension compiled with a recorded profile (an ONNX compile given InputSpec ranges or named-dim bindings) is Range; a dynamic dimension without one is Free. See ShapeDomain (bench.h).
Declared in ClikaRT/graph/model_graph.h, line 1098
input_shape_domain()
ShapeDomain input_shape_domain(std::string_view name) const
The domain of the input named name. Raises ClikaRT::Error (NotFound) when no input carries the name.
Declared in ClikaRT/graph/model_graph.h, line 1105
validate_shapes(vector<InputShape>)
void validate_shapes(const std::vector<InputShape>& shapes) const
Check named shapes against the domains: every runtime input present once, every name an input, every rank equal to the input's, and every extent admitted (DimDomain::admits). Raises ClikaRT::Error (InvalidArgument) naming the input and the dimension that failed.
Declared in ClikaRT/graph/model_graph.h, line 1114
validate_shapes(vector<vector<int64_t>>)
void validate_shapes(const std::vector<std::vector<std::int64_t>>& shapes) const
Check named shapes against the domains: every runtime input present once, every name an input, every rank equal to the input's, and every extent admitted (DimDomain::admits). Raises ClikaRT::Error (InvalidArgument) naming the input and the dimension that failed.
Declared in ClikaRT/graph/model_graph.h, line 1121
bench(vector<InputShape>, BenchOptions)
BenchReport bench(const std::vector<InputShape>& shapes, BenchOptions options = {}) const
Time the graph on synthesized inputs of the given shapes. The shapes are validated as validate_shapes does, one tensor per input is filled per options.fill on the benchmark device, options.warmup untimed runs precede options.iterations timed ones, and the report carries every timed run's wall-clock milliseconds (each output synchronized before the clock stops), the summary statistics, the process's peak resident set and the device's memory counters. Raises ClikaRT::Error on a shape the domains reject, a run that fails, a graph that is not finalized, or one that optimize, finalize or attach_kv_cache is changing (FAILED_PRECONDITION).
Declared in ClikaRT/graph/model_graph.h, line 1137
bench(vector<vector<int64_t>>, BenchOptions)
BenchReport bench(
const std::vector<std::vector<std::int64_t>>& shapes,
BenchOptions options = {}
) const
Time the graph on synthesized inputs of the given shapes. The shapes are validated as validate_shapes does, one tensor per input is filled per options.fill on the benchmark device, options.warmup untimed runs precede options.iterations timed ones, and the report carries every timed run's wall-clock milliseconds (each output synchronized before the clock stops), the summary statistics, the process's peak resident set and the device's memory counters. Raises ClikaRT::Error on a shape the domains reject, a run that fails, a graph that is not finalized, or one that optimize, finalize or attach_kv_cache is changing (FAILED_PRECONDITION).
Declared in ClikaRT/graph/model_graph.h, line 1146
bench(NamedTensors, BenchOptions)
BenchReport bench(const NamedTensors& inputs, BenchOptions options = {}) const
Time the graph on synthesized inputs of the given shapes. The shapes are validated as validate_shapes does, one tensor per input is filled per options.fill on the benchmark device, options.warmup untimed runs precede options.iterations timed ones, and the report carries every timed run's wall-clock milliseconds (each output synchronized before the clock stops), the summary statistics, the process's peak resident set and the device's memory counters. Raises ClikaRT::Error on a shape the domains reject, a run that fails, a graph that is not finalized, or one that optimize, finalize or attach_kv_cache is changing (FAILED_PRECONDITION).
Declared in ClikaRT/graph/model_graph.h, line 1156