--- title: "clika_runtime.graph functions" sidebar_label: "Functions" description: "Module-level functions of clika_runtime.graph." --- ## `extract` ```python extractextract(graph: object | None, inputs: object | None, outputs: object | None) -> object ``` extract(graph: object | None, inputs: object | None, outputs: object | None) -> object extract(graph, inputs, outputs) -> ModelGraph The operators of `graph` that compute `outputs` from `inputs`, as a graph of their own; `inputs` and `outputs` are sequences of Values of `graph`. Its inputs come in the order of `inputs`, each value once: a graph input keeps its name, and any other value becomes an input named after it (its name, else '_'). Its outputs come in the order of `outputs`, each keeping its graph output name when it has one, else named after its value. A KV cache layer whose attention it holds comes with it, its cache inputs among `inputs` and its cache outputs among `outputs`. It holds its operators' weights, one copy of each, and the weights of the operators it leaves out are freed; weights stored inside an ONNX file rather than as external data share one buffer, which the new graph keeps in memory. The call consumes `graph`: it is left empty, and a view of it raises afterwards. A refusal leaves it as it was. Raises InvalidArgumentError for the refusals the runtime names (no output; a value of another graph, a constant or a state value, or one an edit removed; an operator that reads a graph input `inputs` does not list; a KV layer whose cache inputs and outputs are not listed), with the code FAILED_PRECONDITION for a graph that is finalized, left on two devices by a failed to(), or busy with another call; and TypeError for a graph that is not a ModelGraph and an entry that is not a Value. ## `merge` ```python mergemerge(parts: object | None, connections: object | None = (), *, io_names: object = 'prefix_on_collision', cross_device: object = 'move') -> tuple ``` merge(parts: object | None, connections: object | None = (), *, io_names: object = 'prefix_on_collision', cross_device: object = 'move') -> tuple merge(parts, connections=(), *, io_names='prefix_on_collision', cross_device='move') -> tuple[ModelGraph, list[Rename]] Merge graphs into one graph that runs them all. `parts` names each graph: a dict of name to ModelGraph, in its order, or a sequence of (name, graph) pairs; a name is not empty and holds no '/' and no '.'. Each connection is an (output, input) pair, each end written '.', feeding that output of one part into that input of another; parts no connection joins sit side by side. Every operator of part p is named 'p/'. The inputs and outputs no connection joins are the merged graph's, parts in the order given, named as `io_names` says: 'prefix_on_collision' prefixes 'p/' onto a name two parts both use, 'prefix_always' onto every name, and 'keep' keeps every name and refuses one two parts share. A connection between two devices moves its value onto the input's device, or refuses under cross_device='refuse'. Each mode also takes its enum member (IoNames, CrossDevice). Returns the merged graph, ready for optimize() and finalize(), and every name the merge changed (Rename). The merge consumes every part: each is left empty, and a view of one raises afterwards. A refusal leaves every part as it was. Raises InvalidArgumentError for the refusals the runtime names (no parts; a part name that is empty, used twice or holds a separator; a graph given as two parts; a connection naming a missing part, output or input; an input fed twice; a loop of parts; ends that disagree on dtype, rank or a fixed size; a name two inputs or two outputs would share), with the code FAILED_PRECONDITION for a part that is finalized, holds a KV cache or is busy; TypeError for a part that is not a ModelGraph and an argument of another shape; and ValueError for a connection end not written '.' and a mode string that names no mode. ## `split` ```python splitsplit(graph: object | None, partition: collections.abc.Callable) -> tuple ``` split(graph: object | None, partition: collections.abc.Callable) -> tuple split(graph: object | None, cut: object | None) -> tuple Overloaded function. 1. ``split(graph: object | None, partition: collections.abc.Callable) -> tuple`` split(graph, partition) -> tuple[dict[str, ModelGraph], list[tuple[str, str]]] Split `graph` into parts, `partition` naming each operator's part: it takes the operator's Node and returns its part name, a str that is not empty and holds no '/' and no '.'. It is called once per operator, in nodes() order, on the calling thread. Inside it the graph's reads and queries work, while a run, an edit, optimize(), finalize(), to(), a merge, a split or an extract of the graph is refused with the code FAILED_PRECONDITION. Returns the parts, a dict of name to ModelGraph in an order every connection runs forward in (ties go to the part whose first operator comes first), and the connections that join them, (output, input) pairs written '.': what merge() takes, so merge(*split(graph, partition)) gives back a graph with the source's inputs and outputs. A value read in a part other than its producer's is an output there and an input of each reading part, under one name (its name, else '_'). A graph input goes with the first part that reads it, and every later reader receives it through a connection; the graph's own inputs and outputs keep their names. Each KV cache layer goes with the part of its attention, and each part holds its operators' weights, one copy of each. The call consumes `graph`: it is left empty, and a view of it raises afterwards. A refusal leaves it as it was, and an exception `partition` raises leaves split() as that same exception. Raises InvalidArgumentError for the refusals the runtime names (a part name that is empty or holds a '/' or a '.'; a weight or a state buffer two parts would use; parts that feed each other in a loop; a KV layer across two parts; an output that returns a graph input or a constant no operator computes), with the code FAILED_PRECONDITION for a graph that is finalized, left on two devices by a failed to(), or busy with another call; and TypeError for a graph that is not a ModelGraph and a part name that is not a str. 2. ``split(graph: object | None, cut: object | None) -> tuple`` split(graph, cut) -> tuple[dict[str, ModelGraph], list[tuple[str, str]]] Split `graph` in two at `cut`, a ValueCut (what min_value_cut() returns) or a sequence of Values of `graph`: the part 'before' holds the operators that produce the cut values and every operator they depend on, and the part 'after' every other operator. The values 'after' reads from 'before' cross through connections: the cut values it reads and the graph inputs both parts read. The rest is split(graph, partition)'s: the parts and connections it returns, the consumption and the refusals. Also raises InvalidArgumentError for a value of another graph, a constant or a state value, or one an edit removed; a cut that is not closed (an operator before it hands a value that is not a cut value to an operator after it); and a cut with no operator on one side; and TypeError for a cut of another type.