---
title: Run an ONNX model
description: "Open an ONNX file (or build one from scratch), compile it into a ModelGraph, optimize and finalize it, and execute it by position or by name."
---
{/* CERTIFICATION: the python and C++ arms are verified against the pinned
release (tools/tutorial_check.py over the programs the page embeds:
build_onnx, inspect_onnx and run_onnx all match their recordings in both
languages, on the release's cp313 wheel and its payload). Given no path,
the C++ and Python IO-contract programs build the tiny MLP into a
temporary file, so they run with no file on disk. */}
You have a model as an `.onnx` file and want ClikaRT to run it. Three types carry the whole story. `io::OnnxModel` is the format-level object, the ONNX graph as data: open it, inspect it, edit it, save it. `compile()` is the one crossing into the executable world, where every node parses into its runtime operator, weights bind, and shapes resolve. The result is a `ModelGraph` as built. `optimize()` runs the graph optimizer when you want it, `finalize()` places the graph and packs its weights, and the finalized graph has one `run`.
The programs below first open an existing file and read its contract, then build a tiny model from scratch so the page runs with no file on disk, then compile and run it. Any `.onnx` file works in the first section; nothing depends on the architecture.
## Open a model and read its IO contract
Opening is cheap and does not compile anything: you get the graph as data. The IO contract (names, dtypes, dims, with dynamic dims reading as named placeholders or -1) is what you need to prepare feeds; print it before anything else when a model is new to you.
```cpp title="inspect_onnx.cpp (the inspection)"
int main(int argc, char** argv) {
const std::string path = argc > 1 ? argv[1] : tiny_model_path();
OnnxModel model = OnnxModel::open(path);
std::printf("nodes: %zu initializers: %zu opset: %lld\n",
model.num_nodes(), model.num_initializers(),
static_cast(model.opset()));
// Hold the returned vectors: a range-for over a temporary would iterate
// storage that is gone before the first step.
const std::vector inputs = model.inputs();
const std::vector outputs = model.outputs();
for (const auto& s : inputs)
std::printf(" input %s\n", s.name.c_str());
for (const auto& s : outputs)
std::printf(" output %s\n", s.name.c_str());
return 0;
}
```
```python title="inspect_onnx.py (the inspection)"
model = crt.io.OnnxModel.open(sys.argv[1] if len(sys.argv) > 1 else tiny_model_path())
print(f"nodes: {model.num_nodes} initializers: {model.num_initializers} "
f"opset: {model.opset}")
for name, dtype, dims in model.inputs():
print(f" input {name} {list(dims)}") # a dynamic dim reads as -1
for name, dtype, dims in model.outputs():
print(f" output {name} {list(dims)}")
```
The program opens the path on its command line; with none, `tiny_model_path()` builds the MLP of the next section into a temporary file, so the page runs with no file on disk.
## Build a graph from scratch
When there is no file yet (a test, a fixture, a tool that emits ONNX), the same object builds a graph node by node: declare inputs, add initializers (weights enter as ordinary tensors), add nodes by operator type, name the outputs. The tiny MLP here is `y = relu(X W + B)` with a dynamic batch dimension.
```cpp title="build_onnx.cpp (excerpt of the build stage)"
OnnxModel build_tiny_mlp() {
Dim batch; // one Dim object: every use is the SAME dynamic dimension
OnnxModel model = OnnxModel::create("tiny_mlp", /*opset_version=*/21);
model.add_input("X", DataType::Float32, {batch, 4});
const std::string w = model.add_initializer(
Tensor::full({4, 3}, 0.5, DataType::Float32));
const std::string b = model.add_initializer(
Tensor::full({3}, 0.25, DataType::Float32));
const std::vector mm = model.add_node("MatMul", {"X", w}, 1);
const std::vector sum = model.add_node("Add", {mm[0], b}, 1);
const std::vector y = model.add_node("Relu", {sum[0]}, 1);
model.add_output(y[0], DataType::Float32, {batch, 3});
return model;
}
```
```python title="build_onnx.py (the build stage)"
def build_tiny_mlp() -> "crt.io.OnnxModel":
model = crt.io.OnnxModel.create("tiny_mlp", opset_version=21)
model.add_input("X", crt.float32, [-1, 4]) # any value <= 0 is dynamic
w = model.add_initializer(crt.tensor(np.full((4, 3), 0.5, dtype=np.float32)))
b = model.add_initializer(crt.tensor(np.full((3,), 0.25, dtype=np.float32)))
(mm,) = model.add_node("MatMul", ["X", w])
(summed,) = model.add_node("Add", [mm, b])
(out,) = model.add_node("Relu", [summed])
model.add_output(out, crt.float32, [-1, 3])
return model
```
`save(path)` writes the graph; `open(path)` round-trips it structure-intact, and `optimize()` runs the rewrite pipeline to a fixed point (a minimal graph survives unchanged).
## Compile and run
`compile()` can fail like any load of real weights and shapes, so it returns through the error contract; branch on the code name. It returns the graph as built. `optimize()` runs the graph optimizer when you want it, and `finalize()` readies the graph to run ([Optimize and finalize](optimize-and-finalize.mdx) covers both). The finalized `ModelGraph` runs positionally (one tensor per `input_names()` entry, in that order) and, in Python, also by name.
```cpp title="run_onnx.cpp (compile and run)"
int run(ClikaRT::io::OnnxModel& model) {
ClikaRT::Result compiled = CLIKART_TRY(model.compile());
if (!compiled.ok()) {
std::printf("compile: FAILED [%s]\n", compiled.code_name().c_str());
return 1;
}
ModelGraph graph = std::move(compiled.value());
graph.optimize(); // the graph optimizer, when wanted
graph.finalize(); // only a finalized graph runs
const std::vector x = {1.0F, 2.0F, 3.0F, 4.0F, -1.0F, 0.5F, 2.0F, -2.0F};
std::vector outputs = graph.run(
{Tensor::from_data(x.data(), {2, 4}, DataType::Float32)});
std::printf("y = %s\n", outputs[0].to_string().c_str());
return 0;
}
```
```python title="run_onnx.py (compile and run)"
graph = model.compile()
graph.optimize() # the graph optimizer, when wanted
graph.finalize() # only a finalized graph runs
print(f"inputs {graph.input_names()} -> outputs {graph.output_names()}")
x = crt.tensor(np.array([[1, 2, 3, 4], [-1, 0.5, 2, -2]], dtype=np.float32))
(y,) = graph.run([x]) # positional: input_names() order
(y2,) = graph.run({"X": x}) # named: the same result
print(y.numpy())
codes = [n.op_code for n in graph.nodes()] # the compiled graph as data
assert crt.graph.OpCode.MatMul in codes
# The one-call loader: open, compile, optimize and finalize, then call the model with named inputs.
model.save("tiny_mlp.onnx")
loaded = crt.onnx.load("tiny_mlp.onnx", dynamic_axes={"X": {0: "batch"}}, device="cpu")
out = loaded(X=x) # a dict keyed by output name
assert list(out) == loaded.output_names
```
`crt.onnx.load` opens, compiles, optimizes and finalizes in one call: `dynamic_axes` names the axes that vary between calls, `device` places the weights, and the returned model takes its inputs by keyword and returns a dict keyed by output name. `optimize=False` returns the graph as built, which runs after `model.graph.finalize()`. `graph.nodes()` reads the compiled graph as data, one `crt.graph.OpCode` per node; [Trace eager code to graphs](trace-eager-code-to-graphs.mdx) walks that surface.
With the fixed all-half weights and the 0.25 bias above, the math fits in your head; the C++ program's run over the two-row input prints:
```text
y = relu(X W + B):
[5.25, 5.25, 5.25]
[0, 0, 0]
```
The first row is `(1+2+3+4) * 0.5 + 0.25 = 5.25` per output; the second row's pre-activation is negative in every column, so relu zeroes it.
Two pointers from here. A compiled `ModelGraph` is the same type that [tracing eager code](trace-eager-code-to-graphs.mdx) produces, so everything downstream of `compile()` is shared, [optimizing and finalizing](optimize-and-finalize.mdx) included. And a model too big to build by hand arrives as a file: the IO-contract section works unchanged on a checkpoint you downloaded.