Skip to main content

Run an ONNX model

You have a model as an .onnx file and want ClikaRT to run it. Three types carry the whole story. io::OnnxModel is the format-level object, the ONNX graph as data: open it, inspect it, edit it, save it. compile() is the one crossing into the executable world, where every node parses into its runtime operator, weights bind, and shapes resolve. The result is a ModelGraph, the executable graph, with one run.

The programs below first open an existing file and read its contract, then build a tiny model from scratch so the page runs with no file on disk, then compile and run it. Any .onnx file works in the first section; nothing depends on the architecture.

Open a model and read its IO contract​

Opening is cheap and does not compile anything: you get the graph as data. The IO contract (names, dtypes, dims, with dynamic dims reading as named placeholders or -1) is what you need to prepare feeds; print it before anything else when a model is new to you.

inspect_onnx.cpp
#include <cstdio>
#include <string>

#include "ClikaRT/clika_rt.h"

using ClikaRT::io::OnnxModel;
using ClikaRT::unwrap;

int main(int argc, char** argv) {
    const std::string path = argc > 1 ? argv[1] : "model.onnx";
    OnnxModel model = unwrap(OnnxModel::open(path));

    std::printf("nodes: %zu  initializers: %zu  opset: %lld\n",
                model.num_nodes(), model.num_initializers(),
                (long long)model.opset());
    for (const auto& s : unwrap(model.inputs()))
        std::printf("  input  %s\n", s.name.c_str());
    for (const auto& s : unwrap(model.outputs()))
        std::printf("  output %s\n", s.name.c_str());
    return 0;
}

Build a graph from scratch​

When there is no file yet (a test, a fixture, a tool that emits ONNX), the same object builds a graph node by node: declare inputs, add initializers (weights enter as ordinary tensors), add nodes by operator type, name the outputs. The tiny MLP here is y = relu(X W + B) with a dynamic batch dimension.

build_onnx.cpp (excerpt of the build stage)
#include <string>
#include <vector>

#include "ClikaRT/clika_rt.h"

using ClikaRT::DataType;
using ClikaRT::Tensor;
using ClikaRT::spec::Dim;
using ClikaRT::io::OnnxModel;
using ClikaRT::unwrap;

OnnxModel build_tiny_mlp() {
    Dim batch;  // one Dim object: every use is the SAME dynamic dimension
    OnnxModel model = unwrap(OnnxModel::create("tiny_mlp", /*opset_version=*/21));
    unwrap(model.add_input("X", DataType::Float32, {batch, 4}));

    const std::string w = unwrap(model.add_initializer(
        Tensor::full({4, 3}, 0.5, DataType::Float32)));
    const std::string b = unwrap(model.add_initializer(
        Tensor::full({3}, 0.25, DataType::Float32)));

    const std::vector<std::string> mm  = unwrap(model.add_node("MatMul", {"X", w}, 1));
    const std::vector<std::string> sum = unwrap(model.add_node("Add", {mm[0], b}, 1));
    const std::vector<std::string> y   = unwrap(model.add_node("Relu", {sum[0]}, 1));
    unwrap(model.add_output(y[0], DataType::Float32, {batch, 3}));
    return model;
}

save(path) writes the graph; open(path) round-trips it structure-intact, and optimize() runs the rewrite pipeline to a fixed point (a minimal graph survives unchanged).

Compile and run​

compile() can fail like any load of real weights and shapes, so it returns through the error contract; branch on the code name. The ModelGraph runs positionally (one tensor per input_names() entry, in that order) and, in Python, also by name.

run_onnx.cpp (compile and run)
#include <cstdio>
#include <vector>

#include "ClikaRT/clika_rt.h"

using ClikaRT::DataType;
using ClikaRT::Tensor;
using ClikaRT::graph::ModelGraph;
using ClikaRT::unwrap;

int run(ClikaRT::io::OnnxModel& model) {
    ClikaRT::Result<ModelGraph> compiled = CLIKART_TRY(model.compile());
    if (!compiled.ok()) {
        std::printf("compile: FAILED [%s]\n", compiled.code_name().c_str());
        return 1;
    }
    ModelGraph graph = std::move(compiled.value());

    const std::vector<float> x = {1.0F, 2.0F, 3.0F, 4.0F, -1.0F, 0.5F, 2.0F, -2.0F};
    std::vector<Tensor> outputs = unwrap(graph.run(
        {Tensor::from_data(x.data(), {2, 4}, DataType::Float32)}));
    std::printf("y = %s\n", outputs[0].to_string().c_str());
    return 0;
}

With the fixed all-half weights and the 0.25 bias above, the math fits in your head; the C++ program's run over the two-row input prints:

y = relu(X W + B):
  [5.25, 5.25, 5.25]
  [0, 0, 0]

The first row is (1+2+3+4) * 0.5 + 0.25 = 5.25 per output; the second row's pre-activation is negative in every column, so relu zeroes it.

Two pointers from here. A compiled ModelGraph is the same runnable type that tracing eager code produces, so everything downstream of compile() is shared. And a model too big to build by hand arrives as a file: the IO-contract section works unchanged on a checkpoint you downloaded.