ClikaRT at a glance

ClikaRT is CLIKA's inference runtime, a C++ library (with Python, Kotlin, Go, C99, and Rust bindings) that loads models and runs them on CPUs, GPUs, and other hardware accelerators through one public API. It is not a training framework, and not a bundle of separate CUDA, Vulkan and Metal wrappers. You link one library, include one header, and the same code runs on every backend the runtime ships.

The mental model

Five ideas carry the whole library, in the order you meet them when you build.

1. Everything arrives in one bundle. A directory with the public headers, the libraries per platform, a CMake package and the examples. find_package(ClikaRT CONFIG) and the target ClikaRT::ClikaRT are the entire integration.

2. Models load from files you already have. ClikaRT::io reads safetensors, GGUF and NumPy checkpoints, plus images and audio, and a model checkpoint arrives as a name -> tensor map on the device you name.

   C++

   const NamedTensors weights = io::load_safetensors("model.safetensors");
   const Tensor w = weights.get("layer.weight");

   C

   /* api: the handshaken table from clika_rt_get_api(CLIKA_RT_ABI_VERSION); dflt: the default placement. */
   clika_rt_tensors_container* weights = NULL;
   check(api->load_safetensors_to_tensors_container("model.safetensors", 17,
                                                    dflt, 0, &weights), "load");
   clika_rt_tensor* w = NULL;
   check(api->tensors_container_get(weights, "layer.weight", 12, &w), "get");

   Python

   weights = crt.io.load_safetensors("model.safetensors")
   w = weights["layer.weight"]

   Kotlin

   The Kotlin binding reaches checkpoint loading through its generated low-level tier today (ClikaRtGen.loadSafetensorsToTensorsContainer); tensors and operators carry the idiomatic Kotlin surface.

   Go

   weights := must(api.LoadSafetensorsToTensorsContainer("model.safetensors",
       clikart.StreamOrDevice{}, false))
   w := must(api.TensorsContainerGet(weights, "layer.weight"))

   Rust

   // dflt: the default placement (the runtime picks device and stream).
   let weights = api.load_safetensors_to_tensors_container("model.safetensors", dflt, false);
   let w = api.tensors_container_get(&weights, "layer.weight");

3. Tensor is the core building block. Each tensor is assigned to a device (the CPU by default; .to(device) moves it), and ops:: operators run where their inputs live. The CPU backend is always present; CUDA, Vulkan and Metal load at run time where the machine has them. Everything returns values directly and raises ClikaRT::Error on failure.

   C++

   const Tensor a = Tensor::ones({2, 3}, DataType::Float32);
   const Tensor b = ops::add(a, a);
   std::printf("%s\n", b.to_string().c_str());

   C

   const int64_t shape[2] = {2, 3};
   clika_rt_tensor* a = NULL;
   check(api->tensor_ones(shape, 2, CLIKA_RT_DATA_TYPE_FLOAT32, dflt, dflt, &a), "ones");
   
   api->tensor_retain(a);      /* consumed as the operand AND borrowed as the rhs */
   clika_rt_scalar_or_tensor rhs = {.kind = CLIKA_RT_SCALAR_OR_TENSOR_TENSOR, .tensor = a};
   clika_rt_tensor* b = NULL;
   check(api->op_add_tensor(a, rhs, 1.0, CLIKA_RT_ACTIVATION_IDENTITY, &b), "add");
   api->tensor_release(a);     /* the reference retained above */
   
   char buf[256];
   size_t len = sizeof buf;
   api->tensor_to_string(b, buf, &len);
   printf("%s\n", buf);

   Python

   a = crt.ones(2, 3)
   b = a + a
   print(b)

   Kotlin

   val a = Tensors.ones(longArrayOf(2, 3))
   val b = a + a  // operator extensions return a fresh tensor; operands stay live
   println(b.summary())

   Go

   // must: the tutorial's unwrap helper; wrappers are non-consuming.
   a := must(api.TensorOnes([]int64{2, 3}, clikart.Float32,
       clikart.StreamOrDevice{}, clikart.StreamOrDevice{}))
   b := must(a.AddTensor(clikart.ScalarOrTensor{
       Kind: clikart.ScalarOrTensorTensor, Tensor: a,
   }, 1.0, clikart.ActivationIdentity))
   fmt.Println(b)

   Rust

   let a = api.tensor_full(&[2, 3], 1.0, F32)?;  // a ones tensor via the full factory
   let b = &a + &a;  // operators borrow; explicit calls consume their tensor args
   println!("{}", api.tensor_to_string(&b)?);

4. Execution is asynchronous by nature. An ops:: call dispatches work and returns; reads wait for the result, so you never observe unfinished bytes.

5. Serving is built in. The serving runtime adds sessions, continuous batching and pipelines, so the model you loaded answers requests. Declare a schema, serve it with a lambda, drive it through an executor:

   C++

   rt::FunctionModel model{schema};
   model.on_run_once("run", [](rt::PhaseContext& ctx) {
       ctx.outputs->set("y", ctx.inputs->get("x") * 2.0);
   });
   
   rt::Executor exec = rt::Executor::create(model);
   const rt::Response resp = exec.await(exec.enqueue(std::move(req)));

   C

   The C core surface runs compiled graphs (the model_graph_run_vector family); the FunctionModel and Executor family is C++-only today.

   Python

   The Python lane serves a traced graph: crt.trace captures a forward pass as a runnable ModelGraph, and the pipeline executor over graphs is bound but not yet published. Part 5 walks it.

   Kotlin

   The Kotlin binding does not carry the serving runtime; the C++ arm is the serving story today.

   Go

   The Go module rides the C core surface, which runs compiled graphs; the serving runtime stays on the C++ side.

   Rust

   The Rust crate does not carry the serving runtime; the C++ arm is the serving story today.

The first program series turns these into working programs, explaining each where it first appears; the serving runtime has its own example project.

Platforms

Linux (x86_64, arm64), Android (arm64), Windows (x86_64, arm64) and macOS (Apple silicon), one distribution per platform and architecture, and a distribution works out of the box on every machine of its class: the linux-arm64 dist runs on a Jetson the same way it runs on an arm64 server. System requirements has the platform and accelerator tables.