Skip to main content

ClikaRT

ClikaRT is the CLIKA inference runtime: a C++ library, with Python, Kotlin, Go, Rust and C bindings, that loads models and runs them on CPUs, GPUs and other hardware accelerators through one public API. Link one library and include one header, or import one package, and the same code runs on every backend the runtime ships.

Why ClikaRT​

For the AI developer. The API is PyTorch-shaped C++, and the same shape reaches Python, Kotlin, Go and Rust through the bindings, and C through the C ABI. Tensors move with .to(device), and the built-in operators chain the way you expect (.reshape(), .relu(), .matmul()). What PyTorch does not give you is where this runs. The same code covers CPU, CUDA, Vulkan and Metal (mobile GPU via Vulkan, Apple silicon via Metal), and the backend is picked at run time, not at build time. Library built-ins provide the convenience of PyTorch, but you can implement whatever you want, as close to the hardware as you like; the library is built with freedom as a first-class citizen. Serving, tokenizers and pipelines live in the same library, so a model checkpoint becomes a served endpoint.

For the backend engineer. No AI background is required to serve a model well. Pull a packaged model from Modelverse, load it, and hand requests to the serving runtime; batching, device placement and precision are the runtime's job, not model configuration details you need to understand. The library behaves like normal C++, which inference stacks usually do not. Integration is one find_package, one link target, C++17, and no Python in the ship path.

For the embedded developer. Small targets are a first-class platform. A complete Android deployment is roughly 72 MB (the core library plus its Vulkan backend), and the same code runs on a phone's GPU, on Apple silicon and on Jetson, from the same bundle with no device-specific build. The device carries no Python and no package manager, only the libraries your binary links.

For the defense, healthcare and finance developer. Everything runs on your hardware. User data never leaves the device, and the self-contained bundle installs and builds where there is no network at all.

For the business. Nothing external to install, resolve, or keep in sync. The bundle is self-contained, and every third-party component it embeds is listed in its licenses/ directory. There is no dependency tree to audit, no third-party library update that breaks your team's product, and no copyleft surprise for legal. Hardware stays a choice, not a commitment. TensorRT runs only on NVIDIA GPUs; ClikaRT runs the same models on NVIDIA, AMD, Intel and Qualcomm GPUs and on plain CPUs, years-old hardware included. Run your state-of-the-art models on older or legacy machines to reduce inference and cloud costs.

What you get​

  • Tensors and operators. ClikaRT::Tensor plus the ClikaRT::ops library: element-wise math, reductions, convolutions, attention, indexing. This is the operator set a model needs.
  • Backends. CPU always; CUDA, Vulkan and Metal where the platform has them. Accelerator backends load on demand; unavailable ones are absent.
  • Quantized weights. Weight-only quantization schemes (nn::QLinearWoQ, GGUF block formats, FP8, NF4 and more) decode inside the kernels.
  • Model loading. safetensors, GGUF, ONNX and NumPy through ClikaRT::io; tokenizers, chat templates and processors alongside.
  • Serving. A runtime with sessions, continuous batching and pipelines; an HTTP server and client; a CLI framework.
  • Bindings. Python (a wheel), Kotlin (a desktop library and an Android AAR), Go, Rust and C99 over one C ABI; each carries the tensor, operator, loading and readback surface the how-to pages show per language.

Where to go next​