Skip to main content

ClikaRT

ClikaRT is the CLIKA inference runtime: a C++ library, with Python and Kotlin bindings, that loads models and runs them on CPUs, GPUs and other hardware accelerators through one public API. Link one library and include one header, or import one package, and the same code runs on every backend the runtime ships.

Why ClikaRT​

For the AI developer. The API is PyTorch-shaped C++, and the same shape reaches Python in full and Kotlin at the inference level through the bindings. Tensors move with .to(device), and the built-in operators chain the way you expect (.reshape(), .relu(), .matmul()). What PyTorch does not give you is where this runs. The same code covers CPU, CUDA, Vulkan and Metal (mobile GPU via Vulkan, Apple silicon via Metal), and the backend is picked at run time, not at build time. Library built-ins provide the convenience of PyTorch, but you can implement whatever you want, as close to the hardware as you like; the library is built with freedom as a first-class citizen. Serving, tokenizers and pipelines live in the same library, so a model checkpoint becomes a served endpoint.

For the backend engineer. No AI background is required to serve a model well. Pull a packaged model from Modelverse, load it, and hand requests to the serving runtime; batching, device placement and precision are the runtime's job, not model configuration details you need to understand. The library behaves like normal C++, which inference stacks usually do not. Integration is one find_package, one link target, C++17, and no Python in the ship path.

For the embedded developer. Small targets are a first-class platform. A complete Android deployment is the Kotlin artifact's Android AAR, roughly 140 MB to download (the core library, its Vulkan backend and the model library inside), and the same code runs on a phone's GPU, on Apple silicon and on Jetson, from the same bundle with no device-specific build. The device carries no Python and no package manager, only the libraries your binary links.

For the defense, healthcare and finance developer. Everything runs on your hardware. User data never leaves the device, and the self-contained bundle installs and builds where there is no network at all.

For the business. Nothing external to install, resolve, or keep in sync. The bundle is self-contained, and every third-party component it embeds is listed in its licenses/ directory. There is no dependency tree to audit, no third-party library update that breaks your team's product, and no copyleft surprise for legal. Hardware stays a choice, not a commitment. TensorRT runs only on NVIDIA GPUs; ClikaRT runs the same models on NVIDIA, AMD, Intel and Qualcomm GPUs and on plain CPUs, years-old hardware included. Run your state-of-the-art models on older or legacy machines to reduce inference and cloud costs.

What you get​

  • Tensors and operators. ClikaRT::Tensor plus the ClikaRT::ops library: element-wise math, reductions, convolutions, attention, indexing. This is the operator set a model needs.
  • Backends. CPU always; CUDA, Vulkan and Metal where the platform has them. Accelerator backends load on demand; unavailable ones are absent.
  • Quantized weights. Weight-only quantization schemes (nn::QLinearWoQ, GGUF block formats, FP8, NF4 and more) decode inside the kernels.
  • Model loading. safetensors, GGUF, ONNX and NumPy through ClikaRT::io; tokenizers, chat templates and processors alongside.
  • Serving. A runtime with sessions, continuous batching and pipelines; an HTTP server and client; a CLI framework.
  • Bindings. Two levels over the one C++ public surface: Python (a wheel) binds all of it; Kotlin (a desktop library and an Android AAR) binds what runs a ready model, under the model library's own names. Language bindings says what each level promises.

Where to go next​