Use it from your code

Everything the clikart-cli executable did in parts 1 through 3 is a library call. This part writes the smallest program that does what prompt does: resolve a snapshot, load the runnable model, and generate.

The project

For C++, the install directory you already have is also the SDK (the headers, the library, and the ClikaRT runtime beside them): the platform's one archive for your platform carries the runtime and Modelverse together (Get Modelverse), so there is no second download. A two-file CMake project is the whole setup. The C++17 toolchain from the ClikaRT quick install prerequisites is the one requirement.

CMakeLists.txt

cmake_minimum_required(VERSION 3.19)
project(hello_modelverse LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(Modelverse CONFIG REQUIRED)

add_executable(hello main.cpp)
target_link_libraries(hello PRIVATE Modelverse::modelverse)

# Copy the Modelverse library and the ClikaRT runtime next to the binary,
# so the program runs from the build directory as-is.
modelverse_stage_runtime(TARGET hello)

find_package(Modelverse CONFIG) wires everything: it defines the one link target (Modelverse::modelverse), and finds the ClikaRT runtime installed in the same root (a ClikaRT::ClikaRT you already provide is respected instead).

Python arrives in the clika-runtime wheel, which carries the model library as clika_runtime.modelverse; Kotlin arrives in the one Maven artifact, io.clika:clika-runtime, which carries the model library as io.clika.modelverse (the artifact's zip, extracted, is the Maven repository the build names). Both are downloads of the platform (Get ClikaRT):

pip install ./clika_runtime-0.6.4-cp313-cp313-manylinux_2_28_x86_64.whl   # Python: the wheel, the model library inside
# Kotlin/Gradle: implementation("io.clika:clika-runtime:0.6.4")

The program

C++

main.cpp

#include <cstdio>

#include "clika_modelverse/hub/hub.h"
#include "clika_modelverse/registry/model_registry.h"
#include "clika_modelverse/runtime/serving_pipeline.h"
#include "clika_modelverse/runtime/text_nodes.h"

namespace mv  = clika_modelverse;
namespace mvr = clika_modelverse::runtime;

int main() {
    // 1. Model files, exactly as `fetch` gets them. An already-fetched or
    //    local directory passes through untouched.
    mv::hub::SnapshotOptions snap;
    snap.cache_dir = "models";  // beside the program; empty = the shared Hugging Face cache
    const mv::hub::SnapshotResult snapped = mv::hub::snapshot(
        "Qwen/Qwen2.5-0.5B-Instruct", snap);

    // 2. The registry matches the snapshot to its family and returns the
    //    runnable model, on the device you name (CPU by default).
    mv::LoadOptions load;
    load.max_seq = 4096;  // the CLI's default context cap; 0 keeps the
                          // checkpoint's full window and sizes the KV cache from it
    mv::GenerativeModel model =
        mv::ModelRegistry::builtin().load_generative(snapped.local_dir, load);

    // 3. A serving pipeline around it: tokenizer -> decoder -> detokenizer,
    //    the same assembly `prompt` and `serve` run on.
    mv::generation::GenerationConfig gen = model.defaults();
    gen.max_new_tokens = 64;
    gen.stop = {"."};  // raw completion: stop at the first sentence end
    mvr::PipelineOptions opts;
    mvr::GenerativePipeline pipe = mvr::build_generative_pipeline(model, gen, opts);

    // 4. One request through it.
    ClikaRT::runtime::Request req;
    req.inputs.set("prompt", mvr::string_to_byte_tensor(
        "Once upon a time, in a port town by a cold sea,"));
    const auto sid = pipe.pipeline->enqueue(std::move(req));
    const ClikaRT::runtime::Response resp = pipe.pipeline->await(sid);

    std::printf("%s\n", mvr::byte_tensor_to_string(resp.outputs.get("text")).c_str());
    return 0;
}

Python

main.py

import clika_runtime.modelverse as mv

# 1. and 2. Model files and the runnable model in one call: the snapshot
#    downloads into the hub cache (a local directory passes through untouched),
#    the registry matches it to its family and loads it on the device named.
model = mv.AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct", device="cpu")

# 3. and 4. The generation: text in, text out; the keyword arguments are the
#    decode knobs over the checkpoint's own defaults.
print(model.generate("Once upon a time, in a port town by a cold sea,", max_new_tokens=64, stop=["."]))

The model served in part 3 answers any OpenAI client as well (clikart-cli <model> serve, then a client pointed at http://127.0.0.1:8000/v1), which is the path for a program that must not load the model itself.

Kotlin

Chat.kt

import io.clika.modelverse.AutoModelForCausalLM
import io.clika.modelverse.LoadOptions
import io.clika.modelverse.Modelverse

// On a worker thread of the app, never the main thread.
fun reply(context: android.content.Context, license: String): String {
    // 0. The runtime, the model library and the bridge, once per process,
    //    with the license credential placed before any model runs.
    Modelverse.load(context, license)

    // 1. and 2. Model files and the runnable model in one call: a hub
    //    repository id downloads into the app's cache (a snapshot directory
    //    or a .gguf file passes through untouched), and the registry loads
    //    it on the CPU. The context length sizes the key-value cache for a phone.
    val model = AutoModelForCausalLM.fromPretrained(
        "Qwen/Qwen2.5-0.5B-Instruct",
        LoadOptions(contextLength = 4096),
    )

    // 3. and 4. The generation: text in, text out, blocking for the reply;
    //    chat(messages, config, listener) is the streaming form.
    return model.use { it.generate(prompt = "Once upon a time, in a port town by a cold sea,") }
}

Build and run the C++ project:

cmake -S . -B build -DModelverse_DIR="$MODELVERSE_INSTALL_DIR/cmake"
cmake --build build
./build/hello

 there lived a young woman named Akira.

The raw pipeline CONTINUES text; there is no chat template in the loop, which is the visible difference from prompt in part 2 (templated, answers a question). That is why this program feeds it a story opener and stops at the first sentence end: give a raw completion a question and it rambles on in the question's shape. A program that answers a question renders the conversation through the model's chat template first, as the executable does; the next section is that step.

Answer a question

A chat model's prompt format ships with the model as a template, and the tokenizer renders a messages array in the OpenAI API's shape into the exact prompt string; Tokenize and chat templates is the contract. Never hand-build the role markers.

C++

Tokenizer::apply_chat_template renders the messages. In the program above, the rendered string takes the story opener's place as the prompt input, and the . stop goes, so the pipeline answers as the assistant.

chat_template.cpp

int main() {
    Tokenizer tok = Tokenizer::from_huggingface(model_dir());

    Json messages = Json::array();
    Json system = Json::object();
    system["role"] = "system";
    system["content"] = "You are a concise assistant.";
    messages.push_back(std::move(system));
    Json user = Json::object();
    user["role"] = "user";
    user["content"] = "What does a tokenizer do?";
    messages.push_back(std::move(user));

    const std::string prompt = tok.apply_chat_template(messages);
    std::printf("=== rendered prompt ===\n%s\n=======================\n", prompt.c_str());

    // The template writes the prompt's own bos/eos framing, so encode_chat adds
    // no special tokens on top of it.
    const std::vector<std::int32_t> ids = tok.encode_chat(messages, /*add_generation_prompt=*/true);
    std::printf("encode_chat produced %zu tokens\n", ids.size());
    return 0;
}

Python

chat renders and generates in one call, and render is the rendered prompt alone; the conversation is yours to carry, so append the reply and the next call sees the whole history.

chat_messages.py

def main() -> None:
    model = mv.AutoModelForCausalLM.from_pretrained(resolve_model(), device="cpu", offline=True)

    messages = [
        {"role": "system", "content": "You answer in one short sentence."},
        {"role": "user", "content": "Why is the sky blue?"},
    ]

    # A checkpoint without a chat template reads a plain prompt instead, so a
    # program that may meet either one says which it got.
    if not model.has_chat_template:
        print("BLOCKED: this checkpoint ships no chat template")
        sys.exit(3)

    # render is the prompt text the template produced, the same string
    # apply_chat_template returns on the transformers side.
    print(f"=== rendered prompt ===\n{model.render(messages)}\n=======================")

    reply = model.chat(messages, max_new_tokens=32, temperature=0.0)
    print(f"=== reply ===\n{reply}\n=============")

    # The conversation is the caller's: append the reply and the next call
    # carries the whole history.
    messages.append({"role": "assistant", "content": reply})
    followup = model.chat(messages + [{"role": "user", "content": "In fewer words?"}],
                          max_new_tokens=32, temperature=0.0)
    print(f"=== follow-up ===\n{followup}\n=================")

    assert reply.strip(), "the first reply is empty"
    assert followup.strip(), "the follow-up reply is empty"
    assert messages[-1]["role"] == "assistant", "the history did not gain the assistant turn"


if __name__ == "__main__":
    main()

What the four steps are

The program is the executable's anatomy laid bare, and each step is independently useful:

• The snapshot call resolves any source (a Hugging Face Hub repo id, URL, or local directory) to a directory of model files. Point it at a directory you shipped with your application and no network code is ever built in.
• The registry load is the catalog from part 1 as a function: identity match, factory, weights onto the device. Where the weights load is LoadOptions::where, a Stream or a Device: a Device (or nothing; the device field then decides) loads through a fresh stream of the model's own, a Stream loads through yours, behind whatever it already carries. The load synchronizes the weights, so their stream stops mattering once it returns; each inference call then runs on the stream its input tensors arrive on (or the calling thread's stream when they carry none), which is what lets two models fed from two streams overlap. The CLI's --device fills the same option.
• The pipeline build assembles the serving pipeline. It is a ClikaRT serving-runtime pipeline underneath, so sessions, continuous batching and the async model documented there apply unchanged.
• The request is the generation loop. serve from part 3 is this loop behind HTTP; the library's ServeApi lets you mount the same engines, or your own, in-process.

Failures follow the runtime's error model per language: C++ returns values directly and raises ClikaRT::Error, Python raises clika_runtime.ClikaRTError with the same code name, and Kotlin throws ModelverseException with its codeName; a served model reports a failure as the HTTP status the client raises.

You have taken a model from the catalog to your own binary. What to read next.