---
title: Use it from your code
sidebar_label: 4. Use it from your code
description: "Link the Modelverse library and run the same model inside your own program: snapshot, load, pipeline, text."
---

{/* CERTIFICATION: the C++ sample compiles and runs against the pinned release bundle (linux_x86_64, CPU); the output block is that run's capture. load.max_seq mirrors the CLI's 4096 default (the library otherwise keeps the checkpoint's full window and sizes the KV cache from it). The Kotlin arm is a staged binding surface and carries its own in-tab marker. */}

Everything the `clikart-cli` executable did in parts 1 through 3 is a library call. This part writes the smallest program that does what `prompt` does: resolve a snapshot, load the runnable model, and generate.

## The project

For C++, the install directory you already have is also the SDK (the headers, the library, and the ClikaRT runtime beside them): the platform's one archive for your platform carries the runtime and Modelverse together ([Get Modelverse](../get-modelverse.md)), so there is no second download. A two-file CMake project is the whole setup. The C++17 toolchain from the [ClikaRT quick install](/clikart/getting-started/installation) prerequisites is the one requirement.

```cmake title="CMakeLists.txt"
cmake_minimum_required(VERSION 3.19)
project(hello_modelverse LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(Modelverse CONFIG REQUIRED)

add_executable(hello main.cpp)
target_link_libraries(hello PRIVATE Modelverse::modelverse)

# Copy the Modelverse library and the ClikaRT runtime next to the binary,
# so the program runs from the build directory as-is.
modelverse_stage_runtime(TARGET hello)
```

`find_package(Modelverse CONFIG)` wires everything: it defines the one link target (`Modelverse::modelverse`), and finds the ClikaRT runtime installed in the same root (a `ClikaRT::ClikaRT` you already provide is respected instead).

Python arrives in the `clika-runtime` wheel, which carries the model library as `clika_runtime.modelverse`; Kotlin arrives in the one Maven artifact, `io.clika:clika-runtime`, which carries the model library as `io.clika.modelverse` (the artifact's zip, extracted, is the Maven repository the build names). Both are downloads of the platform ([Get ClikaRT](/clikart/getting-started/get-clikart)):

```bash
pip install ./clika_runtime-0.6.4-cp313-cp313-manylinux_2_28_x86_64.whl   # Python: the wheel, the model library inside
# Kotlin/Gradle: implementation("io.clika:clika-runtime:0.6.4")
```

## The program

<LangTabs>

<LangTab value="cpp">

```cpp title="main.cpp"
#include <cstdio>

#include "clika_modelverse/hub/hub.h"
#include "clika_modelverse/registry/model_registry.h"
#include "clika_modelverse/runtime/serving_pipeline.h"
#include "clika_modelverse/runtime/text_nodes.h"

namespace mv  = clika_modelverse;
namespace mvr = clika_modelverse::runtime;

int main() {
    // 1. Model files, exactly as `fetch` gets them. An already-fetched or
    //    local directory passes through untouched.
    mv::hub::SnapshotOptions snap;
    snap.cache_dir = "models";  // beside the program; empty = the shared Hugging Face cache
    const mv::hub::SnapshotResult snapped = mv::hub::snapshot(
        "Qwen/Qwen2.5-0.5B-Instruct", snap);

    // 2. The registry matches the snapshot to its family and returns the
    //    runnable model, on the device you name (CPU by default).
    mv::LoadOptions load;
    load.max_seq = 4096;  // the CLI's default context cap; 0 keeps the
                          // checkpoint's full window and sizes the KV cache from it
    mv::GenerativeModel model =
        mv::ModelRegistry::builtin().load_generative(snapped.local_dir, load);

    // 3. A serving pipeline around it: tokenizer -> decoder -> detokenizer,
    //    the same assembly `prompt` and `serve` run on.
    mv::generation::GenerationConfig gen = model.defaults();
    gen.max_new_tokens = 64;
    gen.stop = {"."};  // raw completion: stop at the first sentence end
    mvr::PipelineOptions opts;
    mvr::GenerativePipeline pipe = mvr::build_generative_pipeline(model, gen, opts);

    // 4. One request through it.
    ClikaRT::runtime::Request req;
    req.inputs.set("prompt", mvr::string_to_byte_tensor(
        "Once upon a time, in a port town by a cold sea,"));
    const auto sid = pipe.pipeline->enqueue(std::move(req));
    const ClikaRT::runtime::Response resp = pipe.pipeline->await(sid);

    std::printf("%s\n", mvr::byte_tensor_to_string(resp.outputs.get("text")).c_str());
    return 0;
}
```

</LangTab>

<LangTab value="python">

```python title="main.py"
import clika_runtime.modelverse as mv

# 1. and 2. Model files and the runnable model in one call: the snapshot
#    downloads into the hub cache (a local directory passes through untouched),
#    the registry matches it to its family and loads it on the device named.
model = mv.AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct", device="cpu")

# 3. and 4. The generation: text in, text out; the keyword arguments are the
#    decode knobs over the checkpoint's own defaults.
print(model.generate("Once upon a time, in a port town by a cold sea,", max_new_tokens=64, stop=["."]))
```

The model served in part 3 answers any OpenAI client as well (`clikart-cli <model> serve`, then a client pointed at `http://127.0.0.1:8000/v1`), which is the path for a program that must not load the model itself.

</LangTab>

<LangTab value="kotlin">

{/* CERTIFICATION: the Kotlin arm shows the Android binding's shape as the pinned release's Kotlin package
declares it (Modelverse.load, AutoModelForCausalLM.fromPretrained, LoadOptions, generate); it is not compiled
by the docs build. */}

```kotlin title="Chat.kt"
import io.clika.modelverse.AutoModelForCausalLM
import io.clika.modelverse.LoadOptions
import io.clika.modelverse.Modelverse

// On a worker thread of the app, never the main thread.
fun reply(context: android.content.Context, license: String): String {
    // 0. The runtime, the model library and the bridge, once per process,
    //    with the license credential placed before any model runs.
    Modelverse.load(context, license)

    // 1. and 2. Model files and the runnable model in one call: a hub
    //    repository id downloads into the app's cache (a snapshot directory
    //    or a .gguf file passes through untouched), and the registry loads
    //    it on the CPU. The context length sizes the key-value cache for a phone.
    val model = AutoModelForCausalLM.fromPretrained(
        "Qwen/Qwen2.5-0.5B-Instruct",
        LoadOptions(contextLength = 4096),
    )

    // 3. and 4. The generation: text in, text out, blocking for the reply;
    //    chat(messages, config, listener) is the streaming form.
    return model.use { it.generate(prompt = "Once upon a time, in a port town by a cold sea,") }
}
```

</LangTab>

</LangTabs>

Build and run the C++ project:

```bash
cmake -S . -B build -DModelverse_DIR="$MODELVERSE_INSTALL_DIR/cmake"
cmake --build build
./build/hello
```

```text
 there lived a young woman named Akira.
```

The raw pipeline CONTINUES text; there is no chat template in the loop, which is the visible difference from `prompt` in part 2 (templated, answers a question). That is why this program feeds it a story opener and stops at the first sentence end: give a raw completion a question and it rambles on in the question's shape. A program that answers a question renders the conversation through the model's chat template first, as the executable does; the next section is that step.

## Answer a question

A chat model's prompt format ships with the model as a template, and the tokenizer renders a `messages` array in the OpenAI API's shape into the exact prompt string; [Tokenize and chat templates](/clikart/how-to/tokenize-and-chat-templates#render-a-conversation-with-the-models-chat-template) is the contract. Never hand-build the role markers.

<LangTabs>

<LangTab value="cpp">

`Tokenizer::apply_chat_template` renders the messages. In the program above, the rendered string takes the story opener's place as the `prompt` input, and the `.` stop goes, so the pipeline answers as the assistant.

```cpp title="chat_template.cpp"
int main() {
    Tokenizer tok = Tokenizer::from_huggingface(model_dir());

    Json messages = Json::array();
    Json system = Json::object();
    system["role"] = "system";
    system["content"] = "You are a concise assistant.";
    messages.push_back(std::move(system));
    Json user = Json::object();
    user["role"] = "user";
    user["content"] = "What does a tokenizer do?";
    messages.push_back(std::move(user));

    const std::string prompt = tok.apply_chat_template(messages);
    std::printf("=== rendered prompt ===\n%s\n=======================\n", prompt.c_str());

    // The template writes the prompt's own bos/eos framing, so encode_chat adds
    // no special tokens on top of it.
    const std::vector<std::int32_t> ids = tok.encode_chat(messages, /*add_generation_prompt=*/true);
    std::printf("encode_chat produced %zu tokens\n", ids.size());
    return 0;
}
```

</LangTab>

<LangTab value="python">

`chat` renders and generates in one call, and `render` is the rendered prompt alone; the conversation is yours to carry, so append the reply and the next call sees the whole history.

```python title="chat_messages.py"
def main() -> None:
    model = mv.AutoModelForCausalLM.from_pretrained(resolve_model(), device="cpu", offline=True)

    messages = [
        {"role": "system", "content": "You answer in one short sentence."},
        {"role": "user", "content": "Why is the sky blue?"},
    ]

    # A checkpoint without a chat template reads a plain prompt instead, so a
    # program that may meet either one says which it got.
    if not model.has_chat_template:
        print("BLOCKED: this checkpoint ships no chat template")
        sys.exit(3)

    # render is the prompt text the template produced, the same string
    # apply_chat_template returns on the transformers side.
    print(f"=== rendered prompt ===\n{model.render(messages)}\n=======================")

    reply = model.chat(messages, max_new_tokens=32, temperature=0.0)
    print(f"=== reply ===\n{reply}\n=============")

    # The conversation is the caller's: append the reply and the next call
    # carries the whole history.
    messages.append({"role": "assistant", "content": reply})
    followup = model.chat(messages + [{"role": "user", "content": "In fewer words?"}],
                          max_new_tokens=32, temperature=0.0)
    print(f"=== follow-up ===\n{followup}\n=================")

    assert reply.strip(), "the first reply is empty"
    assert followup.strip(), "the follow-up reply is empty"
    assert messages[-1]["role"] == "assistant", "the history did not gain the assistant turn"


if __name__ == "__main__":
    main()
```

</LangTab>

</LangTabs>

## What the four steps are

The program is the executable's anatomy laid bare, and each step is independently useful:

- **The snapshot call** resolves any source (a Hugging Face Hub repo id, URL, or local directory) to a directory of model files. Point it at a directory you shipped with your application and no network code is ever built in.
- **The registry load** is the catalog from part 1 as a function: identity match, factory, weights onto the device. Where the weights load is `LoadOptions::where`, a `Stream` or a `Device`: a Device (or nothing; the `device` field then decides) loads through a fresh stream of the model's own, a Stream loads through yours, behind whatever it already carries. The load synchronizes the weights, so their stream stops mattering once it returns; each inference call then runs on the stream its input tensors arrive on (or the calling thread's stream when they carry none), which is what lets two models fed from two streams overlap. The CLI's `--device` fills the same option.
- **The pipeline build** assembles the serving pipeline. It is a [ClikaRT serving-runtime](/clikart) pipeline underneath, so sessions, continuous batching and the async model documented there apply unchanged.
- **The request** is the generation loop. `serve` from part 3 is this loop behind HTTP; the library's `ServeApi` lets you mount the same engines, or your own, in-process.

Failures follow the runtime's error model per language: C++ returns values directly and raises `ClikaRT::Error`, Python raises `clika_runtime.ClikaRTError` with the same code name, and Kotlin throws `ModelverseException` with its `codeName`; a served model reports a failure as the HTTP status the client raises.

You have taken a model from the catalog to your own binary. [What to read next](../next-steps.md).