---
title: Serve it
sidebar_label: 5. Serve it
description: "Wrap part 4's model in the serving runtime: a schema, a lambda serving the forward pass, and an executor answering requests."
---

Part 4 ended with a program that loads a checkpoint and computes once. This part puts the same model behind the serving runtime: declare its input/output contract, serve the forward pass with a lambda, and hand requests to an executor. The tutorial ends where the overview's pitch ends, a model checkpoint answering requests.

## The program

<LangTabs>

<LangTab value="cpp">

```cpp title="main.cpp"
#include <cstdint>
#include <cstdio>
#include <filesystem>
#include <string>

#include <ClikaRT/clika_rt.h>

namespace rt = ClikaRT::runtime;
using ClikaRT::DataType;
using ClikaRT::NamedTensors;
using ClikaRT::Tensor;
using ClikaRT::spec::TensorSpec;
namespace io = ClikaRT::io;
namespace ops = ClikaRT::ops;

namespace {

// The layer's geometry, the checkpoint's names, and the edge names the schema
// declares and every request and response addresses. A misspelled edge name
// routes nothing, silently, so each is spelled once.
constexpr std::int64_t kFeatures = 8;
constexpr std::int64_t kOutputs = 4;
constexpr std::int64_t kBatch = 2;
constexpr const char* kWeight = "mlp.weight";
constexpr const char* kBias = "mlp.bias";
constexpr const char* kX = "x";
constexpr const char* kY = "y";

}  // namespace

int main() {
    // The checkpoint from part 4, re-created so this program stands alone.
    NamedTensors weights;
    weights.set(kWeight, Tensor::full({kOutputs, kFeatures}, 0.5, DataType::Float32));
    weights.set(kBias,   Tensor::full({kOutputs}, 0.25, DataType::Float32));
    const std::string ckpt =
        (std::filesystem::temp_directory_path() / "first_program.safetensors").string();
    io::save_safetensors(weights, ckpt);

    // Load it and declare the model's I/O contract: batches of 8 features
    // in, batches of 4 activations out. kDynamicDim leaves the batch open.
    const NamedTensors loaded = io::load_safetensors(ckpt);
    const Tensor w = loaded.get(kWeight);
    const Tensor b = loaded.get(kBias);

    rt::ModelSchema schema;
    schema.inputs  = {TensorSpec{kX, DataType::Float32, {TensorSpec::kDynamicDim, kFeatures}, false}};
    schema.outputs = {TensorSpec{kY, DataType::Float32, {TensorSpec::kDynamicDim, kOutputs}, false}};

    // Part 4's forward pass, served by a lambda. No subclass needed.
    rt::FunctionModel model{schema};
    model.on_run_once("run", [&](rt::PhaseContext& ctx) {
        const Tensor x = ctx.inputs->get(kX);
        ctx.outputs->set(kY, ops::relu(ops::linear(x, w, b)));
    });

    rt::Executor exec = rt::Executor::create(model);  // borrowed; keep `model` alive

    // One request in, one response out: the serving loop in miniature.
    rt::Request req;
    req.inputs.set(kX, Tensor::ones({kBatch, kFeatures}, DataType::Float32));
    const rt::Response resp = exec.await(exec.enqueue(std::move(req)));

    std::printf("y = %s\n", resp.outputs.get(kY).to_string().c_str());
    exec.shutdown();
    return 0;
}
```

</LangTab>

<LangTab value="python">

{/* CERTIFICATION: the arm as shown is verified against the pinned release's
cp313 wheel (tools/tutorial_check.py over the program the page embeds; trace ->
optimize -> finalize -> run, matching its recording). The pipeline executor over graphs
(clika_runtime._core.runtime: Pipeline, Request, Response, enqueue/wait) is
bound but has no published example chapter in the pinned tree (chapters end at
15_trace_and_compile), so the arm stops at the finalized ModelGraph. Extend it
to the executor when the upstream chapter lands. */}

The Python lane serves a traced graph: `crt.trace` captures the forward pass as a `ModelGraph`, and after `optimize()` and `finalize()` the finalized graph is the servable unit. The pipeline executor over graphs is bound but not yet published, so this arm stops at the graph.

```python title="main.py"
import tempfile

import clika_runtime as crt
import clika_runtime.nn.functional as F


def main() -> None:
    # The checkpoint from part 4, re-created so this program stands alone.
    ckpt = f"{tempfile.gettempdir()}/first_program.safetensors"
    crt.io.save_safetensors({
        "mlp.weight": crt.full((4, 8), 0.5),
        "mlp.bias": crt.full((4,), 0.25),
    }, ckpt)

    loaded = crt.io.load_safetensors(ckpt)
    w, b = loaded["mlp.weight"], loaded["mlp.bias"]

    # The traced callable takes and returns LISTS of tensors. crt.trace runs
    # it once over data-free stand-ins and captures the operator graph, which
    # comes back as recorded: optimize() runs the graph optimizer and
    # finalize() readies the graph to serve.
    def forward(ins: list) -> list:
        return [F.relu(F.linear(ins[0], w, b))]

    g = crt.trace(forward, example_inputs=[crt.ones(2, 8)])
    g.optimize()
    g.finalize()

    # A request in, a response out.
    (y,) = g.run([crt.ones(2, 8)])
    print(f"y = {y}")


if __name__ == "__main__":
    main()
```

</LangTab>

<LangTab value="kotlin">

The Kotlin binding does not carry the serving runtime; the C++ arm is the serving story today.

</LangTab>

</LangTabs>

```text
y = Tensor(shape=[2, 4], dtype=Float32, device=CPU, numel=8, data=[4.25, 4.25, 4.25, 4.25, 4.25, 4.25, ...])
```

The same 4.25s part 4 computed, produced this time by an executor answering a request.

## Schemas, lambdas, executors

`ModelSchema` is the model's I/O contract: named `TensorSpec`s with dtype and dims, and `kDynamicDim` leaves a dimension open, so one served model accepts any batch size. `FunctionModel` serves a callable against that schema with no subclass; the lambda reads its inputs and sets its outputs through the `PhaseContext`. `Executor::create` borrows the model (keep it alive) and turns it into a queue: `enqueue` accepts a `Request`, `await` blocks for its `Response`, and `shutdown` drains the queue. Sessions, continuous batching and pipelines build on this same executor; the `runtime` [example project](/examples.md) walks each one.

## One flag for the serving runtime

The serving runtime is built without RTTI, so a target that uses `runtime::` adds one line to part 1's CMake (the bundle's own runtime examples set the same flag):

```cmake
target_compile_options(hello PRIVATE -fno-rtti)
```

Next: [part 6](06-deploy-to-mobile.mdx), the same tutorial program on a real phone.