Skip to main content

Make it embed

Part 2's family resolves but cannot run. For an embedding family the whole running contract is one small interface, EmbeddingBackend (clika_modelverse/models/base/embedding_model.h): token ids in, embedding rows out, plus the width, the pooling declaration and the device. This part implements the smallest real backend, wires the factory, and opts into the standard commands.

The smallest real backend​

The toy backend embeds each text as the mean of its tokens' embedding-table rows, L2-normalized. That is a real embedding model (the bag-of-words baseline retrieval systems start from), and it needs exactly one weight:

my_model_backend.cpp
#include "clika_modelverse/models/base/embedding_model.h"

namespace {
using namespace clika_modelverse;

class MyModelBackend final : public EmbeddingBackend {
 public:
    MyModelBackend(ClikaRT::Tensor table, ClikaRT::Device device)
        : table_(std::move(table)), device_(device) {}

    ClikaRT::Tensor embed_ids(
        const std::vector<std::vector<std::int32_t>>& sequences) const override {
        std::vector<ClikaRT::Tensor> rows;
        rows.reserve(sequences.size());
        for (const std::vector<std::int32_t>& ids : sequences) {
            const ClikaRT::Tensor picked = ClikaRT::ops::index_select(table_, 0, ids);
            rows.push_back(ClikaRT::ops::mean(picked, /*dim=*/0));
        }
        return ClikaRT::ops::l2_normalize(ClikaRT::ops::stack(rows), /*dim=*/-1);
    }

    std::int64_t embedding_dim() const override { return table_.shape()[1]; }
    Pooling pooling() const override { return Pooling::Mean; }
    bool normalizes() const override { return true; }
    ClikaRT::Device device() const override { return device_; }

 private:
    ClikaRT::Tensor  table_;
    ClikaRT::Device  device_;
};
}  // namespace

A real encoder runs its layers between the lookup and the pooling; where those layers go, and how a checkpoint's names bind onto them, is the full contract's territory. The interface does not change with the depth: however sophisticated the encoder, it enters the family as this same EmbeddingBackend. One optional declaration to know: max_concurrent_sessions() says how many requests one handle's forward may run at once, and its default of 1 is the safe answer for a backend that keeps per-request state; a forward that is a pure function of its inputs and weights can declare itself unbounded.

The registration delta​

The factory reads the snapshot, builds the backend from the one weight it uses, and wraps it with the snapshot's tokenizer; the registration gains the factory and the command surface:

ClikaRT::Result<EmbeddingModel> build_my_model(const std::string& dir,
                                               const Model& identity,
                                               const LoadOptions& options) {
    auto weights = ClikaRT::io::load_safetensors(dir + "/model.safetensors");
    auto table   = weights.get("embeddings.word_embeddings.weight").to(options.device);
    auto backend = std::make_unique<MyModelBackend>(std::move(table), options.device);
    auto tokenizer = ClikaRT::tokenizer::from_huggingface(dir);
    return EmbeddingModel(std::move(backend), std::move(tokenizer),
                          std::string(identity.family()),
                          std::string(identity.model_type()));
}

// The command surface, composed from the same public builders every
// embedding family uses: `similar` and `serve`, two lines.
FamilyApp build_my_model_cli(const FamilyCliContext& context) {
    FamilyApp fam{batteries::cli::make_family_app(
                      context, "commands the 'my-model' family provides"), {}};
    batteries::cli::similar_command(fam, context);
    batteries::cli::serve_command(fam, context,
                                  batteries::serving::generic_openai_server);
    return fam;
}
        .cli = CliSurface{build_my_model_cli, my_model_cli_verbs()},
        .factory = make_model<MyModel>(),
        .build_embedding = build_my_model,

Run it​

Part 1's main works again unchanged (it never named a family, only a directory), now through your own:

wrote /tmp/my_first_model
embeddings: Tensor(shape=[3, 32], dtype=Float32, device=CPU:0, numel=96, ...)
close pair  (a b c ~ a b d): 0.667
far pair    (a b c ~ f g):   -0.041

The close pair scores closer than in part 1, and honestly so: mean-pooled bags of shared words ARE similar, which is exactly what this backend measures. And because the commands came from the shared battery, your family now serves them like any catalog family: similar compares texts from the command line, and serve answers POST /v1/embeddings and POST /v1/similarity (the route table).

Where the full contract lives​

Everything real that the toy skipped is registration fields on the same struct, documented in Register your own model family and field-by-field in clika_modelverse/models/registration.h: encoders with layers and their weight binding, the generative, speech and reranking factories with their GGUF twins, probe_snapshot for repos without a config.json, and custom verb surfaces. For custom logic AROUND a catalog model rather than a model of your own, Add your own node to a model pipeline is the shorter road.