A model from scratch
This series is for adding your own model, as opposed to running the catalog's. It walks the same ground the first tutorial covered from the consumer side, now from the author's side, in three parts with a running result each. The worked model is an embedding model on purpose: text goes in, one vector comes out, and there is no machinery beyond that idea. This part demystifies the checkpoint itself: a model is a directory of files, and you can write one by hand.
The program below synthesizes a tiny embedding checkpoint (an 8-word vocabulary, hidden size 32, a gemma3-shaped text encoder), loads it through the registry, and compares three texts. Nothing downloads and the weights are random; random weights still embed, they are only bad at it, and that is enough to see every file a model needs and where each one enters. Same two-file CMake project as part 4 of the first tutorial; only main.cpp changes.
The files an embedding model needs
config.jsonnames the architecture (model_type,architectures) and its geometry (layers, heads, hidden size). Identity resolution reads exactly this.tokenizer.jsonturns text into token ids; the toy one below is a word-level vocabulary of eight entries.- The weights (
model.safetensorshere) are tensors under the names the architecture expects. - An embedding checkpoint additionally ships its module chain, the sentence-transformers layout:
modules.jsonlists the stages (transformer, pooling, dense heads, normalize), and each configurable stage carries its own small directory.
The program
#include <cstdio>
#include <filesystem>
#include <fstream>
#include <random>
#include <string>
#include <vector>
#include <ClikaRT/clika_rt.h>
#include "clika_modelverse/modules/pooling.h"
#include "clika_modelverse/registry/model_registry.h"
namespace fs = std::filesystem;
namespace mv = clika_modelverse;
using ClikaRT::DataType;
using ClikaRT::Tensor;
// The whole identity: model_type and architectures are what the registry
// matches; the rest is geometry the loader shapes the encoder from.
constexpr const char* kConfigJson = R"({
"model_type": "gemma3_text",
"architectures": ["Gemma3TextModel"],
"hidden_size": 32,
"num_hidden_layers": 4,
"num_attention_heads": 4,
"num_key_value_heads": 2,
"head_dim": 8,
"vocab_size": 8,
"intermediate_size": 64,
"max_position_embeddings": 128,
"rms_norm_eps": 1e-6,
"rope_theta": 1000000.0,
"rope_local_base_freq": 10000.0,
"sliding_window": 16,
"sliding_window_pattern": 2,
"query_pre_attn_scalar": 8,
"use_bidirectional_attention": true
})";
// A word-level tokenizer: eight words, one id each.
constexpr const char* kTokenizerJson = R"({
"version": "1.0",
"pre_tokenizer": {"type": "WhitespaceSplit"},
"model": {"type": "WordLevel",
"vocab": {"a": 0, "b": 1, "c": 2, "d": 3, "e": 4, "f": 5, "g": 6, "[eos]": 7},
"unk_token": "a"},
"decoder": null,
"added_tokens": [{"id": 7, "content": "[eos]", "special": true, "normalized": false}]
})";
// The module chain: transformer -> mean pooling -> two dense heads -> normalize.
constexpr const char* kModulesJson = R"([
{"idx": 0, "name": "0", "path": "", "type": "sentence_transformers.models.Transformer"},
{"idx": 1, "name": "1", "path": "1_Pooling", "type": "sentence_transformers.models.Pooling"},
{"idx": 2, "name": "2", "path": "2_Dense", "type": "sentence_transformers.models.Dense"},
{"idx": 3, "name": "3", "path": "3_Dense", "type": "sentence_transformers.models.Dense"},
{"idx": 4, "name": "4", "path": "4_Normalize", "type": "sentence_transformers.models.Normalize"}
])";
constexpr const char* kPoolingJson = R"({
"word_embedding_dimension": 32,
"pooling_mode_cls_token": false,
"pooling_mode_mean_tokens": true,
"pooling_mode_max_tokens": false,
"pooling_mode_mean_sqrt_len_tokens": false,
"pooling_mode_weightedmean_tokens": false,
"pooling_mode_lasttoken": false
})";
Tensor randn(std::mt19937& rng, std::vector<std::int64_t> shape) {
std::int64_t n = 1;
for (const std::int64_t d : shape) n *= d;
std::normal_distribution<float> dist(0.0f, 0.2f);
std::vector<float> host(static_cast<std::size_t>(n));
for (float& v : host) v = dist(rng);
return Tensor::from_data(host.data(), shape, DataType::Float32);
}
// One dense head: its own directory with a config and one weight.
void write_dense(std::mt19937& rng, const fs::path& dir, std::int64_t in,
std::int64_t out) {
fs::create_directories(dir);
std::ofstream(dir / "config.json")
<< R"({"in_features": )" << in << R"(, "out_features": )" << out
<< R"(, "bias": false, "activation_function": "torch.nn.modules.linear.Identity"})";
ClikaRT::NamedTensors sd;
sd.set("linear.weight", randn(rng, {out, in}));
ClikaRT::io::save_safetensors(sd, (dir / "model.safetensors").string());
}
// Random weights under the names the gemma3-shaped encoder expects (bare
// keys, no prefix); the fixed seed keeps every run identical.
fs::path write_snapshot() {
const fs::path dir = fs::temp_directory_path() / "my_first_model";
fs::create_directories(dir);
std::ofstream(dir / "config.json") << kConfigJson;
std::ofstream(dir / "tokenizer.json") << kTokenizerJson;
std::ofstream(dir / "modules.json") << kModulesJson;
fs::create_directories(dir / "1_Pooling");
std::ofstream(dir / "1_Pooling" / "config.json") << kPoolingJson;
constexpr std::int64_t kVocab = 8, kHidden = 32, kLayers = 4, kHeads = 4;
constexpr std::int64_t kKvHeads = 2, kHeadDim = 8, kFfn = 64;
constexpr std::int64_t kDenseMid = 48, kDim = 16;
std::mt19937 rng(20260831);
ClikaRT::NamedTensors sd;
const auto put = [&](const std::string& name, std::vector<std::int64_t> shape) {
sd.set(name, randn(rng, std::move(shape)));
};
put("embed_tokens.weight", {kVocab, kHidden});
put("norm.weight", {kHidden});
for (std::int64_t l = 0; l < kLayers; ++l) {
const std::string p = "layers." + std::to_string(l) + ".";
put(p + "self_attn.q_proj.weight", {kHeads * kHeadDim, kHidden});
put(p + "self_attn.k_proj.weight", {kKvHeads * kHeadDim, kHidden});
put(p + "self_attn.v_proj.weight", {kKvHeads * kHeadDim, kHidden});
put(p + "self_attn.o_proj.weight", {kHidden, kHeads * kHeadDim});
put(p + "self_attn.q_norm.weight", {kHeadDim});
put(p + "self_attn.k_norm.weight", {kHeadDim});
put(p + "mlp.gate_proj.weight", {kFfn, kHidden});
put(p + "mlp.up_proj.weight", {kFfn, kHidden});
put(p + "mlp.down_proj.weight", {kHidden, kFfn});
put(p + "input_layernorm.weight", {kHidden});
put(p + "post_attention_layernorm.weight", {kHidden});
put(p + "pre_feedforward_layernorm.weight", {kHidden});
put(p + "post_feedforward_layernorm.weight", {kHidden});
}
ClikaRT::io::save_safetensors(sd, (dir / "model.safetensors").string());
write_dense(rng, dir / "2_Dense", kHidden, kDenseMid);
write_dense(rng, dir / "3_Dense", kDenseMid, kDim);
return dir;
}
int main() {
// 1. A checkpoint is a directory; this one is yours, written just now.
const fs::path dir = write_snapshot();
std::printf("wrote %s\n", dir.string().c_str());
// 2. The registry reads config.json, matches architecture
// "Gemma3TextModel" to the gemma-embedding family, and returns the
// runnable model, exactly as for a fetched checkpoint.
mv::EmbeddingModel model =
mv::ModelRegistry::builtin().load_embedding(dir.string(), {});
// 3. Embed three texts: one [3, 16] Float32 tensor, one row per text
// (the module chain pools, projects to 16 dims, and normalizes).
const std::string texts[] = {"a b c", "a b d", "f g"};
const Tensor rows = model.embed(texts);
std::printf("embeddings: %s\n", rows.to_string().c_str());
// 4. Compare them: the cosine matrix is unit rows times their own
// transpose. The explicit normalize keeps the demo self-contained
// (the chain's Normalize stage already produced unit rows).
const Tensor unit = mv::modules::l2_normalize_rows(rows);
const Tensor sim = ClikaRT::ops::matmul(unit, unit, {}, std::nullopt,
false, /*transpose_b=*/true);
const std::vector<float> s = ClikaRT::ops::reshape(sim, {9}).item_as_vec<float>();
std::printf("close pair (a b c ~ a b d): %.3f\n", s[1]);
std::printf("far pair (a b c ~ f g): %.3f\n", s[2]);
return 0;
}
cmake -S . -B build -DModelverse_DIR="$MODELVERSE_INSTALL_DIR/cmake"
cmake --build build
./build/hello
[2026-09-08 06:40:23.915] [modelverse] [info] hub: '/tmp/my_first_model' weights at /tmp/my_first_model
[2026-09-08 06:40:23.917] [gemma-embedding] [info] embedding encoder on CPU:-1 (dim 16)
wrote /tmp/my_first_model
embeddings: Tensor(shape=[3, 16], dtype=Float32, device=CPU, numel=48, data=[0.03671, -0.04182, -0.2858, 0.1808, 0.1262, 0.399, ...])
close pair (a b c ~ a b d): 0.400
far pair (a b c ~ f g): -0.324
Random weights, so the numbers mean little; what matters is the shape of what happened. A directory you wrote from scratch went through the same registry, loader and embedding surface as a fetched checkpoint would, and three texts became three vectors you can compare, because a checkpoint is nothing more than these files.
What the registry did with it
config.json's model_type and architectures are the identity keys; the architecture Gemma3TextModel matched the built-in gemma-embedding family, and that family's loader shaped the encoder from the geometry fields and bound your model.safetensors names to it. The tokenizer file turned each text into ids before the encoder saw them, and modules.json told the loader what follows the encoder: mean pooling, the two dense projections, and the final normalize. Delete the directory and nothing else remembers it.
Your own architecture will not say "architectures": ["Gemma3TextModel"], and then the match fails; that refusal, and fixing it by registering a family of your own, is part 2.