Skip to main content

Run fully offline

Modelverse needs a network exactly once per model, and not necessarily on the machine that runs it. A local directory is a first-class source everywhere a repo id is, and an offline switch turns any accidental network touch into an error. This is the deployment shape for air-gapped and restricted networks: the release archive installs by extraction alone, no package manager, and the models arrive the same way the archive did.

Stage on a connected machine​

Fetch into a directory you control (not the shared cache), so the result is a self-contained folder:

clika-modelverse fetch meta-llama/Llama-3.2-1B-Instruct --cache-dir staged
staged/models--meta-llama--Llama-3.2-1B-Instruct/snapshots/9213176726f574b556790deb65791e0c5aa438b6

The payload path on stdout is the directory to ship. fetch --dry first shows what will be downloaded and how big it is; for a GGUF source, select the quantization at staging time (--weights Q6_K) so only the variant you deploy moves. A gated repo needs HF_TOKEN exported on this machine only; the token never travels with the files.

Move it, run it​

Copy the directory however files reach the target (rsync over an approved channel, physical media; the cp below stands in for that transfer). The cache uses the Hugging Face hub layout, so capture the payload path instead of assuming it; fetching an already-cached source verifies and returns the same path immediately, which makes the capture free. On the offline machine, the copied directory is the source:

SNAPSHOT=$(clika-modelverse fetch meta-llama/Llama-3.2-1B-Instruct --cache-dir staged)
cp -r "$SNAPSHOT" ./Llama-3.2-1B-Instruct
clika-modelverse ./Llama-3.2-1B-Instruct prompt "The capital of France is"

A local directory passes through resolution untouched, so info, prompt, serve and the library's snapshot call all accept it identically. No part of model resolution knows or cares that the machine has no route out. The license credential is the one thing that does, and the section below is that one thing.

Make offline a guarantee​

Trust but verify: add --offline and any operation that would touch the network fails with a readable error instead of hanging on a dead route:

clika-modelverse ./Llama-3.2-1B-Instruct serve --offline

--offline also goes before the command, where it covers the commands that name no model: clika-modelverse --offline info <source>, --offline fetch, --offline list. The environment forms hold the same guarantee process-wide, useful under systemd or in a container where flags are out of reach: CLIKA_MODELVERSE_OFFLINE=1 (Modelverse's own switch) or HF_HUB_OFFLINE=1 (honored for compatibility with other Hugging Face Hub tooling). With the switch set, a repo-id source still works when the snapshot is already in the cache; resolution is cache-only. A cached GGUF repository resolves its identity offline from its own weight file, so a GGUF source needs no hub metadata on the target either.

The credential has to be the offline kind​

The runtime runs compute under a license, so the target machine needs a credential like any other (Get Modelverse). The kind matters here and nowhere else. An offline license is verified locally against Clika's signature and needs no route out, which is what makes it the one to ship to an air-gapped target. An online license needs a route to the platform, so a machine with no route cannot use one. Ask your platform for an offline license, and carry it to the target the way you carry the model directory.

export CLIKA_RT_LICENSE=/srv/clika/clikart.license   # the credential text, or a file holding it
clika-modelverse ./Llama-3.2-1B-Instruct prompt "The capital of France is" --offline

bin/clikart-license-init <credential> stores it once under the user account instead, which suits a service account that starts without a shell profile. An offline license carries an end date, and Runtime licenses covers what happens on it and how to re-issue.

What to verify on the target​

Three commands, no network, in order: clika-modelverse devices proves the runtime loads and sees the hardware, clika-modelverse info ./<model-dir> proves the model resolves, and a one-line prompt proves end to end, the credential included. If the documentation should travel too, the docs site you are reading has a self-contained offline bundle (the "Offline Docs" button in the navigation bar).

Sizing the model to the offline hardware is the usual question in these deployments; Model requirements and Run a specific GGUF quantization together answer it.