Skip to main content

Fetch and prompt

Part 1 established what the model is; this part downloads it and makes it generate. At the end you have the weights on disk in a directory you control and a repeatable one-shot generation command.

Fetch the snapshot​

fetch downloads a source's files (companions first, then the weights, one progress bar per file) and prints exactly one thing on stdout: the local path the download landed at.

clika-modelverse fetch meta-llama/Llama-3.2-1B-Instruct
/home/you/.cache/huggingface/hub/models--meta-llama--Llama-3.2-1B-Instruct/snapshots/9213176726f574b556790deb65791e0c5aa438b6

Progress, timing and the file count print to stderr, so the path is safe to capture:

MODEL_DIR=$(clika-modelverse fetch meta-llama/Llama-3.2-1B-Instruct)

By default snapshots land in the standard Hugging Face Hub cache, shared with other Hugging Face tooling on the machine; --cache-dir models keeps them in a directory of your choosing instead. Fetching a source that is already cached verifies and returns immediately, so scripts can fetch unconditionally. A gated repo needs HF_TOKEN exported in the environment; there is no token flag, deliberately, so a token can never land in shell history. Llama 3.2 is gated: accept its license once on the repo's Hub page, export HF_TOKEN, and the fetch proceeds (without them it refuses by name: 'meta-llama/Llama-3.2-1B-Instruct' is gated or private on https://huggingface.co; set $HF_TOKEN).

Two related commands you already have: fetch --dry prints the total of what the fetch downloads without downloading it (info --dry adds the file listing behind that total), and a local directory used as a source skips fetching entirely.

One refusal worth meeting on purpose: fetch takes checkpoints in the formats from part 1 (safetensors, torch containers, GGUF) and refuses anything else before a byte of weights moves. An ONNX-only export, for example:

$ clika-modelverse fetch onnx-community/Llama-3.2-1B-Instruct-ONNX
error: fetch failed: 'onnx-community/Llama-3.2-1B-Instruct-ONNX' (revision 'main') ships no safetensors or torch-container (pytorch_model.bin) weights, no weight files found, one of the two is required

The first generation​

prompt runs one templated generation of the positional message: the model's own chat template wraps your text, the pipeline decodes, and the generated text is the stdout payload. The template is why a question gets an answer here: a raw, untemplated completion would CONTINUE the question's shape instead (part 4 shows that mode from the library). Prompt answers one shot; a conversation runs against the served model in part 3.

clika-modelverse meta-llama/Llama-3.2-1B-Instruct prompt "The capital of France is"
The capital of France is Paris.

The first run loads the weights (a progress bar on stderr); repeat runs on a warm cache start in seconds. The same invocation works with "$MODEL_DIR" in place of the repo id, which is the fully offline form.

The knobs are flags​

Sampling and context are controlled per invocation. The ones you will use first:

clika-modelverse meta-llama/Llama-3.2-1B-Instruct prompt "Name three rivers." \
    --max-new-tokens 32 \
    --temperature 0.2 \
    --seed 7
Here are three rivers:

1. Nile
2. Amazon
3. Yangtze
  • --max-new-tokens caps the generation length; --max-seq caps the whole context.
  • --temperature shapes sampling; --greedy disables it for the single most likely continuation, and --seed makes a sampled run repeatable.
  • A conversation's knobs move into the request once the model is served: the system message is the first messages entry, and temperature/top_p ride each OpenAI request (part 3).
  • --device cuda places the model explicitly; the default picks the best available device, and clika-modelverse devices shows the candidates.

A vision-capable model takes media the same way. With the flagship multimodal family the message and the image travel together:

clika-modelverse google/gemma-3-4b-it prompt "What is on the sign?" --image photo.jpg

The root commands' options also have a file form: generate-template writes a JSON file of every root option at its default, --template <file> loads one, and explicit flags win over it (a family's own verbs, like prompt and serve, carry their options as flags only). Script clika-modelverse covers that workflow.

Next: part 3, the model behind an HTTP endpoint, with a conversation on its built-in chat page.