Skip to main content

Fetch and prompt

Part 1 established what the model is; this part downloads it and makes it generate. At the end you have the weights on disk in a directory you control and a repeatable one-shot generation command.

Generation is compute, so the credential from Quick install has to be in place before the prompt command below. fetch downloads files and needs none.

Fetch the snapshot​

fetch downloads a source's files (companions first, then the weights, one progress bar per file) and prints exactly one thing on stdout: the local path the download landed at.

clika-modelverse fetch meta-llama/Llama-3.2-1B-Instruct
/home/you/.cache/huggingface/hub/models--meta-llama--Llama-3.2-1B-Instruct/snapshots/9213176726f574b556790deb65791e0c5aa438b6

Progress, timing and the file count print to stderr, so the path is safe to capture:

MODEL_DIR=$(clika-modelverse fetch meta-llama/Llama-3.2-1B-Instruct)

By default snapshots land in the standard Hugging Face Hub cache, shared with other Hugging Face tooling on the machine; --cache-dir models keeps them in a directory of your choosing instead. Fetching a source that is already cached verifies and returns immediately, so scripts can fetch unconditionally. A gated repo needs HF_TOKEN exported in the environment; there is no token flag, deliberately, so a token can never land in shell history. Llama 3.2 is gated: accept its license once on the repo's Hub page, export HF_TOKEN, and the fetch proceeds (without them it refuses by name: 'meta-llama/Llama-3.2-1B-Instruct' is gated or private on https://huggingface.co; set $HF_TOKEN).

Two related commands you already have: fetch --dry prints the total of what the fetch downloads without downloading it (info --dry adds the file listing behind that total), and a local directory used as a source skips fetching entirely.

One refusal worth meeting on purpose: fetch takes checkpoints in the formats from part 1 (safetensors, torch containers, GGUF) and refuses anything else before a byte of weights moves. An ONNX-only export, for example:

$ clika-modelverse fetch onnx-community/Llama-3.2-1B-Instruct-ONNX
error: fetch failed: 'onnx-community/Llama-3.2-1B-Instruct-ONNX' (revision 'main') ships no safetensors or torch-container (pytorch_model.bin) weights, no weight files found, one of the two is required

The first generation​

prompt runs one templated generation of the positional message: the model's own chat template wraps your text, the pipeline decodes, and the generated text is the stdout payload. The template is why a question gets an answer here: a raw, untemplated completion would CONTINUE the question's shape instead (part 4 shows that mode from the library). Prompt answers one shot; a conversation runs against the served model in part 3.

clika-modelverse meta-llama/Llama-3.2-1B-Instruct prompt "The capital of France is"
The capital of France is Paris.

The first run loads the weights (a progress bar on stderr); repeat runs on a warm cache start in seconds. The same invocation works with "$MODEL_DIR" in place of the repo id, which is the fully offline form.

The knobs are flags​

Sampling and context are controlled per invocation. The ones you will use first:

clika-modelverse meta-llama/Llama-3.2-1B-Instruct prompt "Name three rivers." \
    --max-new-tokens 32 \
    --temperature 0.2 \
    --seed 7
Here are three rivers:

1. Nile
2. Amazon
3. Yangtze
  • --max-new-tokens caps the generation length; --max-seq caps the whole context.
  • --temperature shapes sampling; --greedy disables it for the single most likely continuation, and --seed makes a sampled run repeatable.
  • A conversation's knobs move into the request once the model is served: the system message is the first messages entry, and temperature/top_p ride each OpenAI request (part 3).
  • --device cuda places the model explicitly; the default picks the best available device, and clika-modelverse devices shows the candidates. --device auto names that default: the same pick, falling back down the accelerator order (CUDA, TPU, Metal, Vulkan) when a load fails, and the CPU last. GET /props on a served model reports which device it landed on.

A vision-capable model takes media the same way. With the flagship multimodal family the message and the image travel together:

clika-modelverse google/gemma-3-4b-it prompt "What is on the sign?" --image photo.jpg

The root commands' options also have a file form: generate-template writes a JSON file of every root option at its default, --template <file> loads one, and explicit flags win over it (a family's own verbs, like prompt and serve, carry their options as flags only). Script clika-modelverse covers that workflow.

Next: part 3, the model behind an HTTP endpoint, with a conversation on its built-in chat page.