---
title: Fetch and prompt
sidebar_label: 2. Fetch and prompt
description: "Download the model snapshot, generate your first text, and control sampling from the command line."
---

Part 1 established what the model is; this part downloads it and makes it generate. At the end you have the weights on disk in a directory you control and a repeatable one-shot generation command.

Generation is compute, so the credential from [Quick install](../installation.md#2-place-the-license-credential) has to be in place before the `prompt` command below. `fetch` downloads files and needs none.

## Fetch the snapshot

`fetch` downloads a source's files (companions first, then the weights, one progress bar per file) and prints exactly one thing on stdout: the local path the download landed at.

```bash
clikart-cli fetch Qwen/Qwen2.5-0.5B-Instruct
```

```text
/home/you/.cache/huggingface/hub/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/7ae557604adf67be50417f59c2c2f167def9a775
```

Progress, timing and the file count print to stderr, so the path is safe to capture:

```bash
MODEL_DIR=$(clikart-cli fetch Qwen/Qwen2.5-0.5B-Instruct)
```

By default snapshots land in the standard Hugging Face Hub cache, shared with other Hugging Face tooling on the machine; `--cache-dir models` keeps them in a directory of your choosing instead. Fetching a source that is already cached verifies and returns immediately, so scripts can fetch unconditionally. A gated repo needs a token. The CLI reads it the way the other Hugging Face tooling does: `HF_TOKEN` in the environment, else the token a Hub login stored (`hf auth login` writes it to `$HF_HOME/token`; `HF_TOKEN_PATH` names another file). There is no token flag, deliberately, so a token can never land in shell history. Qwen2.5 0.5B Instruct is not gated, so its fetch needs no token; for a gated repo, accept its license once on the repo's Hub page, provide a token, and the fetch proceeds (without them it refuses by name: `'<org>/<repo>' is gated or private on https://huggingface.co; set $HF_TOKEN or log in to the hub`).

Two related commands you already have: `fetch --dry` prints the total of what the fetch downloads without downloading it (`info --dry` adds the file listing behind that total), and a local directory used as a source skips fetching entirely.

One refusal worth meeting on purpose: `fetch` takes checkpoints in the formats from part 1 (safetensors, torch containers, GGUF) and refuses anything else before a byte of weights moves. An ONNX-only export, for example:

```text
$ clikart-cli fetch onnx-community/Qwen2-0.5B-Instruct-ONNX
error: 'onnx-community/Qwen2-0.5B-Instruct-ONNX' ships its weights only in formats the loaders do not read: ONNX (onnx/model.onnx, onnx/model_bnb4.onnx, onnx/model_fp16.onnx and 5 more). The loaders read safetensors, GGUF and the torch container
```

## The first generation

`prompt` runs one templated generation of the positional message: the model's own chat template wraps your text, the pipeline decodes, and the generated text is the stdout payload. The template is why a question gets an answer here: a raw, untemplated completion would CONTINUE the question's shape instead (part 4 shows that mode from the library). Prompt answers one shot; a conversation runs against the served model in part 3.

```bash
clikart-cli Qwen/Qwen2.5-0.5B-Instruct prompt "The capital of France is"
```

```text
Paris.
```

The first run loads the weights (a progress bar on stderr); repeat runs on a warm cache start in seconds. The same invocation works with `"$MODEL_DIR"` in place of the repo id, which is the fully offline form.

## The knobs are flags

Sampling and context are controlled per invocation. The ones you will use first:

```bash
clikart-cli Qwen/Qwen2.5-0.5B-Instruct prompt "Name three rivers." \
    --max-new-tokens 32 \
    --temperature 0.2 \
    --seed 7
```

```text
Three rivers that I can name for you are the Yangtze River, the Yellow River, and the Pearl River. Each of these rivers is significant in Chinese
```

- `--max-new-tokens` caps the generation length; `--max-seq` caps the whole context.
- `--temperature` shapes sampling; `--greedy` disables it for the single most likely continuation, and `--seed` makes a sampled run repeatable.
- A conversation's knobs move into the request once the model is served: the system message is the first `messages` entry, and `temperature`/`top_p` ride each OpenAI request ([part 3](03-chat-and-serve.md)).
- `--device cuda` places the model explicitly; the default picks the best available device, and `clikart-cli devices` shows the candidates. `--device auto` names that default: the same pick, falling back down the accelerator order (CUDA, TPU, Metal, Vulkan) when a load fails, and the CPU last. `GET /props` on a served model reports which device it landed on.

A vision-capable model takes media the same way. With the flagship multimodal family the message and the image travel together:

```bash
clikart-cli google/gemma-4-E2B-it prompt "What is on the sign?" --image photo.jpg
```

The root commands' options also have a file form: `generate-template` writes a JSON file of every root option at its default, `--template <file>` loads one, and explicit flags win over it (a family's own verbs, like `prompt` and `serve`, carry their options as flags only). [Script clikart-cli](../../how-to/script-the-cli.md) covers that workflow.

Next: [part 3](03-chat-and-serve.md), the model behind an HTTP endpoint, with a conversation on its built-in chat page.