Model requirements
What a model needs to run is mostly decided before you fetch it: the variant and quantization fix the weight size, and the context length fixes the working memory on top. This page carries the figures for the models the documentation uses and the classes of hardware they fit; clika-modelverse info <source> --dry gives the on-disk number for any source, and the bench command measures the rest on your machine.
How to read the tables: Disk is the snapshot size (fetch --dry total). Run is the resident memory serving one session at the default context; longer contexts and concurrent sessions add KV cache on top, roughly linear in tokens. Fits names the smallest sensible device class: phone (4 GB), laptop or small board (8 GB), workstation GPU (8 GB VRAM and up).
Text generation
| Model | Variant | Disk | Run | Fits |
|---|---|---|---|---|
| Llama 3.2 1B Instruct | fp16 (safetensors) | 2.3 GB | 3.0 GB | laptop |
| Llama 3.2 1B Instruct | GGUF Q6_K | 1.0 GB | 1.6 GB | phone |
| Llama 3.1 8B Instruct | GGUF Q4_K_M | 4.9 GB | 6.1 GB | workstation GPU, 8 GB |
| Qwen3 0.6B | fp16 (safetensors) | 1.4 GB | 1.9 GB | phone |
| Qwen3 4B | GGUF Q6_K | 3.3 GB | 4.3 GB | laptop |
| Qwen3 4B | GGUF Q8_0 | 4.3 GB | 5.4 GB | workstation GPU, 8 GB |
The quantization ladder trades memory for fidelity in predictable steps: Q8_0 is near-fp16, Q6_K is the everyday default, Q4_K_M is the last stop before quality degrades noticeably in chat use. --kv-quant shrinks the per-token cache the same way when long contexts dominate the budget.
Multimodal
| Model | Variant | Disk | Run | Fits |
|---|---|---|---|---|
| Gemma 3 4B IT (vision) | fp16 (safetensors) | 8.6 GB | 10.2 GB | workstation GPU, 12 GB |
| Gemma 3 4B IT (vision) | GGUF Q6_K | 3.9 GB | 5.6 GB | laptop |
A multimodal run holds the vision tower alongside the language model; the difference between the text-only 4B rows above and these is that tower.
Speech-to-text
| Model | Variant | Disk | Run | Fits |
|---|---|---|---|---|
| Whisper base | fp16 (safetensors) | 0.3 GB | 0.6 GB | phone |
| Whisper large-v3-turbo | fp16 (safetensors) | 1.6 GB | 2.2 GB | laptop |
| Whisper large-v3 | fp16 (safetensors) | 3.1 GB | 4.0 GB | laptop |
Transcription memory is stable with audio length (the model works in 30-second windows); throughput, not memory, is what a faster device buys.
Embeddings
| Model | Variant | Disk | Run | Fits |
|---|---|---|---|---|
| Qwen3 Embedding 0.6B | GGUF Q8_0 | 0.7 GB | 1.1 GB | phone |
Embedding models run one forward per input with no generation loop, so batch size is the only memory knob that matters in practice.
Reading a figure you do not see here
For any other model: info --dry prints the exact disk cost before anything moves, devices prints what this machine offers, and a Run estimate is the disk figure plus 25 to 40 percent for activations and the default context. When the estimate is close to the device's limit, measure with bench before committing the deployment; it reports peak resident memory along with throughput.