Skip to main content

Add a model

A benchmark measures a model. This part registers one. It takes about ten seconds, and the reason it is that fast is worth understanding.

Register the model​

Open Models and select Add Model.

The Models list with the Add Model action

Paste a Hugging Face model URL or its owner/repo identifier. An access token is only needed for a gated or private repository; public ones need nothing.

The Add HuggingFace model dialog, with a model URL field and an optional access token

For a first run, pick something small so the whole loop finishes in minutes rather than hours. Qwen/Qwen2.5-0.5B-Instruct and meta-llama/Llama-3.2-1B-Instruct are both reasonable first models on a workstation-class device.

What registration did, and what it did not​

The platform read the repository's metadata and stored a reference: the canonical URL, the task, and the architecture and framework where the metadata names them. No weights moved. You can register a 70B model from a laptop, because nothing is downloaded until a run needs it, and then it is downloaded by the device that will run it.

The task is the field that does real work. It is the model's Hugging Face pipeline_tag, for example text-generation, automatic-speech-recognition or image-text-to-text, and it decides which quality tests can run against the model in the next part. If the metadata does not carry a task, the model still registers and you can set the task by hand; the picker then offers the right tests.

The list groups models into categories derived from that task: LLM, Vision, Audio, Image, Multimodal, Embedding. It is the same grouping the benchmark picker uses.

The Hugging Face access token​

The dialog has an access-token field, and it is worth understanding what it is for before you need it.

What it is. A personal access token from your Hugging Face account. It proves to the Hub that you are allowed to read a repository that is not public.

Why the platform needs one. Public repositories need nothing at all, and most first benchmarks use one. A token becomes necessary when the model or the dataset behind a benchmark is gated (you accepted terms to get access, as with several Llama and GPQA releases) or private (it belongs to you or your organization). Three moments need it: reading the repository's metadata when you register the model, fetching the checkpoint on the device when a benchmark runs, and fetching it again when you deploy the model as an endpoint.

Where to get one. In Hugging Face, under your account settings, at huggingface.co/settings/tokens. A token with read permission is enough. A fine-grained token also works, as long as it grants access to the specific repositories you plan to benchmark.

Where to put it, and this is the part that catches people. The token you type into the Add-model dialog is used once, for that metadata read, and is not stored: the dialog says so, and it means it. A token that benchmark runs and deployments can use lives in Settings, then Credentials, as a HuggingFace credential. There you choose whether it is Personal (only your own runs use it) or Organization (anyone in the organization can), and the platform stores it encrypted. When both exist, your personal credential wins over the organization's.

What happens without one. A public model behaves exactly as it did above. A gated model registers but the run fails on the device when the fetch is refused, and registering one from the dialog without a token reports that the metadata could not be fetched and to check the URL and, for gated or private models, the token.

The other way in​

You do not have to register a model first. The benchmark flow accepts a Hugging Face URL directly, and registers it as a model on the way through. Registering up front is worth it when several people run against the same model, because everyone then picks it from a list instead of pasting a URL and hoping they pasted the same one.

What you have now​

A model reference, a device, and a project holding both. The next part runs one against the other.