---
title: Serve and chat
sidebar_label: 3. Serve and chat
description: "Host the model behind an OpenAI-compatible HTTP endpoint, hold a conversation on the built-in chat page, and call it with curl."
---

The model generates on demand; this part keeps it loaded. `serve` hosts the model behind an OpenAI-compatible server with a built-in chat page. One command, no configuration files:

```bash
clikart-cli Qwen/Qwen2.5-0.5B-Instruct serve
```

```text
[2026-10-04 07:56:43.933] [modelverse] [info] serving on http://127.0.0.1:8000 (model=Qwen2.5-0.5B-Instruct); open it in a browser for the chat page
[2026-10-04 07:56:43.933] [modelverse] [info] endpoints: POST /v1/chat/completions /v1/messages /v1/messages/count_tokens /v1/audio/transcriptions; GET /v1/models /health /props / (web UI) /dashboard
```

The defaults bind `127.0.0.1:8000`; `--host 0.0.0.0` opens it to the network and `--port` moves it. Three things are now running:

- **The API.** `POST /v1/chat/completions`, streaming (server-sent events) and non-streaming, in the OpenAI request and response shape.
- **A web chat page.** `http://127.0.0.1:8000/` serves a built-in chat UI from inside the binary (`--no-web-ui` disables it).
- **A health probe.** `GET /health` answers `{"status":"ok"}`, for load balancers and scripts.

## Chat in the browser

The chat page is the conversation surface: a scrolling transcript with an input box, the assistant reply streaming token by token as the pipeline decodes it. The conversation carries its history, so follow-up questions see earlier turns. The same conversation is available to any OpenAI client through the API, where the system message is the first `messages` entry and the sampling knobs (`temperature`, `top_p`) ride each request.

One templated turn from the terminal stays `prompt`, part 2's command. `prompt`, `serve` and `bench` are the text-generation surface, and there is no terminal `chat` verb: `clikart-cli <model> chat` is refused by name, `error: family 'qwen' (text->text) provides no chat (it provides: prompt, serve, bench, mm_bench)`.

## Call it over HTTP

From a second terminal:

```bash
curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "messages": [{"role": "user", "content": "Say hello in French."}],
        "max_tokens": 32
      }'
```

```text
{"id":"chatcmpl-1791068204-0","object":"chat.completion","created":1791068204,"model":"Qwen2.5-0.5B-Instruct","choices":[{"index":0,"message":{"role":"assistant","content":"Bonjour! C'est un plaisir de vous rencontrer. Comment puis-je vous aider aujourd'hui ?"},"finish_reason":"stop"}],"usage":{"prompt_tokens":34,"completion_tokens":20,"total_tokens":54,"prompt_tokens_details":{"cached_tokens":0}}}
```

The `id`, `created` and `usage` values are your run's own. Add `"stream": true` and the response arrives as server-sent events, one delta per chunk, the way OpenAI clients expect. An existing client needs one change:

```python
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="unused")
reply = client.chat.completions.create(
    model="qwen",
    messages=[{"role": "user", "content": "Say hello in French."}],
)
print(reply.choices[0].message.content)
```

The endpoint surface is bigger than chat: a Whisper model's `serve` answers `POST /v1/audio/transcriptions`, an embedding model's answers `POST /v1/embeddings`, and so on per family. [Serve a model over the OpenAI and Anthropic APIs](../../how-to/serve-openai-compatible.md) has the full route table and the operational flags.

Next: [part 4](04-use-it-from-code.mdx), the same model inside your own program.