clika_runtime.modelverse
clika_runtime.modelverse: the Modelverse model library.
Load a model by name and run it::
import clika_runtime.modelverse as mv
model = mv.AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen2.5-0.5B-Instruct", device="cpu", dtype="bfloat16")
print(model.generate("hello", max_new_tokens=64))
for piece in model.stream_generate("hello"):
print(piece, end="", flush=True)
print(model.chat([{"role": "user", "content": "hi"}]))
server = mv.serve(model, port=8000) # the OpenAI-compatible API
The model library ships in the clika-runtime package beside the inference engine. Importing it loads the engine first (the parent package), then the model library's extension, which the same release builds.
from_pretrained takes a hub repository id (the snapshot downloads into the
Hugging Face hub cache the other tooling on the machine shares; a gated
repository takes its token argument), a local directory (it
passes through untouched, the offline path) or a .gguf file.
snapshot_download(repo_id) downloads without loading and returns the
local directory; list_models() enumerates the families this library
serves; check(source) says whether a model fits a device before anything
downloads; info(source) reads what a model is from its documents alone.
Every model run needs the runtime's license credential (CLIKA_RT_LICENSE
or the per-user file, see :mod:clika_runtime); without one the first call
raises with the code name LICENSE_FAILED.