clika_runtime.modelverse.serve functions

serve

serve(model_or_source: 'str | GenerativeModel | ImageGenerationModel | VideoGenerationModel | Any', host: 'str' = '127.0.0.1', port: 'int' = 8000, *, task: 'str' = 'text-generation', start: 'bool' = True, ready_timeout: 'float' = 30.0, **options: 'Any') -> 'Server'

Serve a text-generation, a text-to-image or a text-to-video model over the OpenAI-compatible API and return the running :class:Server.

model_or_source is a loaded model, an engine, or a source to load (a local directory, a hub repo id, a .gguf file); task picks the load door for a source string: "text-generation" (the default), "text-to-image" or "text-to-video". The remaining keyword arguments are the load options (device, dtype, max_seq, revision, cache_dir, token, ...) and the server options (model_id, max_active, max_queued, default_budget, enable_web_ui, enable_cors, log_request_timing, max_upload_bytes, read_timeout_seconds, api_key, the key every request must carry when it is set, and codec, the video door's clip codec). start=False returns the server unstarted; otherwise it is started and probed ready within ready_timeout seconds.