Skip to main content

//clika-runtime/io.clika.modelverse/Server

Server

[common]
class Server

The server over a loaded GenerativeModel (Modelverse.serve): the chat API (/v1/chat/completions, the models listing, /health, the web page when enabled) on its own threads. start listens and returns at once; waitUntilReady probes /health; stop ends the listening and joins the server's threads; close stops and releases the served engine. The model stays open, the caller's, and refuses its own close() while a server holds it: close the server first. Several servers may serve one model. http is the server's own http server, borrowed, for routes and middleware of the app's beside the API's, registered before start.

Properties​

NameSummary
baseUrl[common]
val baseUrl: String
http://host:port, the prefix of every route.
deviceFaulted[common]
val deviceFaulted: Boolean
True once the device the model runs on faulted under a request; the server then refuses new ones.
host[common]
val host: String
The host the server binds, as given.
http[common]
val http: HttpServer
The http server the API is mounted on, borrowed (HttpServer.isBorrowed) for the app's own routes, groups, middleware and static files, registered before start: once the server listens the routes are fixed, and a read here throws IllegalStateException. A route on a path the API owns (isReservedRoute) shadows the API's; the borrow is released at stop and close, so a later call on it refuses.
isRunning[common]
val isRunning: Boolean
True between start and stop.
model[common]
val model: GenerativeModel
The model the server answers with.
options[common]
val options: ServeOptions
The options the server was built with.
port[common]
val port: Int
The bound port once started (the requested one, or the free port picked for 0); 0 before.

Functions​

NameSummary
close[common]
open fun close()
Stop, then release the served engine; the model stays open. Idempotent.
isReservedRoute[common]
fun isReservedRoute(path: String): Boolean
True for a path the API owns (/v1/..., /health, the page); a route there shadows the API's.
start[common]
fun start(): Server
Listen on the host and port, on the server's own threads, and return at once; a port in use throws.
stop[common]
fun stop()
Stop listening and join the server's threads, the http borrow released; a no-op when not running.
waitUntilReady[common]
fun waitUntilReady(timeoutMs: Long = DEFAULT_READY_TIMEOUT_MS)
Block until /health answers, polling every POLL_MS milliseconds up to timeoutMs; throws ModelverseException at the deadline.