clika_runtime.runtime

Serve models: nodes, the single-node Executor, the multi-node Pipeline.

A serving node is either built from an executable graph (:func:from_graph) or served by Python callables (:func:function_node). Drive one node with an :class:Executor, or wire several by name into a :class:Pipeline::

import clika_runtime as crt
from clika_runtime import runtime

def double(ctx: runtime.PhaseContext) -> None:
    ctx.outputs["y"] = ctx.inputs["x"] * 2.0

schema = runtime.Schema(
    inputs=[crt.TensorSpec("x", crt.float32, [-1])],
    outputs=[crt.TensorSpec("y", crt.float32, [-1])],
)
node = runtime.function_node(schema, run_once={"run": double})

with runtime.Executor(node) as ex:
    response = ex.run(runtime.Request({"x": crt.tensor([1.0, 2.0])}))
    print(response.outputs["y"])

A stepped node (a decoder) declares an iterate phase and a per-session state through make_state; the runtime steps every active session in one batched call and stops a session when its :class:StepResult says Finish::

class Counter:
    def __init__(self) -> None:
        self.steps = 0

def step(ctx: runtime.StepContext) -> list[runtime.StepResult]:
    results = []
    for slot in ctx.slots:
        slot.state.steps += 1
        done = slot.state.steps == 4
        results.append(
            runtime.StepResult.finish() if done else runtime.StepResult()
        )
    return results

Streaming: :meth:Pipeline.stream returns a :class:ResponseStream that yields a :class:ResponseChunk per streamed step; :func:astream is the same stream as an async iterator for asyncio code.

Threading: every blocking call releases the interpreter lock while it waits, so other Python threads (and Python-served phases running on the engine's threads) keep making progress. A Python-served phase's callables run on the engine's thread while the request executes; an exception raised there fails that session only and surfaces where its result is read (wait(), run(), or the stream). Shut an Executor or Pipeline down before dropping it when work may still be in flight (use it as a context manager).

Name	Kind
`Disposition`	class
`ExecConfig`	class
`Executor`	class
`GraphNode`	class
`Node`	class
`PhaseContext`	class
`PhaseKind`	class
`PhaseOutputs`	class
`PhaseSpec`	class
`Pipeline`	class
`Request`	class
`Response`	class
`ResponseChunk`	class
`ResponseStream`	class
`Schema`	class
`StepContext`	class
`StepResult`	class
`StepSlot`	class
functions	module functions