clika_runtime
clika-runtime: the Python package of the ClikaRT on-device inference runtime.
Importing this package loads the runtime library shipped in-package (under
clika_runtime/lib/). The Modelverse model library ships in the same
package as :mod:clika_runtime.modelverse (import clika_runtime.modelverse as mv); importing it loads the runtime first and resolves it from here.
The top level publishes :class:Tensor, :class:Size, :class:Device,
:class:Stream, the dtype objects (float32,
bfloat16, ...), the tensor factories (tensor, zeros,
randn, ...), every operator of :mod:clika_runtime.ops as a
free function (clika_runtime.matmul(a, b)), the placement and
execution scopes (device, stream, synchronous, tracing,
eager, meta_init), eval / async_eval / synchronize,
save / load / load_with_metadata, compile / trace, the device functions, and the
exception classes. Modes are strings (approximate="tanh"); the typed
enumerations stay under clika_runtime._core.ops. Models are built with
:mod:clika_runtime.nn; :mod:clika_runtime.torch (imported on first use,
never as a side effect) exchanges tensors and modules with PyTorch.
Execution is asynchronous by default: an operator returns at once and its
work rides the calling thread's current lane on the input's device. A
value settles when it is read (numpy(), item(), tolist(),
print), when eval(*trees) or synchronize() is called, or inside
a synchronous() region where every operator completes before it
returns. tracing() records operations instead of running them until
eval materializes them; eager() restores the default inside a
tracing region. Reading the value of a traced or storage-free tensor raises
:class:ClikaRTError.
Every process that runs an operator or a model needs the license credential
CLIKA issued for the project: CLIKA_RT_LICENSE in the environment (the
CLIKA1-... text, or the path of a file holding it), set before this
package is imported, else the per-user file the clikart-license-init
console script writes once. Without one the first call raises
:class:ClikaRTError with the code name LICENSE_FAILED; every error
carries a stable code_name beside its message, and the code name is the
thing to branch on (:mod:clika_runtime.errors).
A model from a model hub loads through :mod:clika_runtime.modelverse:
mv.AutoModelForCausalLM.from_pretrained(repo_id) downloads the snapshot
into the hub cache and loads it, mv.snapshot_download(repo_id) downloads
without loading, and a local directory is a source everywhere a repository
id is. A model written here from the operators starts from
:mod:clika_runtime.nn (nn.Module, nn.Linear, nn.KVCache) and
the fused attention operator; the documentation's examples walk one end
to end.
| Name | Kind |
|---|---|
ClikaRTError | class |
CompiledFunction | class |
CompiledModule | class |
Device | class |
DeviceProperties | class |
InternalError | class |
InvalidArgumentError | class |
NotFoundError | class |
OutOfMemoryError | class |
QTensor | class |
Size | class |
Stream | class |
Tensor | class |
TensorSpec | class |
TracedGraph | class |
UnavailableError | class |
UnsupportedError | class |
dtype | class |
finfo | class |
iinfo | class |
| functions | module functions |