---
title: "ClikaRT::device"
sidebar_label: "device"
description: "Namespace ClikaRT::device"
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--namespace">namespace</span>

## Classes

| Name | Description |
| --- | --- |
| [`DeviceProperties`](./DeviceProperties.md) | A device's static capabilities. A plain value type; copy it freely. |
| [`MemoryStats`](./MemoryStats.md) |  |

## Enumerations

### enum ComputeAPI {#ComputeAPI}

<div class="api-signature">

`enum class` **`ComputeAPI`** `:` `std::int32_t`

</div>

The compute backend a device dispatches through. Names the runtime/SDK, not the physical hardware (one GPU may be reachable via more than one).

`int32`-backed for a stable ABI. ABI note: new APIs append at the end; existing ones never reorder or drop.

| Enumerator | Value | Description |
| --- | --- | --- |
| `CPU` | `0` | Host CPU (SIMD kernels). |
| `CUDA` |  | NVIDIA CUDA. |
| `Vulkan` |  | Cross-vendor GPU compute. |
| `Metal` |  | Apple Metal. |
| `HTP` |  | Qualcomm Hexagon DSP/NPU (HTP). |
| `TPU` |  | Google Cloud TPU (PJRT). |

<div class="api-declared">

Declared in `ClikaRT/compute/device.h`, line 20

</div>

### enum DeviceKind {#DeviceKind}

<div class="api-signature">

`enum class` **`DeviceKind`** `:` `std::int32_t`

</div>

Where a device sits relative to the host: what an application reads to decide how much to place on it and whether it shares the host's memory.

`int32`-backed for a stable ABI. ABI note: new kinds append at the end; existing ones never reorder or drop.

| Enumerator | Value | Description |
| --- | --- | --- |
| `Unknown` | `0` | The backend reports no kind. |
| `Cpu` |  | The host processor itself: the CPU backend, or a software GPU driver. |
| `IntegratedGpu` |  | A GPU sharing the host's memory (phones, Jetson boards, Apple silicon). |
| `DiscreteGpu` |  | A GPU with its own memory behind a bus (desktop and data-center cards). |
| `VirtualGpu` |  | A GPU reached through a virtualization layer. |
| `Accelerator` |  | An NPU or TPU class device. |

<div class="api-declared">

Declared in `ClikaRT/compute/device_properties.h`, line 24

</div>

### enum MemorySource {#MemorySource}

<div class="api-signature">

`enum class` **`MemorySource`** `:` `std::uint8_t`

</div>

A point-in-time snapshot. Physical numbers come from the driver (GPU devices) or the OS's memory accounting (the CPU device: reclaimable+free as "free", MemAvailable on Linux). A device whose driver reports free memory as a count of free system pages (an integrated CUDA part) reports the larger of that count and the OS's MemAvailable, since the page count leaves out the reclaimable cache; a device whose driver already reports a budget (a unified-memory GPU's working-set headroom) reports that budget as is. Pool numbers come from the runtime's caching allocator for that device. Where a platform has no physical source the device\_\* fields are 0 ("unknown"); the pool counters still report. How the physical figures of a [`MemoryStats`](./MemoryStats.md) were obtained: read from the device's driver or the OS's accounting (`Measured`), derived from a declared budget or a clamped read (`Heuristic`), or not available (`Unknown`: the device\_\* fields are then the honest 0, or a figure whose source the device did not declare). A decision that must not misfire (refusing to load a model for lack of memory) keys on `Measured` alone; the other two carry at most a warning.

| Enumerator | Description |
| --- | --- |
| `Unknown` |  |
| `Heuristic` |  |
| `Measured` |  |

<div class="api-declared">

Declared in `ClikaRT/compute/memory_stats.h`, line 38

</div>

## Variables

### kWholeMachineCpuIndex {#kWholeMachineCpuIndex}

<div class="api-signature">

`std::int32_t` **`kWholeMachineCpuIndex`** `= -1`

</div>

The whole-machine CPU: every core, over memory the OS places on first touch. CPU indexing is uniform around it: a NEGATIVE index is the whole machine, a NON-NEGATIVE index is that CPU socket (its own cores, its own local memory). On a single-socket machine the two describe the same hardware.

<div class="api-declared">

Declared in `ClikaRT/compute/device.h`, line 46

</div>

## Functions

### is\_backend\_compiled() {#is_backend_compiled}

<div class="api-signature">

`bool` **`is_backend_compiled`**([`ComputeAPI`](./index.md#ComputeAPI) `backend`) `noexcept`

</div>

Was `backend` compiled into this build of [ClikaRT](../../index.md)? A pure build-time fact; it does not attempt to load anything. CPU is always compiled in.

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 25

</div>

### is\_backend\_available() {#is_backend_available}

<div class="api-signature">

`bool` **`is_backend_available`**([`ComputeAPI`](./index.md#ComputeAPI) `backend`)

</div>

Can `backend` serve work on this machine? True only when the backend loads and initializes AND at least one of its devices is accepted and comes up. Returns false if the backend was not compiled in, its shared library is absent, it fails to bring up (e.g. no driver), or it loads but every device it finds is refused (a Vulkan device whose driver is below Vulkan 1.2, a GPU this build's kernels cannot run); [`backend_load_status`](./index.md#backend_load_status) answers the load-only question and [`get_device_properties`](./index.md#get_device_properties) names a refused device's reason. Brings the backend and its first serving device up on first call, then caches the result; CPU is always available.

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 36

</div>

### is\_backend\_loaded() {#is_backend_loaded}

<div class="api-signature">

`bool` **`is_backend_loaded`**([`ComputeAPI`](./index.md#ComputeAPI) `backend`) `noexcept`

</div>

Is `backend` loaded and initialized RIGHT NOW, without bringing it up? A plain read of the runtime's registry, which publishes a backend only once it has initialized: false for a backend nothing has used yet, one that failed to come up, and one this build does not carry. The probe for code that must never start a backend as a side effect (a memory-pressure handler, a status line) where [`is_backend_available`](./index.md#is_backend_available) would load it first.

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 44

</div>

### is\_cpu\_available() {#is_cpu_available}

<div class="api-signature">

`bool` **`is_cpu_available`**()

</div>

Convenience predicates equivalent to [`is_backend_available`](./index.md#is_backend_available)`(<api>)`.

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 47

</div>

### is\_cuda\_available() {#is_cuda_available}

<div class="api-signature">

`bool` **`is_cuda_available`**()

</div>

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 48

</div>

### is\_vulkan\_available() {#is_vulkan_available}

<div class="api-signature">

`bool` **`is_vulkan_available`**()

</div>

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 49

</div>

### is\_metal\_available() {#is_metal_available}

<div class="api-signature">

`bool` **`is_metal_available`**()

</div>

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 50

</div>

### is\_tpu\_available() {#is_tpu_available}

<div class="api-signature">

`bool` **`is_tpu_available`**()

</div>

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 51

</div>

### backend\_load\_status() {#backend_load_status}

<div class="api-signature">

`void` **`backend_load_status`**([`ComputeAPI`](./index.md#ComputeAPI) `backend`)

</div>

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 69

</div>

### backend\_device\_status() {#backend_device_status}

<div class="api-signature">

`void` **`backend_device_status`**([`ComputeAPI`](./index.md#ComputeAPI) `backend`)

</div>

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 83

</div>

### device\_count() {#device_count}

<div class="api-signature">

`std::int32_t` **`device_count`**([`ComputeAPI`](./index.md#ComputeAPI) `backend`)

</div>

How many devices `backend` exposes on this machine (e.g. the number of CUDA GPUs, or Hexagon DSPs). 0 if the backend is unavailable. For CPU this is the number of exposed host compute domains (one per socket). Brings the backend up on first call.

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 89

</div>

### enumerate\_devices() {#enumerate_devices}

<div class="api-signature">

`std::vector<`[`Device`](../Device.md)`>` **`enumerate_devices`**([`ComputeAPI`](./index.md#ComputeAPI) `backend`)

</div>

The concrete [`Device`](../Device.md) handles `backend` exposes on this machine, in index order (`{api, 0}`, `{api, 1}`, …). Empty if the backend is unavailable; an empty list is a valid answer, not an error. Pass each [`Device`](../Device.md) to [`get_device_properties`](./index.md#get_device_properties) for its capabilities, or to [`Tensor::to`](../Tensor.md#to) to place work on it.

<div class="api-declared">

Declared in `ClikaRT/compute/backends.h`, line 96

</div>

### compute\_api\_name() {#compute_api_name}

<div class="api-signature">

`const char*` **`compute_api_name`**([`ComputeAPI`](./index.md#ComputeAPI) `api`) `noexcept`

</div>

A short, human-readable backend name (e.g. "CUDA"). Never null.

<div class="api-declared">

Declared in `ClikaRT/compute/device.h`, line 30

</div>

### parse\_device() {#parse_device}

<div class="api-signature">

[`Device`](../Device.md) **`parse_device`**(`std::string_view` `spec`)

</div>

<div class="api-declared">

Declared in `ClikaRT/compute/device.h`, line 138

</div>

### manual\_seed() {#manual_seed}

<div class="api-signature">

`void` **`manual_seed`**([`Device`](../Device.md) `device`, `std::uint64_t` `seed`)

</div>

Seed [`device`](./index.md)'s random-number generator and reset its draw counter, making subsequent random ops (e.g. [`ops::multinomial`](../ops/multinomial.md#multinomial)) reproducible on that device. Seeding is device-scoped; every stream on the device shares one sequence. Do not call concurrently with in-flight random ops on the same device.

<div class="api-declared">

Declared in `ClikaRT/compute/device.h`, line 147

</div>

### current\_seed() {#current_seed}

<div class="api-signature">

`std::uint64_t` **`current_seed`**([`Device`](../Device.md) `device`)

</div>

The current seed of [`device`](./index.md)'s generator. A device never explicitly seeded reports the default seed (0); results are reproducible out of the box.

<div class="api-declared">

Declared in `ClikaRT/compute/device.h`, line 151

</div>

### synchronize() {#synchronize}

<div class="api-signature">

`void` **`synchronize`**([`Device`](../Device.md) `device`)

</div>

Wait for every operation queued on [`device`](./index.md)'s lanes: the calling thread's current lane there, every `Stream::create(device)` lane, and the device's default stream. On return the device's queued work is complete and each of its lanes reports [`Stream::query_idle()`](../Stream.md#query_idle). Only [`device`](./index.md)'s lanes are waited on: work queued on another device runs on, and a completion callback ([`Tensor::on_complete`](../Tensor.md#on_complete)) posted by the drained work may still be running. A device with nothing queued returns at once.

<div class="api-declared">

Declared in `ClikaRT/compute/device.h`, line 161

</div>

### get\_device\_properties() {#get_device_properties}

<div class="api-signature">

[`DeviceProperties`](./DeviceProperties.md) **`get_device_properties`**([`Device`](../Device.md) `device`)

</div>

<div class="api-declared">

Declared in `ClikaRT/compute/device_properties.h`, line 97

</div>

### memory\_stats() {#memory_stats}

<div class="api-signature">

[`MemoryStats`](./MemoryStats.md) **`memory_stats`**([`Device`](../Device.md) `device`)

</div>

Snapshot the memory state of [`device`](./index.md). Fails only when the device itself is invalid or its backend is unavailable.

<div class="api-declared">

Declared in `ClikaRT/compute/memory_stats.h`, line 97

</div>

### release\_cached\_memory() {#release_cached_memory}

<div class="api-signature">

`void` **`release_cached_memory`**([`Device`](../Device.md) `device`)

</div>

Return the runtime pool's idle memory for [`device`](./index.md) to the driver or the OS. The pool keeps memory it handed out and got back (`cached_bytes`) so the next allocation is cheap; after a model is unloaded, or before a second model must fit beside the first, that idle reserve is memory the device (or, on a shared-memory part, the host) cannot use for anything else until it is released. Memory still in use, and memory whose last use has not finished on the device, is never touched: the call releases what is provably idle at the moment it runs and leaves the rest cached, so a later call can release more once that work has completed. A closed model's bytes stay in the pool until this call or memory pressure, so a program that closes one model and loads another holds both until it calls this in between; the Kotlin Modelverse binding calls it at the model's close. Fails only when the device itself is invalid or its backend is unavailable.

<div class="api-declared">

Declared in `ClikaRT/compute/memory_stats.h`, line 115

</div>

### reset\_peak\_memory\_stats() {#reset_peak_memory_stats}

<div class="api-signature">

`void` **`reset_peak_memory_stats`**([`Device`](../Device.md) `device`)

</div>

Restart the high-water marks of the runtime pool for [`device`](./index.md) from its present occupancy. [`MemoryStats::peak_active_bytes`](./MemoryStats.md#peak_active_bytes), `peak_requested_bytes` and `peak_reserved_bytes` only ever rise, so after a load or a warm-up they record the largest footprint the process has held so far and say nothing about the step that follows. This call sets the active mark to the pool's present `active_bytes`, the requested mark to `requested_bytes` (and the reserved mark to `reserved_bytes` where the pool keeps one apart from the active mark); each tracks upward from there. The measuring idiom: reset, read `active_bytes`, run one step, snapshot again; `peak_active_bytes` less the `active_bytes` read before the step is the largest transient the step needed on top of what was already resident, even after the step's temporaries are gone. The same reading over `requested_bytes` and `peak_requested_bytes` counts the bytes the step asked for, which a larger block serving a request does not inflate. Nothing is allocated, freed, or waited for: the marks move, the memory does not. Fails only when the device itself is invalid or its backend is unavailable.

<div class="api-declared">

Declared in `ClikaRT/compute/memory_stats.h`, line 136

</div>

## `ClikaRT/compute/backends.h` {#backends-h}

```cpp
#include <ClikaRT/compute/backends.h>
```

Backend + device discovery: which compute backends this build was compiled with, which actually load on this machine, how many devices each exposes, and the concrete `Device` handles to address them.

`get_device_properties` ([device\_properties.h](./DeviceProperties.md)) describes ONE device; these functions let you find out which devices exist in the first place; the starting point for a program that adapts to whatever hardware it runs on (e.g. shard work across every available GPU).