Skip to main content

ClikaRT::device::MemoryStats

struct

Header: ClikaRT/compute/memory_stats.h

Per-device memory telemetry: the device's physical free/total plus the runtime memory pool's counters for that device. memory_stats(device) answers "where did the memory go" for exactly the device a load or a serve targets, the load-time companion of is_backend_available(); release_cached_memory(device) returns the pool's idle reserve to the driver or the OS; reset_peak_memory_stats(device) restarts the pool's high-water marks so one step's transient can be measured.

Data members​

device_free_bytes​

std::uint64_t device_free_bytes = 0

physical bytes currently free

Declared in ClikaRT/compute/memory_stats.h, line 41

device_total_bytes​

std::uint64_t device_total_bytes = 0

physical capacity

Declared in ClikaRT/compute/memory_stats.h, line 42

device_memory_source​

MemorySource device_memory_source = MemorySource::Unknown

How the two physical figures above were obtained.

Declared in ClikaRT/compute/memory_stats.h, line 44

active_bytes​

std::uint64_t active_bytes = 0

pool bytes currently handed out

Declared in ClikaRT/compute/memory_stats.h, line 46

requested_bytes​

std::uint64_t requested_bytes = 0

Bytes the live allocations asked for: active_bytes less the pool's carve slack (a free block at or above the pool's split ceiling serves a smaller request whole). The residency gauge: a second live copy of a weight doubles it, a bigger chunk does not move it.

Declared in ClikaRT/compute/memory_stats.h, line 51

reserved_bytes​

std::uint64_t reserved_bytes = 0

pool bytes held from the driver (active + cached)

Declared in ClikaRT/compute/memory_stats.h, line 52

cached_bytes​

std::uint64_t cached_bytes = 0

idle pool bytes retained for reuse

Declared in ClikaRT/compute/memory_stats.h, line 53

peak_active_bytes​

std::uint64_t peak_active_bytes = 0

high-water mark of active_bytes

Declared in ClikaRT/compute/memory_stats.h, line 55

peak_reserved_bytes​

std::uint64_t peak_reserved_bytes = 0

high-water mark of reserved_bytes

Declared in ClikaRT/compute/memory_stats.h, line 56

num_allocs​

std::uint64_t num_allocs = 0

successful pool allocations

Declared in ClikaRT/compute/memory_stats.h, line 58

num_frees​

std::uint64_t num_frees = 0

pool frees

Declared in ClikaRT/compute/memory_stats.h, line 59

num_oom​

std::uint64_t num_oom = 0

pool out-of-memory events

Declared in ClikaRT/compute/memory_stats.h, line 60

num_inflight_park_waits​

std::uint64_t num_inflight_park_waits = 0

The number of allocation requests that waited for an in-flight chunk to retire: the only pool memory that fit was still owned by device work that had not completed, so the request paused until it did (a host running ahead of the device, under a pool at its size). Only ever rises; 0 where no request has waited.

Declared in ClikaRT/compute/memory_stats.h, line 66

wrapped_bytes​

std::uint64_t wrapped_bytes = 0

adopted subset of active_bytes

The ADOPTED subset of active_bytes: memory the runtime did not allocate but took over in place, a model file mapped straight into device-visible memory, loaded without copying. Those bytes count against the device exactly like allocated ones (on a unified-memory part they are wired 1:1), so they belong in active_bytes; reported apart because the two are released differently; adopted memory goes away when the mapping is dropped, the rest when the data is freed. wrapped_allocs is how many such mappings are held RIGHT NOW (a live count, unlike num_allocs above, which only ever rises). Both are 0 where the device does not adopt memory in place.

Declared in ClikaRT/compute/memory_stats.h, line 78

wrapped_allocs​

std::uint64_t wrapped_allocs = 0

mappings currently adopted

Declared in ClikaRT/compute/memory_stats.h, line 79