Wrap existing memory without copying

Your data already sits in memory that some other part of the program owns: a decoder's output buffer, an arena, a mapped file, another library's array. Tensor::from_blob wraps that memory as a tensor with no copy; the tensor's data pointer is your pointer. What needs deciding is ownership, and the API makes the two contracts explicit:

No deleter passed: borrowed. You keep ownership. The buffer must stay alive, and its layout unchanged, for as long as the tensor or any view of it is in use. Writes through the buffer are visible through the tensor and the other way around; it is the same memory.
Deleter passed: adopted. The tensor takes ownership and calls deleter(data) once, when the last reference drops.

The Python variant borrows a numpy array's memory.

Borrow a buffer and compute on it

C++
Python

borrow.cpp
#include <cstdio>
#include <vector>

#include <ClikaRT/clika_rt.h>

using ClikaRT::DataType;
using ClikaRT::Tensor;
namespace ops = ClikaRT::ops;

int main() {
    std::vector<float> buf(8, 1.0F);  // memory the application owns

    const Tensor view = Tensor::from_blob(buf.data(), {8}, DataType::Float32);
    std::printf("shares memory: %s\n",
                view.const_data_ptr() == buf.data() ? "yes (no copy)" : "no");
    std::printf("sum = %g\n", ops::sum(view).item<float>());

    buf[0] = 100.0F;  // write through the buffer...
    std::printf("sum after buf[0] = 100: %g\n", ops::sum(view).item<float>());
    return 0;
}

main.py
import numpy as np

import clika_runtime as crt


def main() -> None:
    # Borrow: from_numpy wraps the array's own memory, no copy. The array
    # and the tensor see the same bytes, mutation aliases both ways, and
    # the tensor keeps the array alive. The array must be writable and
    # contiguous; the refusals name the fix.
    buf = np.zeros(4, dtype=np.float32)
    view = crt.from_numpy(buf)

    buf[0] = 100.0
    print(f"sum after buf[0] = 100: {view.sum().item():g}")

    # The other direction: numpy() on a CPU tensor is a zero-copy view of
    # the tensor's memory; a device tensor asks you to move it first
    # (t.to('cpu').numpy()).
    t = crt.ones(2, 2)
    arr = t.numpy()
    print(f"shared bytes: {arr.sum():g}")

    # A non-contiguous array does not borrow; the error names the fix.
    try:
        crt.from_numpy(np.zeros((4, 4), dtype=np.float32)[:, ::2])
    except TypeError as e:
        print(f"non-contiguous refused: {e}")


if __name__ == "__main__":
    main()

crt.tensor(array) stays the copying entry when an independent tensor is wanted.

shares memory: yes (no copy)
sum = 8
sum after buf[0] = 100: 107

The borrow contract in one sentence: the runtime never reuses or overwrites borrowed memory, and in exchange you guarantee it outlives every tensor that sees it. A vector that reallocates (or a stack buffer that goes out of scope) under a live view is the bug this contract exists to name.

Hand ownership over with a deleter

When the producer wants to fire and forget, pass a deleter. The tensor (and every tensor computed from it) keeps the buffer alive; the deleter runs exactly once, when the last reference drops, and it runs in your runtime: an exception it throws never crosses the library boundary. It also frees the runtime to reuse the buffer as scratch, which the borrow contract forbids.

C++
Python

adopt.cpp
#include <cstdint>
#include <cstdio>
#include <cstdlib>

#include <ClikaRT/clika_rt.h>

using ClikaRT::DataType;
using ClikaRT::Tensor;
namespace ops = ClikaRT::ops;

namespace {
constexpr std::int64_t kCount = 8;   // elements in the caller-owned buffer
}  // namespace

int main() {
    float* buf = static_cast<float*>(std::malloc(kCount * sizeof(float)));
    for (std::int64_t i = 0; i < kCount; ++i) buf[i] = static_cast<float>(i);

    {
        const Tensor adopted = Tensor::from_blob(
            buf, {kCount}, DataType::Float32, {},
            [](void* p) { std::printf("deleter: buffer released\n"); std::free(p); });
        std::printf("mean = %g\n", ops::mean(adopted).item<float>());
        std::printf("leaving the tensor's scope...\n");
    }
    std::printf("scope closed\n");
    return 0;
}

Handing ownership over with a deleter is part of the C++ API today; the C++ tab shows it. The python borrow keeps the ARRAY as the owner: crt.from_numpy(array) holds the array alive for the tensor's lifetime, so no deleter changes hands.

mean = 3.5
leaving the tensor's scope...
deleter: buffer released
scope closed

The deleter must not throw (a throw is swallowed). Adoption is the right contract at module boundaries: the producer allocates, the consumer wraps and forgets the allocation ever existed.

Wrap non-contiguous memory with strides

The strided overload views memory that is not laid out contiguously, without rearranging a byte. Strides are in elements, one per dimension. A worked case: cropping a region of interest out of a pitched image buffer, the layout every camera API and GPU readback hands you (rows padded to a pitch wider than the image).

C++
Python

strided_roi.cpp
#include <cstdint>
#include <cstdio>
#include <vector>

#include <ClikaRT/clika_rt.h>

using ClikaRT::DataType;
using ClikaRT::Tensor;
namespace ops = ClikaRT::ops;

namespace {
constexpr std::int64_t kRows = 4, kPitch = 6;       // the whole image, row-major
constexpr std::int64_t kRoiRow = 1, kRoiCol = 2;    // where the region starts
constexpr std::int64_t kRoiRows = 2, kRoiCols = 3;  // its extent
}  // namespace

int main() {
    // A 4x6 single-channel image, row-major. buf[r][c] = r*10 + c.
    std::vector<float> buf(kRows * kPitch);
    for (std::int64_t r = 0; r < kRows; ++r)
        for (std::int64_t c = 0; c < kPitch; ++c) buf[r * kPitch + c] = static_cast<float>(r * 10 + c);

    // The 2x3 region starting at row 1, column 2: shape {2, 3}, and the
    // ORIGINAL row pitch as the row stride. No pixel is copied.
    const Tensor roi = Tensor::from_blob(buf.data() + kRoiRow * kPitch + kRoiCol,
                                         {kRoiRows, kRoiCols}, {kPitch, 1}, DataType::Float32);

    std::printf("roi = %s\n", roi.to_string().c_str());
    std::printf("sum = %g (12+13+14+22+23+24 = 108)\n", ops::sum(roi).item<float>());
    return 0;
}

Wrapping strided memory with explicit strides is part of the C++ API today; the C++ tab shows it. crt.from_numpy takes a C-contiguous array; a strided source enters through np.ascontiguousarray (the refusal names it).

roi = Tensor(shape=[2, 3], dtype=Float32, device=CPU, numel=6, data=[12, 13, 14, 22, 23, 24])
sum = 108 (12+13+14+22+23+24 = 108)

The same shape covers any pitched or tiled layout: a submatrix of a row-major matrix, a plane in a planar image, a batch entry inside a larger allocation. ops::contiguous materializes an owned compact copy when a consumer needs one.

Device moves and pinned memory

.to(device) on a wrapped tensor behaves like on any other: on a discrete accelerator the move is a real transfer to device memory (the wrap saved the host-side copy, not the transfer), while unified-memory hardware moves for free. Two related notes. from_data is the copying cousin: it copies your bytes into an owned tensor so the source's lifetime stops mattering; take it when the buffer is short-lived and the tensor is not. And from_blob's pinned_for parameter is tag-only: it asserts pages you already page-locked for a device, letting transfers take the pinned path; it cannot pin memory for you.

Give the pool's idle reserve back

The runtime side of the memory story: the pool keeps memory it handed out and got back (MemoryStats::cached_bytes), so the next allocation is cheap. After a model unloads, or before a second model must fit beside the first, that idle reserve is memory the device (on a shared-memory part, the host) cannot use for anything else. device::release_cached_memory(device) returns it to the driver or the OS: deferred reservations drain, parked buffers retire and empty blocks release, the same three steps the runtime takes before it reports out-of-memory. Memory still in use, or whose last use has not completed on the device, is never touched; a later call can release more once that work retires. Automatic placement calls it on a failed accelerator before the CPU fallback loads.

release_cached.cpp
#include <cstdio>

#include <ClikaRT/clika_rt.h>

using ClikaRT::DataType;
using ClikaRT::Device;
using ClikaRT::Tensor;

static void report(const char* when) {
    const ClikaRT::device::MemoryStats s =
        ClikaRT::device::memory_stats(Device::cpu());
    std::printf("%-14s active %8.1f MiB, cached %8.1f MiB\n", when,
                s.active_bytes / 1048576.0, s.cached_bytes / 1048576.0);
}

int main() {
    {
        // 256 MiB of Float32 work: the pool reserves real memory for it.
        Tensor big = Tensor::zeros({64, 1024, 1024}, DataType::Float32);
        big.synchronize();
        report("in use:");
    }
    // The tensor is gone, but the pool keeps its bytes idle for reuse.
    report("dropped:");

    // Hand the idle reserve back to the system. Memory still in use, or
    // whose last use has not completed, is never touched.
    ClikaRT::device::release_cached_memory(Device::cpu());
    report("released:");
    return 0;
}

in use:        active    256.0 MiB, cached      0.0 MiB
dropped:       active      0.0 MiB, cached    256.0 MiB
released:      active      0.0 MiB, cached      0.0 MiB

The bundle's compute example covers device movement and this wrap in its 03_data_movement and 06_zero_copy chapters; the custom-operator guide uses from_blob to hand a hand-written kernel's output back to the runtime.

Borrow a buffer and compute on it​

Hand ownership over with a deleter​

Wrap non-contiguous memory with strides​

Device moves and pinned memory​

Give the pool's idle reserve back​

Borrow a buffer and compute on it

Hand ownership over with a deleter

Wrap non-contiguous memory with strides

Device moves and pinned memory

Give the pool's idle reserve back