Skip to main content

ClikaRT::runtime::ExecConfig

struct

Header: ClikaRT/runtime/serving.h

One executor/node's configuration.

Data members​

batcher​

BatcherConfig batcher

batching knobs

Declared in ClikaRT/runtime/serving.h, line 89

scheduler​

SchedulerConfig scheduler

scheduling knobs

Declared in ClikaRT/runtime/serving.h, line 90

stream​

std::optional<Stream> stream

The stream this node runs on. Unset = a new asynchronous stream, created for this node (a CPU stream when the node's model device is unknown to the executor; the serving pipeline sets each node's stream from its model). Pass Stream::default_stream(device) only to opt into synchronous execution. A pipeline warns when steps share a stream or run on a synchronous one.

Declared in ClikaRT/runtime/serving.h, line 97

max_queued​

std::size_t max_queued = 0

in-flight cap for backpressure (0 = unbounded)

Declared in ClikaRT/runtime/serving.h, line 98