Skip to main content

ClikaRT::processor

namespace

Namespaces

NameDescription
ClikaRT::processor::processor_defaultsDefault knob values the from_args factories apply when an argument is left {} (the documented HuggingFace/PIL defaults). Named here so callers see them in the header, arg.value_or(kImageDefault…), without reading the docs. The library static_asserts each against its internal default, so these and the internal defaults are one source of truth (a drift is a compile error).

Classes

NameDescription
AudioProcessOptionsPer-call overrides for AudioProcessor::process. Only the knobs the pipeline reads fresh each call are here; nothing that would invalidate the processor's precomputed window / DFT / mel-filterbank tensors or its output-shape contract (sampling rate, FFT/window geometry, mel-bank geometry, feature type). A set field replaces the loaded config's value for THAT call only. norm_mean/norm_std are scalar operands (element [0] is read); both fields OWN their values.
AudioProcessorA config-driven audio feature processor. Move-only.
AudioProcessorConfigSettable configuration for an AudioProcessor (raw waveform / log-mel / filterbank). Default-constructed to the log-mel defaults; tune the STFT/mel knobs, or round-trip a preprocessor_config.json through from_json / to_json. Move-only.
ImageProcessOptionsPer-call overrides for ImageProcessor::process / process_batch. Every field is optional: a set field replaces the loaded config's value for THAT call only; an unset field keeps the config. Resolution is field-wise; setting resize does not flip do_resize (mirror of the HuggingFace per-call kwargs). The per-image dynamic-resize path (a target computed from the input image, e.g. an aspect-preserving patch-budget fit) rides resize.
ImageProcessorA config-driven static image preprocessor. Move-only.
ImageProcessorConfigSettable configuration for an ImageProcessor (convert-rgb → resize → center-crop → rescale → normalize → pad). Default-constructed to the documented HuggingFace defaults (resize + rescale by 1/255 + normalize on; crop + pad off); tune with the fluent setters, or round-trip a preprocessor_config.json through from_json / to_json. Move-only.
ProcessedVideoA processed clip PLUS the sampling metadata a timestamp/alignment video model needs upstream (which frames were sampled, at what times). Returned by VideoProcessor::process_with_indices, the generic seam that lets a model build grid_thw / timestamp tokens from frame_indices + pts_seconds without folding any model-specific logic into ClikaRT.
ProcessorA multimodal preprocessor bundle: the OPTIONAL per-modality sub-processors a model directory provides (a tokenizer + one media processor: audio / image / video). Move-only.
VideoMetadataTiming metadata for a decoded clip (only fps-based sampling consults it).
VideoProcessOptionsPer-call overrides for VideoProcessor::process: the temporal sampling knobs, plus per-frame spatial overrides forwarded to the inner image pipeline. A set field replaces the loaded config's value for THAT call only.
VideoProcessorA config-driven video preprocessor: temporal frame sampling composed over the channels-last image pipeline. Move-only.
VideoProcessorConfigSettable configuration for a VideoProcessor: the per-frame ImageProcessorConfig plus temporal sampling (num_frames / fps). Move-only.

Enumerations

enum ImageResample

enum class ImageResample : std::uint8_t

Resampling filter for the image resize step, in the PIL/HuggingFace resample vocabulary. Enum values ARE the PIL codes a preprocessor_config.json carries, so a config round-trips exactly.

EnumeratorValueDescription
Nearest0PIL NEAREST.
Bilinear2PIL BILINEAR.
Bicubic3PIL BICUBIC (the HuggingFace default).

Declared in ClikaRT/processor/config.h, line 36

enum AudioFeatureType

enum class AudioFeatureType : std::uint8_t

Which standard feature pipeline an AudioProcessor runs.

EnumeratorValueDescription
Raw0raw mono waveform [1, S] + normalization
LogMellog-mel spectrogram [1, T, n_mels]
KaldiFbankkaldi-style log-mel filterbank [1, T, n_mels]

Declared in ClikaRT/processor/config.h, line 146

enum MelScale

enum class MelScale : std::uint8_t

Mel-scale hertz↔mel mapping (the analytic curve behind the filterbank).

EnumeratorValueDescription
Htk0
Slaney
Kaldi

Declared in ClikaRT/processor/config.h, line 153

enum MelNorm

enum class MelNorm : std::uint8_t

Mel filterbank area normalization.

EnumeratorValueDescription
None0
Slaney

Declared in ClikaRT/processor/config.h, line 156

enum LogMode

enum class LogMode : std::uint8_t

How the mel/power spectrum is compressed before normalization.

EnumeratorValueDescription
None0leave the (linear) power/mel spectrum
Lognatural log
Log10base-10 log
Db10·log10 (power → decibels)

Declared in ClikaRT/processor/config.h, line 159

enum WindowFunction

enum class WindowFunction : std::uint8_t

STFT analysis window function (rectangular = None).

EnumeratorValueDescription
Hann0Hann cosine window (the default).
HammingHamming window.
Nonerectangular (no windowing)
PoveyKaldi's Povey window (a Hann-family power variant).

Declared in ClikaRT/processor/config.h, line 167

enum StftPadStyle

enum class StftPadStyle : std::uint8_t

How the center pad distributes around the signal: symmetric n_fft/2 each side (the torch STFT convention), or LEFT-only n_fft/2 (semicausal, a streaming front-end sees no future samples).

EnumeratorValueDescription
Center0
Semicausal

Declared in ClikaRT/processor/config.h, line 177

enum LogFloorMode

enum class LogFloorMode : std::uint8_t

How the mel floor guards the log: clamp to the floor (log(max(x, floor))) or add it (log(x + floor)).

EnumeratorValueDescription
Clamp0
Additive

Declared in ClikaRT/processor/config.h, line 181

enum AudioNormKind

enum class AudioNormKind : std::uint8_t

Output normalization kind (operands set alongside).

EnumeratorValueDescription
None0
ZeroMeanUnitVarper-utterance zero-mean unit-variance
GlobalMeanStdsubtract norm_mean, divide by norm_std (scalar operands)

Declared in ClikaRT/processor/config.h, line 184

ClikaRT/processor/config.h

#include <ClikaRT/processor/config.h>

Public, settable configuration for the media preprocessors. Build a processor from EXPLICIT arguments instead of a model directory: default-construct a config (sensible defaults, no file needed), tune the knobs, and hand it to the matching processor's from_config. Or inspect what a loaded config resolved to by serializing it with to_json.

Each config type PIMPL-wraps its internal parameter set; the accepted schema round-trips through from_json / to_json (a preprocessor_config.json-shaped object), and the common knobs have direct fluent setters.

ClikaRT/tokenizer/tokenizer.h

#include <ClikaRT/tokenizer/tokenizer.h>

Text <-> token-id conversion.