Skip to main content

ClikaRT::nn::RotaryEmbeddingOptions

struct

Header: ClikaRT/nn/rotary_embedding.h

Settings of RotaryEmbedding: chain the setters, or assign the fields. The defaults are the plain rotary embedding of most decoder checkpoints: base 10000, no context scaling, half-split pairs.

Member functions​

theta(double)​

RotaryEmbeddingOptions& theta(double v)

Sets theta, the frequency base.

Declared in ClikaRT/nn/rotary_embedding.h, line 37

scaling(ops::RopeScaling)​

RotaryEmbeddingOptions& scaling(ops::RopeScaling v)

Sets scaling, the context-extension family.

Declared in ClikaRT/nn/rotary_embedding.h, line 39

scale(double)​

RotaryEmbeddingOptions& scale(double v)

Sets scale, the family's scale factor.

Declared in ClikaRT/nn/rotary_embedding.h, line 41

low_freq_factor(double)​

RotaryEmbeddingOptions& low_freq_factor(double v)

Sets low_freq_factor (the Llama3 family).

Declared in ClikaRT/nn/rotary_embedding.h, line 43

high_freq_factor(double)​

RotaryEmbeddingOptions& high_freq_factor(double v)

Sets high_freq_factor (the Llama3 family).

Declared in ClikaRT/nn/rotary_embedding.h, line 45

original_max_pos(int64_t)​

RotaryEmbeddingOptions& original_max_pos(std::int64_t v)

Sets original_max_pos, the context length the checkpoint was trained at.

Declared in ClikaRT/nn/rotary_embedding.h, line 50

beta_fast(double)​

RotaryEmbeddingOptions& beta_fast(double v)

Sets beta_fast (the YaRN family).

Declared in ClikaRT/nn/rotary_embedding.h, line 55

beta_slow(double)​

RotaryEmbeddingOptions& beta_slow(double v)

Sets beta_slow (the YaRN family).

Declared in ClikaRT/nn/rotary_embedding.h, line 57

mode(ops::RotaryMode)​

RotaryEmbeddingOptions& mode(ops::RotaryMode v)

Sets mode, the pair layout the rotation uses.

Declared in ClikaRT/nn/rotary_embedding.h, line 59

device(Device)​

RotaryEmbeddingOptions& device(Device v)

Sets device, where the tables live.

Declared in ClikaRT/nn/rotary_embedding.h, line 61

theta()​

double theta() const

Reads theta.

Declared in ClikaRT/nn/rotary_embedding.h, line 64

scaling()​

ops::RopeScaling scaling() const

Reads scaling.

Declared in ClikaRT/nn/rotary_embedding.h, line 66

scale()​

double scale() const

Reads scale.

Declared in ClikaRT/nn/rotary_embedding.h, line 68

low_freq_factor()​

double low_freq_factor() const

Reads low_freq_factor.

Declared in ClikaRT/nn/rotary_embedding.h, line 70

high_freq_factor()​

double high_freq_factor() const

Reads high_freq_factor.

Declared in ClikaRT/nn/rotary_embedding.h, line 72

original_max_pos()​

std::int64_t original_max_pos() const

Reads original_max_pos.

Declared in ClikaRT/nn/rotary_embedding.h, line 74

beta_fast()​

double beta_fast() const

Reads beta_fast.

Declared in ClikaRT/nn/rotary_embedding.h, line 76

beta_slow()​

double beta_slow() const

Reads beta_slow.

Declared in ClikaRT/nn/rotary_embedding.h, line 78

mode()​

ops::RotaryMode mode() const

Reads mode.

Declared in ClikaRT/nn/rotary_embedding.h, line 80

device()​

const Device& device() const

Reads device.

Declared in ClikaRT/nn/rotary_embedding.h, line 82

Data members​

theta_​

double theta_ = 10000.0

The frequency base. Default 10000.0.

Declared in ClikaRT/nn/rotary_embedding.h, line 85

scaling_​

ops::RopeScaling scaling_ = ops::RopeScaling::None

The context-extension family. Default ops::RopeScaling::None.

Declared in ClikaRT/nn/rotary_embedding.h, line 87

scale_​

double scale_ = 1.0

The family's scale factor. Default 1.0.

Declared in ClikaRT/nn/rotary_embedding.h, line 89

low_freq_factor_​

double low_freq_factor_ = 1.0

Llama3: the low-frequency factor. Default 1.0.

Declared in ClikaRT/nn/rotary_embedding.h, line 91

high_freq_factor_​

double high_freq_factor_ = 4.0

Llama3: the high-frequency factor. Default 4.0.

Declared in ClikaRT/nn/rotary_embedding.h, line 93

original_max_pos_​

std::int64_t original_max_pos_ = 0

The context length the checkpoint was trained at, read by the Llama3 / YaRN / LongRoPE families. Default 0 (not set).

Declared in ClikaRT/nn/rotary_embedding.h, line 96

beta_fast_​

double beta_fast_ = 32.0

YaRN: the fast beta. Default 32.0.

Declared in ClikaRT/nn/rotary_embedding.h, line 98

beta_slow_​

double beta_slow_ = 1.0

YaRN: the slow beta. Default 1.0.

Declared in ClikaRT/nn/rotary_embedding.h, line 100

mode_​

ops::RotaryMode mode_ = ops::RotaryMode::NeoX

The pair layout: NeoX pairs entry i with i + dim / 2, Interleaved pairs 2i with 2i + 1. Default ops::RotaryMode::NeoX.

Declared in ClikaRT/nn/rotary_embedding.h, line 103

device_​

Device device_ = Device::cpu()

Where the tables live. Default CPU.

Declared in ClikaRT/nn/rotary_embedding.h, line 105