Skip to main content

//clika-runtime/io.clika.runtime/Encoded

Encoded

[common]
class Encoded(val ids: Array<IntArray>, val inputIds: Tensor?, val attentionMask: Tensor?, val seqLengths: Tensor?, val cuSeqlens: Tensor?)

A tensor-shaped encode: ids always carries one id row per input text; the tensor columns are present only when EncodeOptions.returnTensors was set: inputIds[B, S] padded Int32 (or one flat row with varlen), attentionMask[B, S] Int32 with 1 for a token and 0 for pad (padded layout only), seqLengths[B] Int32, the real token count per sequence, and cuSeqlens[B + 1] Int32, the exclusive prefix sums an attention over the flat row takes. Release the tensors you were given when done.

Constructors​

Encoded[common]
constructor(ids: Array<IntArray>, inputIds: Tensor?, attentionMask: Tensor?, seqLengths: Tensor?, cuSeqlens: Tensor?)

Properties​

NameSummary
attentionMask[common]
val attentionMask: Tensor?
cuSeqlens[common]
val cuSeqlens: Tensor?
ids[common]
val ids: Array<IntArray>
inputIds[common]
val inputIds: Tensor?
seqLengths[common]
val seqLengths: Tensor?