Skip to main content

//clika-runtime/io.clika.runtime/Tokenizer/decodeBatch

decodeBatch

[common]
fun decodeBatch(sequences: List<IntArray>, skipSpecialTokens: Boolean = true): List<String>

Decode many id sequences, one string per sequence, in order.

[common]
fun decodeBatch(ids: Tensor, skipSpecialTokens: Boolean = true): List<String>

Decode an ids tensor (Int32 or Int64): a [S] sequence yields one string, a padded [B, S] batch one string per row.

[common]
fun decodeBatch(ids: Tensor, cuSeqlens: Tensor, skipSpecialTokens: Boolean = true): List<String>

Decode a flat varlen ids row sliced by its [B + 1] offset table, one string per sequence.