---
title: "ClikaRT::tokenizer::Encoded"
sidebar_label: "Encoded"
description: "Tensor-shaped encode output. ids always carries the per-sequence id rows (one inner vector per input text)."
---

<!-- Generated by tools/doxygen2mdx from Doxygen XML. Do not edit. -->

<span class="api-kind api-kind--struct">struct</span>

**Header:** `ClikaRT/tokenizer/tokenizer.h`

Tensor-shaped encode output. [`ids`](./Encoded.md#ids) always carries the per-sequence id rows (one inner vector per input text). The tensor columns are present only when [`EncodeOptions::return_tensors`](./EncodeOptions.md#return_tensors) is set:

- [`input_ids`](./Encoded.md#input_ids): `[B, S]` padded Int32 (per [`EncodeOptions::padding_side`](./EncodeOptions.md#padding_side), right by default) or `[ΣS]` flat Int32 (varlen).
- [`attention_mask`](./Encoded.md#attention_mask): `[B, S]` Int32, 1 for a token / 0 for pad (padded path only).
- [`seq_lengths`](./Encoded.md#seq_lengths): `[B]` Int32, the per-sequence token counts.
- [`cu_seqlens`](./Encoded.md#cu_seqlens): `[B + 1]` Int32, the exclusive prefix-sum offset table, the varlen attention contract's index dtype, so it feeds attention directly.

## Data members

### ids {#ids}

<div class="api-signature">

`std::vector<std::vector<std::int32_t>>` **`ids`**

</div>

per-sequence id rows; always present

<div class="api-declared">

Declared in `ClikaRT/tokenizer/tokenizer.h`, line 81

</div>

### input\_ids {#input_ids}

<div class="api-signature">

`std::optional<`[`Tensor`](../Tensor.md)`>` **`input_ids`**

</div>

\[B, S\] padded or \[SumS\] varlen Int32 (see the struct block)

<div class="api-declared">

Declared in `ClikaRT/tokenizer/tokenizer.h`, line 82

</div>

### attention\_mask {#attention_mask}

<div class="api-signature">

`std::optional<`[`Tensor`](../Tensor.md)`>` **`attention_mask`**

</div>

\[B, S\] Int32, 1 = token / 0 = pad (padded path only)

<div class="api-declared">

Declared in `ClikaRT/tokenizer/tokenizer.h`, line 83

</div>

### seq\_lengths {#seq_lengths}

<div class="api-signature">

`std::optional<`[`Tensor`](../Tensor.md)`>` **`seq_lengths`**

</div>

\[B\] Int32 per-sequence token counts

<div class="api-declared">

Declared in `ClikaRT/tokenizer/tokenizer.h`, line 84

</div>

### cu\_seqlens {#cu_seqlens}

<div class="api-signature">

`std::optional<`[`Tensor`](../Tensor.md)`>` **`cu_seqlens`**

</div>

\[B + 1\] Int32 exclusive prefix sums (varlen attention's table)

<div class="api-declared">

Declared in `ClikaRT/tokenizer/tokenizer.h`, line 85

</div>