Skip to main content

//clika-runtime/io.clika.runtime/Tokenizer/encodeChat

encodeChat

[common]
fun encodeChat(messages: String, addGenerationPrompt: Boolean = true, addSpecialTokens: Boolean = false): IntArray

Render chat messages with the template and encode the prompt in one step. The template writes its own bos and eos framing, so the ids are encoded without the special tokens by default; pass addSpecialTokens only for a template that writes no bos itself.

[common]
fun encodeChat(inputs: ChatTemplateInputs, options: ChatTemplateOptions? = null, addSpecialTokens: Boolean = false): IntArray

Render and encode with the full input set and, when given, per-render options.