//clika-runtime/io.clika.runtime/Processor/tokenizer

tokenizer

[common]
fun tokenizer(): Tokenizer

The tokenizer the model directory shipped, with the special ids its configuration names.