//clika-runtime/io.clika.runtime/Tokenizer/Companion/fromSentencepiece

fromSentencepiece

[common]
fun fromSentencepiece(path: String): Tokenizer

Load a bare SentencePiece model file: the pieces keep their own ids, nothing frames a sequence.