//clika-runtime/io.clika.runtime/Tokenizer/Companion/fromBertVocab

fromBertVocab

[common]
fun fromBertVocab(directory: String): Tokenizer

Load a BERT WordPiece tokenizer from a directory holding a bare vocab.txt.