//clika-runtime/io.clika.runtime/Tokenizer/Companion/fromHuggingface

fromHuggingface

[common]
fun fromHuggingface(path: String): Tokenizer

Load a Hugging Face tokenizer from a model directory or one artifact file, with the sibling tokenizer_config.json read for the special ids and the chat template when present.