---
title: "tokenizer"
sidebar_label: "tokenizer"
description: "Kotlin binding reference: tokenizer."
---

<!-- Generated by tools/api_reference/generate_api_docs.py. Do not edit. -->

//[clika-runtime](../../../index.md)/[io.clika.runtime](../index.md)/[Processor](index.md)/[tokenizer](tokenizer.md)

# tokenizer

[common]\
fun [tokenizer](tokenizer.md)(): [Tokenizer](../Tokenizer/index.md)

The tokenizer the model directory shipped, with the special ids its configuration names.