Tokenizer

type Tokenizer struct {
	// contains filtered or unexported fields
}

Tokenizer is a `clika_rt_tokenizer*` handle (single-owner).

Methods

Close

func (h *Tokenizer) Close()

Close frees the single-owner handle; safe to call twice.