//clika-runtime/io.clika.runtime/KVSpecOptions
KVSpecOptions
[common]
data class KVSpecOptions(val batch: Long = 1, val pastLenMin: Long = 0, val pastLenOpt: Long = 0, val pastLenMax: Long = 0)
The session cache sizing of a model served from a graph with a KV cache: the batch every cache input is specced at and the past length as a min / opt / max range (the one-value form fixes it). A graph without a KV cache ignores it.
Constructors
| KVSpecOptions | [common] constructor(batch: Long, pastLen: Long) Every cache input at batch with a fixed pastLen. constructor(batch: Long = 1, pastLenMin: Long = 0, pastLenOpt: Long = 0, pastLenMax: Long = 0) |
Properties
| Name | Summary |
|---|---|
| batch | [common] val batch: Long = 1 |
| pastLenMax | [common] val pastLenMax: Long = 0 |
| pastLenMin | [common] val pastLenMin: Long = 0 |
| pastLenOpt | [common] val pastLenOpt: Long = 0 |