---
title: "attention"
sidebar_label: "attention"
description: "Kotlin binding reference: attention."
---

<!-- Generated by tools/api_reference/generate_api_docs.py. Do not edit. -->

//[clika-runtime](../../../index.md)/[io.clika.runtime](../index.md)/[Ops](index.md)/[attention](attention.md)

# attention

[common]\
fun [attention](attention.md)(query: [Tensor](../Tensor/index.md), key: [Tensor](../Tensor/index.md), value: [Tensor](../Tensor/index.md), attnMask: [Tensor](../Tensor/index.md)? = null, headSink: [Tensor](../Tensor/index.md)? = null, isCausal: [Boolean](https://kotlinlang.org/api/core/kotlin-stdlib/kotlin/-boolean/index.html)? = null, qScale: [Tensor](../Tensor/index.md)? = null, softcap: [Double](https://kotlinlang.org/api/core/kotlin-stdlib/kotlin/-double/index.html)? = null, slidingWindow: [Long](https://kotlinlang.org/api/core/kotlin-stdlib/kotlin/-long/index.html)? = null, smoothSoftmax: [Boolean](https://kotlinlang.org/api/core/kotlin-stdlib/kotlin/-boolean/index.html)? = null, kScale: [Tensor](../Tensor/index.md)? = null, vScale: [Tensor](../Tensor/index.md)? = null, keptPrefix: [Tensor](../Tensor/index.md)? = null): [Tensor](../Tensor/index.md)

`attention(query: Tensor, key: Tensor, value: Tensor, attnMask: Tensor? = null, headSink: Tensor? = null, isCausal: Boolean? = null, qScale: Tensor? = null, softcap: Double? = null, slidingWindow: Long? = null, smoothSoftmax: Boolean? = null, kScale: Tensor? = null, vScale: Tensor? = null, keptPrefix: Tensor? = null)`: the `attention` operator. Dense attention with the serving riders: per-head sink, logit soft-cap, sliding window, smoothed softmax. The core is `scaled_dot_product_attention`; each rider adjusts the softmax stage: - `head_sink``[H_q]`: a per-head virtual logit folded into the softmax denominator (attention that can &quot;go nowhere&quot;). - `softcap`: logits pass through `cap * tanh(x / cap)` before the softmax. - `sliding_window`: each query attends only the last N key positions. - `smooth_softmax`: adds one to the softmax denominator. Default `false`.