ClikaRT::ops::mla_attention

function

mla_attention()

Tensor mla_attention(
    Tensor q_nope,
    Tensor q_pe,
    Tensor new_ckv,
    Tensor new_kpe,
    Tensor ckv_cache,
    Tensor kpe_cache,
    Tensor kvcache_start,
    Tensor cu_seqlens_q,
    double scale,
    OptionalTensor slot_ids = {}
)

Multi-head Latent Attention over a compressed KV cache, latent-space end to end: q_nope [ΣS, H, Dl] (W_UK-absorbed) + q_pe [ΣS, H, Dr] score against the per-token compressed rows; the caches ([max_seqs, max_seq, Dl/Dr]) take this step's new_ckv/new_kpe appends IN PLACE (write offsets = kvcache_start [B] Int32; cu_seqlens_q [B+1] locates each sequence's packed tokens); the returned [ΣS, H, Dl] output stays latent (apply the W_UV un-absorption after). scale is REQUIRED; the absorbed query's magnitude lives in the model's un-absorbed head dim.

Declared in ClikaRT/compute/ops.h, line 3797