clika_runtime.nn functions

fuse_linears

fuse_linears(*layers: 'Linear', name: 'str | None' = None, activation: 'ActivationLike' = None) -> 'Linear'

fuse_linears(*layers, name=None, *, activation=None) -> Linear

Fuse layers (two or more :class:~clika_runtime.nn.Linear layers over the same input width, bound and not yet run) into one matmul over their concatenated weights, and return the group as a :class:~clika_runtime.nn.Linear: its forward maps [*, in_features] to [*, sum(out_features)], the parts' outputs concatenated in the order given (split it with :func:~clika_runtime.split_with_sizes).

The parts keep their state-dict names and their weight views, each reading back its rows of the fused weight (a view held from before the fuse becomes storage-free and answers its metadata; part.weight reconstructs the rows on demand), so state_dict() of the model stays the checkpoint's and the group's pack is the one resident copy of every fused weight; the group itself lists no parameters (group.state_dict() is empty), so holding it as a module attribute adds nothing to a save. After the fuse the parts no longer serve a forward of their own: calling a part raises :class:~clika_runtime.ClikaRTError until a load_state_dict over it dissolves the group (the group then serves the parts side by side plus the concatenation). A group the runtime cannot fuse (mixed quantization schemes, unequal input widths) is returned in that same shape: the output never depends on whether the fusion engaged, only the dispatch count does.

activation fuses an epilogue into the group's forward; a gated one ("swiglu", "geglu", "reglu") emits half the concatenated width, the gate / up convention, and needs bias-free parts. name labels the group in its repr.

Raises: TypeError: when an argument is not a :class:~clika_runtime.nn.Linear. ValueError: when fewer than two layers are given. ClikaRTError: when a part is unbound or already packed (fuse before the first forward), or the parts mix biased and bias-free layers.