Kernels

Source: lib/Kernels.cs

This page is generated from the source by scripts/gen-reference.py.

SIMD math kernels for LLM inference, in --stdlib:zero.

Each kernel has three paths, picked at compile time:

arm64: NEON (AdvSimd), 4 floats per vector; DotI8 uses SDOT with -m dotprod
x64:   AVX2 + FMA, 8 floats per vector (needs -m x86-64-v3)
other: scalar

Any length works: vector loops cover the bulk, a scalar loop the tail.

Exp is Cephes' expf: reduce by ln 2, degree-6 polynomial, scale by 2n through the exponent bits. Max relative error is a few ulp (samples/Kernels.cs measures it). Inputs are clamped to [-87.33, 88.72], so results below 2-126 come out as ~1.2e-38 instead of a subnormal or 0, which doesn't matter for softmax or SiLU.

All kernels take raw pointers and are marked unsafe: callers guarantee each pointer covers n elements (rows * cols for the matrix).

Members

Path

public static string Path

Dot

public static unsafe float Dot(float* a, float* b, int n)

Sum of a[i] * b[i].

Safety: a and b must each cover n floats.

MatVec

public static unsafe void MatVec(
    float* y,
    float* W,
    float* x,
    int rows,
    int cols)

y[r] = dot(W[r, :], x) for a row-major rows x cols matrix W.

Safety: y covers rows floats, W covers rows * cols floats, x covers cols floats.

DotI8

public static unsafe int DotI8(sbyte* a, sbyte* b, int n)

Exact int8 dot product. Values must be in [-127, 127] (the x64 path needs it).

Safety: a and b must each cover n bytes.

Exp

public static unsafe void Exp(float* x, float* y, int n)

y[i] = exp(x[i]). x and y may be the same buffer.

Safety: x and y must each cover n floats.

Softmax

public static unsafe void Softmax(float* x, int n)

In place: x = exp(x - max) / sum.

Safety: x must cover n floats, n > 0.

Silu

public static unsafe void Silu(float* x, int n)

In place: x = x * sigmoid(x) = x / (1 + exp(-x)).

Safety: x must cover n floats.

RmsNorm

public static unsafe void RmsNorm(
    float* y,
    float* x,
    float* weight,
    int n,
    float eps)

y = x / sqrt(mean(x^2) + eps) * weight.

Safety: y, x and weight must each cover n floats; y may equal x.