Linalg Tools

Vector and matrix operations for embeddings and numeric analysis — no LLM, no credits, fully deterministic.

Linalg tools operate entirely in-process. There are no external calls, no AI generation, and no credit cost. The heavy decompositions (svd, pca) run in a native Rust NIF (nalgebra); the rest are pure Elixir. Every response includes a deterministic receipt under _meta.datagrout so the gateway can verify the operation was reproducible. All Linalg tools are available at data-grout@1/linalg.*@1.

The suite covers two areas:

• Similarity & distance: cosine_sim, distance, project — compare vectors, rank against a query, or decompose onto a basis.
• Structure & reduction: cluster, svd, pca — group vectors, factor matrices, and reduce dimensionality.

Vector inputs resolve from explicit arguments (a/b/vector/vectors/matrix/query) or from payload/cache_ref, so you can pipe embeddings or Data/Frame tool output straight in without re-sending through the LLM context. Non-numeric entries are filtered; ragged matrices are rejected with a clear error.

Binary payload input

Alongside JSON arrays, every numeric input X also accepts X_b64 — a base64 string of packed little-endian IEEE-754 floats. The dtype argument selects float64 (default) or float32. A matrix X_b64 additionally needs a column count: X_cols (or the shared cols), interpreted row-major. This is a compact, exact alternative for large embedding sets. An explicit JSON array under X always wins over X_b64.

linalg.cosine_sim@1

Cosine similarity between numeric vectors, in three modes. Returns values in [-1, 1]. Max 5,000 vectors of 4,096 dims.

• Pair — pass a and b → a single similarity score.
• Query — pass matrix and query → each row’s similarity to the query, ranked descending, with optional top_k (semantic nearest-neighbour).
• Pairwise — pass matrix alone → the full symmetric similarity matrix.

A zero-magnitude vector yields 0.0 (never a divide-by-zero).

Parameters

Example

{
  "name": "data-grout@1/linalg.cosine_sim@1",
  "arguments": { "matrix": "$embeddings.vectors", "query": [0.12, -0.04, 0.88], "top_k": 5 }
}

Response (query mode): { "records": [{ "index": 7, "similarity": 0.94 }, ...], "n": 500 }.

linalg.distance@1

Distance between numeric vectors. Metrics: euclidean (L2, default), manhattan (L1), chebyshev (L∞), cosine (1 − cosine similarity). Same three modes as cosine_sim; query mode ranks rows ascending (nearest first). Max 5,000 vectors of 4,096 dims.

Parameters

Example

{
  "name": "data-grout@1/linalg.distance@1",
  "arguments": { "a": [0, 0], "b": [3, 4], "metric": "euclidean" }
}

Response: { "distance": 5.0, "metric": "euclidean", "n": 2 }.

linalg.project@1

Project vectors onto a basis. For each input vector, returns its coordinates in the basis (the dot products), the reconstruction (Σ coord·basis), and the residual with its residual_norm. Set orthonormal: true only when the basis is already unit-length and orthogonal (skips per-basis normalization). Max 5,000 vectors, 512 basis vectors, 4,096 dims.

Parameters

Example

{
  "name": "data-grout@1/linalg.project@1",
  "arguments": { "vector": [3, 4], "basis": [[1, 0], [0, 1]], "orthonormal": true }
}

Response: { "records": [{ "index": 0, "coordinates": [3, 4], "reconstruction": [3, 4], "residual": [0, 0], "residual_norm": 0 }], "basis_size": 2, "dims": 2 }.

linalg.cluster@1

K-means clustering (Lloyd’s algorithm) with deterministic seeding — the same input and k always produce the same clusters, so results are reproducible across runs, servers, and time. Returns per-vector assignments, final centroids, cluster sizes, total inertia, and iterations run. Empty clusters keep their previous centroid so k stays fixed. Max 20,000 vectors of 4,096 dims.

Parameters

Example

{
  "name": "data-grout@1/linalg.cluster@1",
  "arguments": { "vectors": "$pca.scores", "k": 4 }
}

Response: { "k": 4, "records": [{ "index": 0, "cluster": 2 }, ...], "centroids": [[...]], "sizes": [...], "inertia": 12.4, "iterations": 6 }.

linalg.svd@1

Thin Singular Value Decomposition of a matrix: A = U · diag(S) · Vᵀ (native, nalgebra). Returns u (left singular vectors as columns, rows × k), s (singular values, descending), vt (right singular vectors, k × cols), and each singular value’s explained_variance_ratio. Pass k to truncate to the top triples. Max 4,000,000 cells.

Parameters

Example

{
  "name": "data-grout@1/linalg.svd@1",
  "arguments": { "matrix": [[1, 0], [0, 1], [1, 1]] }
}

Response: { "s": [1.732051, 1.0], "explained_variance_ratio": [0.75, 0.25], "u": [[...]], "vt": [[...]], "rows": 3, "cols": 2, "k": 2 }.

linalg.pca@1

Principal Component Analysis (SVD-based, native). Given vectors as n samples × d features, centers each feature, then decomposes to obtain the top-k principal components. Returns components (principal axes), explained_variance and explained_variance_ratio per component, the centering mean, singular_values, and scores (samples projected into PC space). Explained-variance ratios are computed against the total variance, so they stay correct even when k truncates. Needs ≥ 2 samples. Max 4,000,000 cells.

Parameters

Example

{
  "name": "data-grout@1/linalg.pca@1",
  "arguments": { "vectors": "$embeddings.vectors", "k": 2 }
}

Response: { "n_samples": 500, "n_features": 768, "k": 2, "components": [[...]], "explained_variance_ratio": [0.61, 0.18], "mean": [...], "scores": [[...]] }.

linalg.normalize@1

Scales vectors to unit norm. ord selects the norm: l2 (default, Euclidean), l1 (sum of absolute values), or max (L∞). Pass a single vector or a vectors matrix (each row normalized independently). This is vector normalization — distinct from math.normalize, which rescales the distribution of a series (z-score / min-max / percentile). A zero vector is returned unchanged (norm 0), never divided by zero. Max 20,000 vectors of 4,096 dims.

Parameters

Example

{
  "name": "data-grout@1/linalg.normalize@1",
  "arguments": { "vector": [3, 4] }
}

Response: { "vector": [0.6, 0.8], "norm": 5.0, "ord": "l2" }.

linalg.matmul@1

Matrix product A · B (native, nalgebra). For ergonomics it also does matrix–vector and dot products by coercing 1-D inputs: a flat a is treated as a row vector (1 × n), a flat b as a column vector (n × 1). The 2-D product is always returned; a single-row/column result is also flattened into vector, and a 1 × 1 result into scalar. Errors on inner-dimension mismatch. Max 4,000,000 cells per operand.

Parameters

Example

{
  "name": "data-grout@1/linalg.matmul@1",
  "arguments": { "a": [[1, 0], [0, 2]], "b": [3, 4] }
}

Response: { "product": [[3], [8]], "vector": [3, 8], "rows": 2, "cols": 1, "inner": 2 }.

Composition

Pull embeddings or numeric columns through Data / Frame tools, then rank, reduce, or cluster them here. pca.scores, cluster assignments, and the pairwise similarity matrix are all chart-ready — pass them to Prism Chart for visualization. Pair with Math for descriptive statistics on the projected components, or with Signal when spectral features feed clustering or similarity.