{
  "access": "public",
  "type": "reference",
  "format": "markdown",
  "title": "Linalg Tools Reference",
  "chunked": true,
  "url": "https://library.datagrout.ai/linalg-tools",
  "summary": "Vector and matrix operations for embeddings and numeric analysis — no LLM, no credits, fully deterministic.",
  "content_markdown": "# Linalg Tools\n\nVector and matrix operations for embeddings and numeric analysis — no LLM, no credits, fully deterministic.\n\nLinalg tools operate entirely in-process. There are no external calls, no AI generation, and no credit cost. The heavy decompositions (`svd`, `pca`) run in a native Rust NIF (nalgebra); the rest are pure Elixir. Every response includes a deterministic receipt under `_meta.datagrout` so the gateway can verify the operation was reproducible. All Linalg tools are available at `data-grout@1/linalg.*@1`.\n\nThe suite covers two areas:\n\n- **Similarity & distance**: `cosine_sim`, `distance`, `project` — compare vectors, rank against a query, or decompose onto a basis.\n- **Structure & reduction**: `cluster`, `svd`, `pca` — group vectors, factor matrices, and reduce dimensionality.\n\nVector inputs resolve from explicit arguments (`a`/`b`/`vector`/`vectors`/`matrix`/`query`) or from `payload`/`cache_ref`, so you can pipe embeddings or Data/Frame tool output straight in without re-sending through the LLM context. Non-numeric entries are filtered; ragged matrices are rejected with a clear error.\n\n### Binary payload input\n\nAlongside JSON arrays, every numeric input `X` also accepts `X_b64` — a base64 string of packed **little-endian** IEEE-754 floats. The `dtype` argument selects `float64` (default) or `float32`. A **matrix** `X_b64` additionally needs a column count: `X_cols` (or the shared `cols`), interpreted row-major. This is a compact, exact alternative for large embedding sets. An explicit JSON array under `X` always wins over `X_b64`.\n\n---\n\n## `linalg.cosine_sim@1`\n\nCosine similarity between numeric vectors, in three modes. Returns values in `[-1, 1]`. Max 5,000 vectors of 4,096 dims.\n\n- **Pair** — pass `a` and `b` → a single `similarity` score.\n- **Query** — pass `matrix` and `query` → each row's similarity to the query, ranked descending, with optional `top_k` (semantic nearest-neighbour).\n- **Pairwise** — pass `matrix` alone → the full symmetric similarity `matrix`.\n\nA zero-magnitude vector yields `0.0` (never a divide-by-zero).\n\n### Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `a`, `b` | number[] | pair mode | Two equal-length vectors |\n| `matrix` | number[][] | query/pairwise | List of vectors |\n| `query` | number[] | query mode | Vector to rank rows against |\n| `top_k` | integer | no | In query mode, return only the K most similar rows |\n\n### Example\n\n```json\n{\n  \"name\": \"data-grout@1/linalg.cosine_sim@1\",\n  \"arguments\": { \"matrix\": \"$embeddings.vectors\", \"query\": [0.12, -0.04, 0.88], \"top_k\": 5 }\n}\n```\n\nResponse (query mode): `{ \"records\": [{ \"index\": 7, \"similarity\": 0.94 }, ...], \"n\": 500 }`.\n\n---\n\n## `linalg.distance@1`\n\nDistance between numeric vectors. Metrics: `euclidean` (L2, default), `manhattan` (L1), `chebyshev` (L∞), `cosine` (1 − cosine similarity). Same three modes as `cosine_sim`; query mode ranks rows **ascending** (nearest first). Max 5,000 vectors of 4,096 dims.\n\n### Parameters\n\n| Parameter | Type | Required | Default | Description |\n|-----------|------|----------|---------|-------------|\n| `a`, `b` | number[] | pair mode | -- | Two equal-length vectors |\n| `matrix` | number[][] | query/pairwise | -- | List of vectors |\n| `query` | number[] | query mode | -- | Vector to measure rows against |\n| `metric` | string | no | `euclidean` | `euclidean` \\| `manhattan` \\| `chebyshev` \\| `cosine` |\n| `top_k` | integer | no | -- | In query mode, return only the K nearest rows |\n\n### Example\n\n```json\n{\n  \"name\": \"data-grout@1/linalg.distance@1\",\n  \"arguments\": { \"a\": [0, 0], \"b\": [3, 4], \"metric\": \"euclidean\" }\n}\n```\n\nResponse: `{ \"distance\": 5.0, \"metric\": \"euclidean\", \"n\": 2 }`.\n\n---\n\n## `linalg.project@1`\n\nProject vectors onto a basis. For each input vector, returns its `coordinates` in the basis (the dot products), the `reconstruction` (Σ coord·basis), and the `residual` with its `residual_norm`. Set `orthonormal: true` only when the basis is already unit-length and orthogonal (skips per-basis normalization). Max 5,000 vectors, 512 basis vectors, 4,096 dims.\n\n### Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `basis` | number[][] | yes | Basis vectors to project onto |\n| `vectors` | number[][] | one of | Vectors to project |\n| `vector` | number[] | one of | A single vector (convenience) |\n| `orthonormal` | boolean | no | Treat the basis as orthonormal (default `false`) |\n\n### Example\n\n```json\n{\n  \"name\": \"data-grout@1/linalg.project@1\",\n  \"arguments\": { \"vector\": [3, 4], \"basis\": [[1, 0], [0, 1]], \"orthonormal\": true }\n}\n```\n\nResponse: `{ \"records\": [{ \"index\": 0, \"coordinates\": [3, 4], \"reconstruction\": [3, 4], \"residual\": [0, 0], \"residual_norm\": 0 }], \"basis_size\": 2, \"dims\": 2 }`.\n\n---\n\n## `linalg.cluster@1`\n\nK-means clustering (Lloyd's algorithm) with **deterministic seeding** — the same input and `k` always produce the same clusters, so results are reproducible across runs, servers, and time. Returns per-vector assignments, final centroids, cluster sizes, total inertia, and iterations run. Empty clusters keep their previous centroid so `k` stays fixed. Max 20,000 vectors of 4,096 dims.\n\n### Parameters\n\n| Parameter | Type | Required | Default | Description |\n|-----------|------|----------|---------|-------------|\n| `vectors` | number[][] | yes | -- | Vectors to cluster (or via payload/cache_ref) |\n| `k` | integer | yes | -- | Number of clusters (1 ≤ k ≤ number of vectors) |\n| `max_iters` | integer | no | `50` | Maximum Lloyd iterations |\n\n### Example\n\n```json\n{\n  \"name\": \"data-grout@1/linalg.cluster@1\",\n  \"arguments\": { \"vectors\": \"$pca.scores\", \"k\": 4 }\n}\n```\n\nResponse: `{ \"k\": 4, \"records\": [{ \"index\": 0, \"cluster\": 2 }, ...], \"centroids\": [[...]], \"sizes\": [...], \"inertia\": 12.4, \"iterations\": 6 }`.\n\n---\n\n## `linalg.svd@1`\n\nThin Singular Value Decomposition of a matrix: A = U · diag(S) · Vᵀ (native, nalgebra). Returns `u` (left singular vectors as columns, rows × k), `s` (singular values, descending), `vt` (right singular vectors, k × cols), and each singular value's `explained_variance_ratio`. Pass `k` to truncate to the top triples. Max 4,000,000 cells.\n\n### Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `matrix` | number[][] | yes | Row-major matrix (or via payload/cache_ref) |\n| `k` | integer | no | Keep only the top-k singular triples |\n\n### Example\n\n```json\n{\n  \"name\": \"data-grout@1/linalg.svd@1\",\n  \"arguments\": { \"matrix\": [[1, 0], [0, 1], [1, 1]] }\n}\n```\n\nResponse: `{ \"s\": [1.732051, 1.0], \"explained_variance_ratio\": [0.75, 0.25], \"u\": [[...]], \"vt\": [[...]], \"rows\": 3, \"cols\": 2, \"k\": 2 }`.\n\n---\n\n## `linalg.pca@1`\n\nPrincipal Component Analysis (SVD-based, native). Given `vectors` as n samples × d features, centers each feature, then decomposes to obtain the top-`k` principal components. Returns `components` (principal axes), `explained_variance` and `explained_variance_ratio` per component, the centering `mean`, `singular_values`, and `scores` (samples projected into PC space). Explained-variance ratios are computed against the **total** variance, so they stay correct even when `k` truncates. Needs ≥ 2 samples. Max 4,000,000 cells.\n\n### Parameters\n\n| Parameter | Type | Required | Default | Description |\n|-----------|------|----------|---------|-------------|\n| `vectors` | number[][] | yes | -- | Samples as rows (n × d); `matrix` is an alias |\n| `k` | integer | no | `min(n, d)` | Number of components to keep |\n\n### Example\n\n```json\n{\n  \"name\": \"data-grout@1/linalg.pca@1\",\n  \"arguments\": { \"vectors\": \"$embeddings.vectors\", \"k\": 2 }\n}\n```\n\nResponse: `{ \"n_samples\": 500, \"n_features\": 768, \"k\": 2, \"components\": [[...]], \"explained_variance_ratio\": [0.61, 0.18], \"mean\": [...], \"scores\": [[...]] }`.\n\n---\n\n---\n\n## `linalg.normalize@1`\n\nScales vectors to unit norm. `ord` selects the norm: `l2` (default, Euclidean), `l1` (sum of absolute values), or `max` (L∞). Pass a single `vector` or a `vectors` matrix (each row normalized independently). This is **vector** normalization — distinct from `math.normalize`, which rescales the *distribution* of a series (z-score / min-max / percentile). A zero vector is returned unchanged (norm 0), never divided by zero. Max 20,000 vectors of 4,096 dims.\n\n### Parameters\n\n| Parameter | Type | Required | Default | Description |\n|-----------|------|----------|---------|-------------|\n| `vector` | number[] | one of | -- | A single vector to normalize |\n| `vectors` | number[][] | one of | -- | Vectors to normalize, each row independently |\n| `ord` | string | no | `l2` | `l2` \\| `l1` \\| `max` |\n\n### Example\n\n```json\n{\n  \"name\": \"data-grout@1/linalg.normalize@1\",\n  \"arguments\": { \"vector\": [3, 4] }\n}\n```\n\nResponse: `{ \"vector\": [0.6, 0.8], \"norm\": 5.0, \"ord\": \"l2\" }`.\n\n---\n\n## `linalg.matmul@1`\n\nMatrix product A · B (native, nalgebra). For ergonomics it also does matrix–vector and dot products by coercing 1-D inputs: a flat `a` is treated as a **row** vector (1 × n), a flat `b` as a **column** vector (n × 1). The 2-D `product` is always returned; a single-row/column result is also flattened into `vector`, and a 1 × 1 result into `scalar`. Errors on inner-dimension mismatch. Max 4,000,000 cells per operand.\n\n### Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `a` | number[][] or number[] | yes | Left operand (matrix, or flat = row vector) |\n| `b` | number[][] or number[] | yes | Right operand (matrix, or flat = column vector) |\n\n### Example\n\n```json\n{\n  \"name\": \"data-grout@1/linalg.matmul@1\",\n  \"arguments\": { \"a\": [[1, 0], [0, 2]], \"b\": [3, 4] }\n}\n```\n\nResponse: `{ \"product\": [[3], [8]], \"vector\": [3, 8], \"rows\": 2, \"cols\": 1, \"inner\": 2 }`.\n\n---\n\n## Composition\n\nPull embeddings or numeric columns through [Data](data-tools) / [Frame](frame-tools) tools, then rank, reduce, or cluster them here. `pca.scores`, `cluster` assignments, and the pairwise similarity matrix are all chart-ready — pass them to [Prism Chart](prism-tools) for visualization. Pair with [Math](math-tools) for descriptive statistics on the projected components, or with [Signal](signal-tools) when spectral features feed clustering or similarity.\n"
}