Linalg Tools

Why this matters

Embeddings are everywhere — semantic search, RAG, clustering, dedup — but comparing and reducing them means real linear algebra, not something an LLM can eyeball. Linalg tools give an agent exact cosine similarity for nearest-neighbour ranking, honest distance metrics, and SVD-backed PCA for dimensionality reduction. The math is identical across runs, and the expensive decompositions (SVD, PCA) are computed in a native Rust NIF rather than the model's head.

Similarity & distance

• ✓linalg.cosine_sim — Cosine similarity: a single pair, a matrix ranked against a query vector (nearest-neighbour with top_k), or a full pairwise matrix
• ✓linalg.distance — Euclidean, Manhattan, Chebyshev, and cosine distance in the same three modes
• ✓linalg.project — Project vectors onto a basis; returns coordinates, the reconstruction, and the residual with its norm
• ✓linalg.normalize — Scale vectors to unit norm (l2/l1/max) — e.g. normalize embeddings before storage or dot products

Structure & reduction

• ✓linalg.matmul — Matrix product A·B (native), with matrix–vector and dot products via 1-D coercion
• ✓linalg.cluster — K-means (Lloyd's algorithm) with deterministic seeding: same input and k always yield the same clusters, centroids, sizes, and inertia
• ✓linalg.svd — Thin Singular Value Decomposition A = U·Σ·Vᵀ (native, nalgebra) with each singular value's explained-variance ratio; optional top-k truncation
• ✓linalg.pca — Principal Component Analysis over centered data: components, explained variance and ratio, the centering mean, and samples projected into PC space

Use cases

• ◆Semantic search & RAG — Rank a corpus of embeddings against a query with cosine_sim + top_k, entirely server-side
• ◆Dimensionality reduction — pca to 2–3 components for plotting or to compress features before downstream modeling
• ◆Clustering & segmentation — Group embeddings or feature vectors with reproducible cluster assignments
• ◆Latent structure — svd for low-rank approximation, rank estimation, and matrix factorization

Example: search, reduce, cluster

// Rank documents against a query embedding (semantic nearest-neighbour)
{ "name": "data-grout@1/linalg.cosine_sim@1",
  "arguments": { "matrix": "$embeddings.vectors", "query": [0.12, -0.04, 0.88], "top_k": 5 }
}

// Reduce high-dimensional embeddings to their top 2 principal components
{ "name": "data-grout@1/linalg.pca@1",
  "arguments": { "vectors": "$embeddings.vectors", "k": 2 }
}

// Cluster the reduced points into 4 groups (deterministic k-means)
{ "name": "data-grout@1/linalg.cluster@1",
  "arguments": { "vectors": "$pca.scores", "k": 4 }
}

scores and similarity records are chart-ready — pass them to prism.chart to plot the reduced space.