The embedded
search engine for
structured records
Records with textual fields (color, material, behavior, condition) deserve better than substring matching or one monolithic vector. PrismDB gives each field its own retrieval strategy and fuses the results, like light through a prism.
Your records say "midnight navy"; your users search "blue". PrismDB learns that at write time: no synonym lists, no per-query LLM. Queries are pure retrieval, ~5ms, deterministic. One Rust binary.
Structured records with textual fields are everywhere. No search engine handles them well.
Product catalogs, analytics records, IoT anomaly logs, legal documents: each record is a set of independent dimensions, each holding a brief factual description. Existing engines force a bad choice on all of them:
Monolithic embeddings
Vector databases encode the entire record as one point. Dimensions cancel each other: a query for "green jacket" competes with "pacing nervously" in the same vector space.
Lexical ceiling
Full-text search handles stemming but cannot bridge semantic gaps. "distressed" never finds "sitting on floor holding head".
One strategy fits all
No database adapts its retrieval strategy per field. A color enum and a free-text behavior description receive the same treatment.
Query-time expansion tax
Bridging vocabulary at query time (LLM rewriting, on-the-fly synonyms) makes every search slow, costly, and non-deterministic. Curated synonym lists rot. Nobody closes the gap at write time.
Your records say "midnight navy". Your users search "blue".
PrismDB closes the vocabulary gap at write time through a cost ladder of expansion tiers. Each gap is closed by the cheapest tier that closes it. ConceptNet graphs, NLI sense resolution, synonym expansion, enum alias stripping, and a lexicon-projection tier cover vocabulary the corpus never contains.
No curated synonym lists. No domain configuration. All inference at write time, cached per unique value. Queries are pure retrieval. The cost ladder is a comparison rule, not a prohibition: every tier is considered, the cheapest one that works is used.
Real expansions learned on the public regression datasets. None of these pairs is hardcoded anywhere in the engine.
The intelligence lives in the storage layer
Per-dimension retrieval
Each field gets its own lens (exact, BM25 sparse, or dense vector) in its own semantic space. No monolithic embeddings, no dimension dilution.
Query decomposition + rewrites
Multi-term queries are split and routed per dimension by IDF + embedding routing. Spell correction, compound splitting, and term merging handle messy input.
Cost-ladder vocabulary expansion
Seven tiers: ConceptNet, NLI sense resolution, enum aliases, synonyms, lexicon projection. Each gap closed by the cheapest tier that closes it. All at write time, cached.
Temporal axis
Point-in-time, range, and trajectory queries over timestamped streams. Track who was where, when, and in what sequence. Built into the query engine.
Unified in-process engine
One embedded engine (qdrant-edge) holds per-dimension BM25 sparse and dense 1024d vectors with native RRF fusion. Deployment is a single binary.
Built-in embeddings
BGE-M3 dense (1024d) + NLI judge (DeBERTa-v3) for write-time expansion. Backends: CPU ONNX, NVIDIA GPU, or Apple-GPU MLX sidecar, auto-detected and spawned by the binary. No external services.
Cross-dimension coherence
Results must hold across the dimensions your query actually touched: a precision mechanism monolithic vector search cannot express.
Graduation lifecycle
Seal a segment, learn vocabulary from the new data, reindex, and open the next segment. Incremental cost: re-graduation of known vocabulary judges zero pairs.
Single binary, zero external dependencies
A Rust workspace of focused crates. The unified storage engine (qdrant-edge), embedding
and NLI models (CPU ONNX, NVIDIA GPU, or Apple-GPU MLX, auto-detected), and SQLite
ship in one binary. Deployment is ./prismdb. Like SQLite, it embeds where your data lives.
Built for records described in words
Wherever a pipeline (often an LLM) produces structured records with free-text fields, the vocabulary it writes never matches the vocabulary people search. That gap is PrismDB's home turf: attribute-decomposed queries over structured records.
Video Analytics Pipelines
Product Catalogs
Legal Discovery
IoT / Industrial
Same color, same dimension: each query term routes to its field and matches vocabulary learned at write time.
One binary. One HTTP API.
No cluster, no external services. Embeddings run in-process (CPU) or on the local GPU via an auto-spawned sidecar. Declare your dimensions, insert records as plain text, search. Client SDKs (Python, Rust, Go) are on the roadmap; the HTTP API is the stable surface today.
$ prismdb serve --port 6340 --data-dir ./data
$ curl -X POST localhost:6340/api/v1/collections \
-d '{
"name": "products",
"dimensions": [
{"name": "category", "type": "exact"},
{"name": "material", "type": "semantic"},
{"name": "color", "type": "semantic"},
{"name": "style", "type": "semantic"}
]
}'$ curl -X POST localhost:6340/api/v1/collections/products/search \
-d '{"query": "blue leather shoes", "limit": 10}'
{
"results": [{
"id": "SKU-001",
"score": 0.94,
"facts": {
"color": "midnight navy", ← matched "blue"
"material": "Italian leather",
"style": "low-top sneaker" ← matched "shoes"
},
"matched_dimensions": ["color", "material", "style"]
}]
}