The embedded
search engine for
structured records
Records with textual fields (color, material, behavior, condition) deserve better than substring matching or one monolithic vector. PrismDB gives each field its own retrieval strategy and fuses the results, like light through a prism.
Your records say "midnight navy"; your users search "blue". PrismDB learns that at write time: no synonym lists, no per-query LLM. Queries are pure retrieval, ~5ms, deterministic. One Rust binary.
Structured records with textual fields are everywhere. No search engine handles them well.
Product catalogs, analytics records, IoT anomaly logs, legal documents: each record is a set of independent dimensions, each holding a brief factual description. Existing engines force a bad choice on all of them:
Monolithic embeddings
Vector databases encode the entire record as one point. Dimensions cancel each other: a query for "green jacket" competes with "pacing nervously" in the same vector space.
Lexical ceiling
Full-text search handles stemming but cannot bridge semantic gaps. "distressed" never finds "sitting on floor holding head".
Fuzzy where it must be exact
An enum that should only ever match exactly and a free-text behavior description that must match loosely get the same treatment. No database lets a field choose its own retrieval strategy, so one of the two is always wrong.
Query-time expansion tax
Bridging vocabulary at query time (LLM rewriting, on-the-fly synonyms) makes every search slow, costly, and non-deterministic. Curated synonym lists rot. Nobody closes the gap at write time.
Your records say "midnight navy". Your users search "blue".
PrismDB closes the vocabulary gap at write time through a cost ladder of seven expansion tiers. Each gap is closed by the cheapest tier that closes it: modifiers, NLI sense resolution, ConceptNet parents, synonym edges, lexicon projection, enum aliases, and enum projection cover vocabulary the corpus never contains.
Every tier proposes; none of them decides. A candidate enters the index only if a DeBERTa-v3 MNLI cross-encoder judges that the record's own value entails it, above a fixed 0.70 cutoff. The verdict is cached per unique pair, including the refusals, so a second graduation judges zero pairs it has already seen.
No curated synonym lists. No domain configuration. All inference at write time, cached per unique value. Queries are pure retrieval: the query path runs no generative inference at all. The cost ladder is a comparison rule, not a prohibition: every tier is considered, the cheapest one that works is used.
Real expansions learned on the public regression datasets. None of these pairs is hardcoded anywhere in the engine.
A tier proposed each of these; the judge declined. The refusals are cached with the same key as the acceptances, so the ladder never pays to re-litigate them.
The intelligence lives in the storage layer
Per-dimension retrieval
Each field gets its own lens (exact, BM25 sparse, or dense vector) in its own semantic space. No monolithic embeddings, no dimension dilution.
Cross-dimension coherence
Results must hold across the dimensions your query actually touched: a precision mechanism monolithic vector search cannot express.
Cost-ladder vocabulary expansion
Seven tiers: modifiers, NLI sense resolution, ConceptNet parents, synonym edges, lexicon projection, enum aliases, enum projection. Each gap closed by the cheapest tier that closes it, and every candidate gated by an NLI cross-encoder before it enters the index. All at write time, cached per pair.
Query decomposition + rewrites
Multi-term queries are split and routed per dimension by IDF + embedding routing. Spell correction, compound splitting, and term merging handle messy input.
Built-in embeddings
BGE-M3 dense (1024d) + NLI judge (DeBERTa-v3) for write-time expansion. Backends: CPU ONNX, NVIDIA GPU, or Apple-GPU MLX sidecar, auto-detected and spawned by the binary. No external services.
Unified in-process engine
One embedded engine (qdrant-edge) holds per-dimension BM25 sparse and dense 1024d vectors with native RRF fusion, one collection per dimension rather than one per corpus. Everything above it is PrismDB: query decomposition and routing, the write-time expansion ladder, the cross-dimension gate, and the temporal axis. Deployment is a single binary.
Temporal axis
Point-in-time, range, and trajectory queries over timestamped streams. Track who was where, when, and in what sequence. Built into the query engine.
Graduation lifecycle
Seal a segment, learn vocabulary from the new data, reindex, and open the next segment. Incremental cost: re-graduation of known vocabulary judges zero pairs.
Single binary, zero external dependencies
A Rust workspace of focused crates. The unified storage engine (qdrant-edge), embedding
and NLI models (CPU ONNX, NVIDIA GPU, or Apple-GPU MLX, auto-detected), and SQLite
ship in one binary. Deployment is ./prismdb. Like SQLite, it embeds where your data lives.
Built for records described in words
Wherever a pipeline (often an LLM) produces structured records with free-text fields, the vocabulary it writes never matches the vocabulary people search. That gap is PrismDB's home turf: attribute-decomposed queries over structured records.
Video Analytics Pipelines
Product Catalogs
Legal Discovery
IoT / Industrial
Same color, same dimension: each query term routes to its field and matches vocabulary learned at write time.
One binary. One HTTP API.
No cluster, no external services. Embeddings run in-process (CPU) or on the local GPU via an auto-spawned sidecar. Declare your dimensions, insert records as plain text, search. Client SDKs (Python, Rust, Go) are on the roadmap; the HTTP API is the stable surface today.
$ prismdb serve --port 6340 --data-dir ./data
$ curl -X POST localhost:6340/api/v1/collections \
-d '{
"name": "products",
"dimensions": [
{"name": "category", "type": "exact"},
{"name": "material", "type": "semantic"},
{"name": "color", "type": "semantic"},
{"name": "style", "type": "semantic"}
]
}'$ curl -X POST localhost:6340/api/v1/collections/products/search \
-d '{"query": "blue leather shoes", "limit": 10}'
{
"results": [{
"id": "SKU-001",
"score": 0.94,
"facts": {
"color": "midnight navy", ← matched "blue"
"material": "Italian leather",
"style": "low-top sneaker" ← matched "shoes"
},
"matched_dimensions": ["color", "material", "style"]
}]
}Not released. Measured, and still moving.
PrismDB is in active development and is not publicly available. The engine runs, the six regression suites are re-measured on every routing change, and the boundaries it has not crossed yet are written down next to the ones it has.
If you work on retrieval over structured records, the interesting conversation is not the feature list: it is where the cross-dimension gate trades recall for precision, what the NLI cutoff refuses, and which of these tiers survives contact with a corpus shaped like yours.