Core Concepts

The Semantic Cube

PrismDB borrows the multidimensional structure of OLAP cubes but operates in a fundamentally different domain. Where OLAP dimensions hold categorical or numeric values for aggregation, PrismDB dimensions hold semantic values: short textual facts that require similarity-based retrieval.

The analogy holds structurally (dimensions are independent axes, queries slice across them, results emerge from intersections), but the content and operations are entirely different. PrismDB is not an analytics engine. It is a retrieval engine for structured records whose field values are semantic, not numeric.

Core Abstractions

Eight abstractions, one containment hierarchy: a collection holds the schema (dimensions, each with lenses, plus the artifact-field declaration) and the segments; segments hold records; a record holds its facts across dimensions plus its artifact pointers. The prism stands outside the data model: it is the query-time plan that reads the schema.

Collection

A named container that defines:

• The set of dimensions and their types (exact or semantic)
• Artifact fields (what pointer data each record carries)
• Configuration: temporal mode, graduation window, retention policy, mapping mode
• Whether dynamic mapping is enabled (auto-detect dimension types from incoming data)

A collection is analogous to a table in a relational database, but with semantic awareness built into its schema definition.

Dimension

An independent searchable axis within a collection. Every record has values across one or more dimensions.

Dimensions come in two types:

• Exact: Enumerated values with known cardinality. Matched by equality. Examples: gender (Male, Female), vehicle class (Car, Truck, SUV), category (footwear, outerwear).
• Semantic: Free-text values that require similarity-based matching. A query for "blue leather shoes" must find a record whose value is "midnight navy genuine Italian leather minimalist low-top sneaker".

When temporal mode is enabled on a collection, semantic dimensions become semantic streams: ordered sequences of timestamped facts that capture how the dimension’s value evolves over time.

Lens

A retrieval strategy active on a dimension. Each lens represents a different approach to finding matches:

Multiple lenses are active simultaneously on the same dimension. On semantic dimensions, the BM25 and vector legs always run in parallel and fuse via RRF; both indices exist from the moment of insert. Exact dimensions match by equality over the value and its learned alias tails.

Segment

A logical data grouping tracked in a SQLite registry. Segments organize the graduation lifecycle.

• Open segment: The currently active segment, receiving new records. Records are searchable immediately via all lenses (BM25 sparse, dense vector, exact) in the unified qdrant-edge engine.
• Sealed segment: A closed segment that has graduated. Graduation is a logical event: seal the segment → learn vocabulary (NLI judges only novel pairs) → reindex → compile entity documents (for temporal collections) → open the next segment.

Graduation is triggered via POST /collections/{name}/graduate. It is incremental: re-graduation of previously learned vocabulary judges zero pairs (all cached). The cost is proportional to novel vocabulary only.

Record

One entry in a collection, living inside a segment. A record is the union of its facts (the value it holds for each dimension; a stream per dimension in temporal mode) and its artifact fields (the pointer values a query ultimately returns).

Records are searchable immediately on insert, through every lens. Facts are the search path; the artifact is the destination.

Fact

The value a record holds for a dimension. On semantic dimensions, a fact is a short free-text phrase (material = "genuine Italian leather"); on exact dimensions, it is the enumerated value itself (category = "footwear"). Facts are the atomic units of content in PrismDB.

Facts are intentionally short: a phrase or sentence, not a paragraph. This is a key distinction from document-oriented databases: PrismDB stores structured dimensions of independent facts, not chunks of prose.

In temporal collections, facts are timestamped entries in a dimension’s stream:

dimension: activity
  t=0s    "entering through main door"
  t=120s  "talking to receptionist"
  t=300s  "walking toward elevators"

Artifact

What you actually retrieve. An artifact is a set of pointer fields (IDs, URLs, references) that identify an external asset. PrismDB stores and indexes the metadata (dimensions); the heavy assets (images, videos, documents, sensor data) live in external storage systems.

A query returns artifacts, not raw dimension values. The dimensions are the search path; the artifact is the destination.

Prism

The one abstraction outside the containment hierarchy: the query plan. It owns no data; it reads the schema. When a query arrives, PrismDB decomposes it into dimensions and constructs a prism, a plan that specifies:

• Which dimensions are involved and which terms route to each (IDF + embedding routing, DF guard)
• Which extra channels are active (whole-query dense for noisy terms, semantic-residual boost for dropped phrases)
• Temporal constraints (point-in-time, range, trajectory)

The prism is the core intelligence of PrismDB: it encodes where each query term carries signal, so every dimension searches only what belongs to it.

Mapping

PrismDB supports both explicit and dynamic mapping, similar to Elasticsearch:

Explicit mapping: The user defines dimensions upfront with their types and lens configuration:

dimensions:
  category: { type: exact, values: ["footwear", "outerwear"] }
  material: { type: semantic, lens: ["exact", "bm25", "vector"] }

Dynamic mapping: PrismDB infers dimension types from incoming data:

• Few distinct values → exact
• Free-text → semantic (BM25 sparse + dense indexed on insert)
• Unknown fields with dynamic: "strict" → rejected

Both modes coexist: explicit for known dimensions, dynamic for emergent ones.

Semantic Time Series

When temporal mode is enabled, PrismDB gains an additional axis: time.

Traditional time-series databases store numeric values that change over time (temperature, CPU usage, stock price). PrismDB’s temporal mode stores semantic values that change over time: textual facts describing an evolving entity.

Entity X, dimension "activity":

  t=0s    "entering through main door"
  t=120s  "talking to receptionist"
  t=300s  "walking toward elevators"

Entity X, dimension "carrying":

  t=0s    "laptop bag and coffee cup"
  t=120s  "laptop bag"              ← coffee was set down

Each dimension is a stream. A “record” at any point in time is a snapshot across all dimension streams. This enables query patterns that no existing database handles natively:

• Point-in-time: “who was at the reception at 9:02?” (time_at)
• Range: “everyone in the lobby between 9:00 and 9:05” (time_range)
• Trajectory: “find entities whose activity went from stationary to agitated” (sequence)

Evolution summaries (“how did this entity’s behavior change?”) and anomaly detection (“which entities had abrupt state changes?”) are natural extensions of the stream model; on the roadmap, not in the current API.

The temporal axis is optional. A product catalog collection has no temporal dimension (facts are static). An activity monitoring collection has temporal dimensions (facts evolve). The same engine serves both, configured per collection.