Cost ladder

Cheapest rung first. Expensive rungs genuinely considered, never reflexively avoided.

Why this matters (economics)

The naive design fires an LLM agent for every capability on every inbound reply. At volume:

10,000 replies/day × 6 agents × ~2k tokens each ≈ 120M tokens/day

That’s a token furnace. Unit economics only close if the cheap path handles the majority of traffic and the LLM handles the residue.

The ladder is also a latency doctrine: classical rungs respond in milliseconds and free budget for the hard cases.

The meta-doctrine

• “Cheaper-first” is a comparison rule, not a prohibition. Expensive rungs are genuinely considered when a gap is measured.
• “LLM-free for capability X” is an outcome (cheaper tiers closed the gap), never a constraint.
• If cheaper rungs are refuted with evidence, take the expensive rung without hesitation.
• Every rung decision is backed by an eval-harness measurement, not intuition.

The ladder, per capability

Each agent capability gets its own ladder. A reply climbs only as far as it needs.

Intent classification

The core cascade — seven rungs from header checks to full LLM reasoning:

Confidence gate: each rung emits a calibrated probability. Below the threshold, the reply climbs. The threshold is learned from feedback data.

Tone and style profiling

Style mirroring is feature extraction, not reasoning. This capability ships LLM-free.

Scheduling

80%+ of scheduling language resolves at rungs 1–3.

Delegation and referral extraction

Detection (is this a delegation?) is rungs 1–3. Interpretation (how to address the new contact) is generation territory.

No-show and ghosting detection

Rung 2 is a genuine classical win: re-engagement timing learned from reply-latency distributions beats fixed “wait 48 hours” policies.

Objection classification and response

Same ladder shape as intent classification, over a finer taxonomy (15 types, 30 approved angles). Response selection from the playbook:

The Thompson-sampling bandit (rung 2) directly optimizes the headline conversion metric with zero per-decision LLM cost.

Response generation

The one capability where the LLM is genuinely load-bearing. Still laddered:

Confidence routing

Every approval and rejection is a labeled example of “should this have escalated?” The system generates its own training data as a side effect of normal operation.

The triage cascade

The ladder inverts the naive design:

Reply arrives
    │
    ▼
RUNG 0 — Header check (~0 cost, ~ms)
    bounces, OOO, unsubscribe, calendar responses
    → resolved without any model. ~15–25% of traffic.
    │
    ▼
RUNGS 2–3 — Classical triage (~0 cost, ~ms)
    intent classifier + sentiment + style + datetime + referrals
    → ALL capabilities run, always, cheap
    │
    ▼ confidence gate (calibrated, learned)
    │
    ├── HIGH confidence + template action
    │       → template response, done. No LLM touched.
    │
    ├── HIGH confidence + open reply
    │       → ONE LLM call: generation only, with classical
    │         outputs as structured context
    │
    └── LOW confidence / conflict / high stakes
            → full cascade (rungs 4–7), possibly
              escalate to human

Classical rungs are the default execution path. The LLM is the escalation.

Current eval results

Held-out test set: 75 items, touched once per cascade stage.

The one remaining “miss” at rung 6 — “This really isn’t something I’m involved in” classified as not_interested instead of wrong_person — produces the same withdrawal action. No wrong email was sent.

The refuted ledger

Every lever that failed its measurement, kept so nobody re-tries it by intuition:

Rung 4 — similar examples (kNN)

Nearest-neighbor matching over the labeled corpus. A reply below the rung-3 gate is answered by unanimous agreement among its 5 nearest exemplars by cosine similarity.

Two embedding approaches were measured: a lightweight CPU model (~110 MB) and PrismDB’s sidecar with a larger multilingual model. Both perform within one item of each other on held-out; the lightweight model is the default because it needs no GPU.

Gate tuning: leave-one-out on the cross-validated training residue (233 items the classifier escalates), selecting k=5 unanimous as the widest point at ≥98% precision. The test split was touched once.

Rung 4 is the first lever that widens coverage at the 100% bar. Its labels are copies of corpus exemplars and never enter the learning loop without explicit human correction.

Rung 5 — trained classifier

A 15-way classification head on the same backbone as the rung-6 judge, fine-tuned on the labeled corpus (~100 seconds on a GPU). It speaks the same agreement protocol as rung 6: accept only when the head and the statistical classifier agree, with probability above a floor (0.30).

The fine-tuned NLI head — the rung-5 candidate the rung-6 controls suggested — was measured at 52/55 and refuted at the bar. The classification head is the shipped runtime.

Rung 6 — meaning match via PrismDB

The PrismDB sidecar runs an NLI model as a zero-shot judge: each reply is scored against 15 intent hypotheses, and the system accepts only when the judge’s top intent matches the classifier’s and the confidence clears a floor (0.80).

Controls showed the PrismDB judge is numerically identical to the reference runtime of the same model (zero decision flips across 4,620 pairs) and the fastest judge tested (21 ms). Three different models all miss the same held-out sentence — 97% is a method ceiling, not a model ceiling.

The agreement rule is the key design: neither the statistical classifier nor the judge is safe alone, but when they concur they are nearly always right.

Anti-doctrine guards

• Never cite this document to refuse an LLM rung when a gap is measured. The ladder ranks options; evals pick them.
• Never ship a cheap rung below its accuracy floor to avoid an LLM call — in enterprise sales, one wrong email is a fire.
• Generation (rung 7) is equally acceptable when the classical rung cannot reach the floor — it is the standing example.