SallyIP

Home / Hallucinations: what the data says and what helps

Hallucinations: what the data says and what helps

Fluent, false legal output — invented cases, misquoted statutes, real citations behind wrong conclusions. The peer-reviewed numbers are sobering; the defences are measurable.

The numbers

What measurably helps

  1. Evidence-first generation (no retrieval, no assertion).
  2. Mechanical quote and citation checks before display.
  3. Entailment grading separate from citation existence.
  4. Refusal as a designed output, with abstention measured not hidden.
  5. Published miss rates, including blocked releases.

Anatomy of a legal hallucination

Three species recur. Invented authorities — cases and patents that do not exist (SallyIP’s adversarial traps include a fake Federal Circuit decision and a zero-number patent; both refused). Altered quotations — real sources quoted with changed words (caught word-for-word in ablation, 2/2 with zero false flags). Unsupported attachment — real citations behind claims they do not support (the P0 run’s 81.0% unsupported-proposition rate, published with the release blocked). Each species needs a different gate, which is why single-score "accuracy" marketing misleads.

What to ask any vendor

Refusal is a feature

SallyIP’s adversarial runs show what healthy refusal looks like: 14 grounded refusals across 23 ABIGAIL cases where offline evidence could not support an answer, and 6 conservative refusals in 12 live traps. Each refusal is logged with the evidence state that caused it — reviewers see what was missing, not just that the system declined. A tool that has never refused has either never been tested or never admits it. Refusal telemetry is preserved the same way quote telemetry is: de-quoted, recorded, and counted in the next run’s denominators.

Hallucination benchmark · ablation study