Investigation — the archive answers "what is connected to this?"

Status: design, awaiting review · Date: 2026-08-18
Companion research: docs/research/2026-08-15-neo4j-graph-research.md (why this is Postgres
tables and not a graph engine) · docs/superpowers/specs/2026-08-15-contract-intelligence-design.md
(the entity layer this builds on).

The doctrine

Obscura already records who did what to which document. It just cannot be asked.

Every relationship an investigator wants is already in the database, spread across tables that
only answer one direction. signatures knows who signed what. letter_dispositions knows who
sent what to which position. contract_parties knows which company is party to which agreement.
document_links knows which document supersedes which. Nothing can walk between them.

So the archive can answer "show me this document" and cannot answer "show me everything
connected to this vendor" — which is the question an Inspektorat, an internal auditor, or a
lawyer preparing a dispute actually arrives with.

Why this is the module to build (and AGE is not)

The graph research recommended plain Postgres tables first, a graph engine only if the walks got
deep. Nine months later the walks have not got deep, and the reason is now measurable: there is
exactly one edge type in the product. The whole "graph" is this, in
internal/contracts/adapters/entity_pg.go:

SELECT DISTINCT cp.document_id
  FROM contract_parties cp
  JOIN documents d ON d.id = cp.document_id AND d.deleted_at IS NULL
 WHERE cp.entity_id = ANY($1)

An entity is party to a contract. That is the entire relationship model. Installing Apache AGE
on top of it would be a graph engine querying a graph with one kind of edge in it.

The missing thing is edges, and edges are a table. Once they exist, AGE remains available as
a drop-in later — same Postgres, same pg_dump, same schema-per-tenant isolation — without
changing a line of the UI this spec describes.

🔴 What already exists and is unused

GET /api/v1/entities/{id} is implemented, module-gated on contracts, and ACL-correct —
it resolves an entity's contracts, then filters them through s.dms.SearchAdvanced with the
caller's own subjects and fails closed if the checker is absent
(internal/httpapi/handlers_entities.go:66-83). It also resolves the secure-folder step-up once
per request rather than once per row.

Nothing in the web app calls it. The first slice below is a page over an endpoint that is
already written and already safe. That is the demo screen, and it costs no backend work.

What ships

1. Entity page — "PT Virtue Digital Indonesia: 12 documents, 4 signers, 3 related parties",
   every row ACL-filtered, each linking to the document.
2. document_edges — one derived table projecting the existing relational tables into a
   uniform (src, rel, dst) shape.
3. Connections tab on a document — who signed it, who it was dispositioned to, what it
   supersedes, which entities are party to it, what else those entities touch.
4. Investigation search — "everything connected to X within N hops", N bounded, results
   ACL-filtered at every hop.
5. Module + permission + audit — investigation is a licensed capability and a logged act.

Does not ship: free-form Cypher, a force-directed graph canvas (a picture of 400 nodes
answers nothing — the deliverable is a list an auditor can cite), risk scoring, or any
automated inference beyond what a foreign key already asserts.

🔴 Hard rule 1: a relationship view leaks EXISTENCE, not just content

This is the whole security surface of the module, and it is a different failure from the ones
the ACL audit closed. A document you cannot read can still be revealed by a count:

"PT Virtue Digital Indonesia — 12 documents" shown to someone who may read 2 tells them
ten documents exist, roughly when, and with whom. That is the leak. No document was opened.

So, unlike an admin registry count (Findings: 14, which is deliberately the true total because
its reader is records.admin and the number decides delete-versus-disable), every number this
module renders counts only what the caller may read. A hidden document contributes nothing —
not to a count, not to a date range, not to a "3 related parties" tally derived from documents
behind the ACL.

Concretely:
- Resolve the candidate id set from the edge table (unfiltered — the edge table has no ACL).
- Hand it to dms.SearchAdvanced with Scope.IncludeDocIDs and the caller's subjects, exactly
as handlers_entities.go already does.
- Derive every displayed figure from the filtered set, never the candidate set.
- 🔴 An empty filtered set renders "nothing you can see", never "nothing exists" — those are
different sentences and only one of them is true.

The same rule at each hop: hop 2 expands only from documents that survived hop 1's filter.
Expanding from hidden nodes and filtering at the end leaks the middle of the path.

🔴 Hard rule 2: edges are DERIVED and rebuildable, never authored

document_edges is a projection, not a source of truth. Every row records which table it
came from and that row's id. Nothing writes an edge by hand; nothing reads an edge to make an
authorization decision.

This is what makes the sync problem tractable. The graph research noted that keeping a separate
graph store in step is where these projects die, and that AGE removes the consistency problem
by living in the same transaction. It does not remove the work — every edge still has to be
written either way. Deriving them means the failure mode is "the projection is stale", and the
answer to a stale projection is obscura-server rebuild-edges, not a data-loss incident.

Projection runs in two places: incrementally on the write that creates the underlying row (same
transaction), and wholesale from the rebuild command (idempotent, safe to run on a live system).

🔴 Hard rule 3: a wrong entity merge is close to defamatory

Entity resolution is alias matching, and it is fuzzy. In a contract register a bad merge is an
annoyance. In an investigation view, "these two vendors are the same company" is an allegation,
and the screen it appears on is one someone may act on.

Therefore edges carry confidence, and the UI distinguishes two things that look identical
otherwise:

A reader must always be able to see why the system believes two things are connected. Merges
stay reviewable, the same suggests-until-confirmed doctrine the contract profile uses.

Data model (migration 00201 — ls go/migrations | tail says 00200 is taken)

-- One row per relationship, projected from the tables that already hold it.
CREATE TABLE document_edges (
    id         uuid PRIMARY KEY,
    -- Endpoints are (kind, id) rather than typed FKs: a src may be a document, a letter, a
    -- user, a position or an entity, and five nullable FK columns would be worse in every way.
    -- The projection is the only writer, so referential integrity is its job, not the schema's.
    src_kind   text NOT NULL,   -- document | letter | entity | user | position
    src_id     text NOT NULL,
    rel        text NOT NULL,   -- party_to | signed | dispositioned_to | links_to | supersedes | owns
    dst_kind   text NOT NULL,
    dst_id     text NOT NULL,

    -- 🔴 Provenance. A wrong edge must be traceable to the row that produced it, and the
    -- rebuild must be able to delete exactly what it is about to re-derive.
    source     text NOT NULL,   -- signatures | letter_dispositions | contract_parties | document_links | documents
    source_id  text NOT NULL,

    occurred_at timestamptz,    -- when the underlying act happened; NULL when the source has no time
    -- 1.0 = a foreign key asserts this. < 1.0 = entity resolution matched a name (hard rule 3).
    confidence real NOT NULL DEFAULT 1.0,

    UNIQUE (source, source_id, rel, src_kind, src_id, dst_kind, dst_id)
);

-- Both directions are first-class: "what did this person sign" and "who signed this" are the
-- same table read two ways, and an investigator asks both.
CREATE INDEX document_edges_src ON document_edges (src_kind, src_id, rel);
CREATE INDEX document_edges_dst ON document_edges (dst_kind, dst_id, rel);
CREATE INDEX document_edges_source ON document_edges (source, source_id);

What projects into it, and from where

⚠️ documents.owner_id is TEXT, not uuid (the account-deletion work found this the hard way),
so the projection compares as text and does not assume a valid user row exists behind it.

Permission and licensing

Reuse the persona, not the permission. protection.investigate already exists and already
means "this person may attribute a leak" — a genuinely powerful role, gated on the watermarking
module. That precedent settles the shape: investigation is a permission + module pair, and the
person who holds it is trusted with cross-cutting visibility.

• New permission investigation.view, seeded, described, and added to the catalogue — the
  perm-guard asserts the router and catalogue agree in both directions (58 enforced / 59 seeded
  today), so a permission enforced but ungrantable fails the build.
• New module investigation in config.KnownModules. Alias-safe: no existing licence names it.
• 🔴 The module gate does not replace the ACL. It decides whether the screen exists; the ACL
  decides what is on it. A licensed investigator still sees only documents they may read.

Audit

Every investigation query is itself an audited act — subject, query, hop count, and the number
of results the caller was permitted to see. Two reasons, and the second matters more:

1. B2G buyers ask who looked at what. This is the answer.
2. The audit trail is how a misuse of the role becomes visible. A capability that shows one
   person the shape of the whole archive should leave a record when it is used.

Slices (each independently shippable)

1. Entity page over the endpoint that already exists. Route, page, i18n, nav entry. No
   backend, no migration. This is the demo screen and it can ship first.
2. Migration 00201 + the projection + rebuild-edges. No UI. Verified by comparing edge
   counts to the source tables.
3. Connections tab on document detail, reading document_edges through the ACL filter.
4. Investigation search — n-hop, bounded, filtered at every hop.
5. Module + investigation.view + audit + licence, and the nav/route gating that goes with
   them.

Slices 1 and 3 are what a buyer sees. Slice 2 is what makes them possible. Slice 4 is what
distinguishes the module from a nicer document page.

Failure honesty

• A stale projection shows fewer edges, never wrong ones — the incremental write and the
  rebuild both delete-then-insert by (source, source_id).
• An entity with no readable documents renders "nothing you can see", not "no results".
• Hop limits are stated in the UI. A result set truncated at the hop bound says so, because a
  silent cap on an investigation screen reads as "there is nothing further", which is the one
  thing an investigator must never be told by accident.

Open questions for review

1. Do users and entities unify? A signer is a users row; a counterparty is an entities
   row. If a director signs as an individual on one contract and their company is the
   counterparty on another, are those one node or two? (Recommendation: two, linked by an
   explicit reviewed edge — guessing they are the same person is exactly hard rule 3.)
2. Is investigation its own module, or part of contracts? (Recommendation: its own. The
   buyer is different — an Inspektorat/SPI buys oversight, a legal team buys contract admin —
   and one licence line per buyer story is how the other modules are priced.)
3. Does the Connections tab need investigation.view, or is it core for anyone who can
   already read the document? (Recommendation: core. Every edge shown there is already visible
   to that reader on some other tab; the module earns its price on the cross-document views,
   not on restating one document's own facts.)