Spec — Public conversation features (clarify, English, status tracking, guided flows)

Date: 2026-07-18 · Decided by: Efran
Scope guard: chatbot = data Q&A only. Internal → SQL data; Public → gated
SQL + RAG. NO document processing (explicitly rejected 2026-07-18).

Four features, ordered by dependency and shipping order:

1. Clarify-before-answer (public orchestrator)

When a question is genuinely ambiguous (e.g. "berapa biayanya?" — biaya apa?),
the planner may ask ONE clarifying question instead of guessing or refusing.

Design
- @ahu/orchestrator-types: new variant Clarify { kind: "clarify"; question: string };
PlanOutcome = Plan | Refusal | Clarify; isClarify() guard. (PlanOrRefusal
stays as an alias so existing imports compile.)
- plan.ts: schema gains "clarify": <true|false>, "clarify_question": <string|null>;
prompt rule: clarify only when the question cannot be answered without one
missing fact, never twice in a row.
- Loop guard in code, not just prompt: if the last assistant history entry
was itself a clarify question (tracked via a [?]-free heuristic: orchestrate
passes allowClarify = !history.slice(-1)[0]?.wasClarify — implemented by
checking the previous assistant turn against the exact question we streamed;
simplest deterministic rule: clarify allowed only when the LAST assistant
message did not end with "?"). When not allowed, planner instruction omits the
clarify option and code coerces clarify:true responses into a normal plan or
refusal.
- orchestrate.ts: on Clarify → emit the question as a normal content event +
done. No UI change needed — the reply arrives in history next turn.
- Audit: clarifications are NOT refusals; no error_code.

2. English mode (public orchestrator)

Mirror the user's language (ID default, EN supported). No UI toggle — detection.

Design
- New lib/orchestrator/lang.ts: deterministic detectLang(text): "id" | "en"
(wordlist/stopword scoring; ties → "id"). Unit-tested; no LLM call.
- compose.ts: system prompt parameterised — answer in the user's language;
tool prompts/RAG queries STAY Indonesian (corpus is Indonesian).
- plan.ts: prompt note that user questions may be English; tool prompts must
remain Indonesian; clarify questions in the user's language.
- Bilingual fixed strings in orchestrate.ts (statuses "Mencari dokumen…",
queue "Dalam antrean — posisi N", busy "Sistem sedang sibuk") via a tiny
copy(lang) table in lang.ts.
- Out of scope: admin-configured L1 refusal messages (stay as configured),
UI chrome, staff surface.

3. Application status tracking (public, gated)

"Cek status permohonan/pesan nama/transaksi saya" → identity-verified,
single-row lookup. Generalizes the existing verify_pesan_nama business-rule
pattern (knowledge rule + pre_aggregated table allowlist + k-anon validator
exception already supports it).

Design
- New business-rule knowledge entries (seeded via
/api/admin/knowledge/seed-templates route additions) per flow, PP-priority
order: PP registration status → Apostille application status → PT (pesan
nama exists; add pendirian transaction status).
- Each rule: trigger phrases; required identifiers (application/transaction
number); PIC verification step (nama + email pemohon as WHERE filters, never
displayed) BEFORE revealing status; allowed output columns (status, tanggal,
tahap) — no PII columns.
- Policy: add the status-bearing tables to pre_aggregated_tables +
allowed_tables defaults (shared JSON) after checking real column names
against docs/knowledge-sources + SABH/PTP/apostille codebases.
- Eval: extend ahu_test_cases.py with per-flow cases (rubric: must verify
before reveal).
- Orchestrator plan prompt: status questions are NOT identity refusals — route
to public_data (rule handles gating).

4. Guided procedural flows / wizard (public)

Step-by-step guidance for PP/Apostille/PT procedures with next-step chips —
not free-form prose dumps.

Design (v1, content + chips — no new backend state)
- Flow content: dedicated RAG "flow" articles (one per procedure, numbered
steps, one screen per step) — PP first per priority.
- compose.ts prompt: when the answer is procedural, present ONE step at a
time ("Langkah N dari M"), ending with what to do next.
- Suggestions: extend the existing SuggestedFollowups machinery — orchestrate
emits a suggestions SSE event (new frame on the existing channel) when
compose's answer contains a flow-step marker; UI chips become "Lanjut ke
langkah N+1" / "Ulangi" / "Selesai". Static category chips remain the
fallback.
- v2 (deferred): server-side flow state per thread; branching wizards.

Constraints

• Suites green (pnpm check, agent pytest); spec→plan→completion notes per
  repo mandate; mirrors byte-identical where public/internal agents share
  files; no engine-repo (ai-ahu-rag) changes — flow articles go through the
  existing knowledge/ingest path.