think
16px
820px
← all files

AHU AI Observatory — BI Dashboard Tutorial

A practical guide to the Observatory Dashboard: the read-only BI surface that turns the platform's live AI audit stream into charts, tables, and an auditable call trail.

  • Live URL: http://192.168.83.20:8320/ (deployed on the GPU host ai-ahu, host port 8320)
  • Local dev: pnpm dev → http://localhost:5173
  • What it reads: the observatory query API on :8300 only. It is 100% read-only — it changes nothing about the gateway, engines, or the audit data. It cannot write, retry, or delete.
  • Stack: React 19 + IBM Carbon Design System + @carbon/charts. Light (g10) / dark (g90) toggle in the header.

Golden rule of this dashboard: it never invents numbers. Anything the API doesn't serve is shown as — or a labeled placeholder, never faked. If a field reads —, the data genuinely isn't there — that's honesty, not a bug.

Throughout this guide, metrics are tagged with which direction is good: ↓ lower is better, ↑ higher is better, • neutral / context-dependent (a volume or count that's neither good nor bad on its own).

1. How it fits together

flowchart LR subgraph Engines["AI Engines"] CB[ahu-ai-chatbot] OCR[ahu-ocr-akta-notaris] CLS[ahu-doc-classifier] end GW["Model Gateway :8200
(all AI egress)"] RS["Redis Stream
ahu.ai.audit"] OBS["Observatory :8300
(Go API + TimescaleDB)"] DCGM["DCGM exporter :9400
(GPU telemetry)"] NGX["nginx (same-origin proxy)
serves SPA + /api/*"] UI["📊 Dashboard :8320
your browser"] Engines -->|audit events v1| GW GW -->|fire-and-forget| RS RS --> OBS DCGM --> OBS OBS --> NGX NGX --> UI UI -->|Bearer token, same-origin| NGX

Every AI call in the platform emits a schema-v1 audit event onto the Redis stream. The observatory ingests those (plus GPU samples from DCGM) into TimescaleDB and exposes them via a role-gated REST API. The dashboard is just a well-designed reader on top of that API.

2. Logging in

There is no username/password. At the login gate you provide two things:

Field What to enter
Bearer token An observatory API token (issued in the observatory's deploy config). Paste it into "Bearer token".
Select a role Pick executive, operator, or auditor — this only controls which nav sections show.

The token lives in sessionStorage — it dies when you close the tab, and it is sent only as an Authorization: Bearer … header on same-origin API calls. The dashboard stores no secret of its own.

The role you pick is a UI convenience, not a security boundary. Real permissions are enforced by the server against your token. Picking auditor with an executive token just gets you a 403 on the gated calls — it never bypasses anything.

3. Roles & sections

Roles are additive — each higher role sees everything below it plus more:

flowchart TD E["executive
Overview · SLA · Analytics · Report"] O["operator
+ Calls"] A["auditor
+ Audit"] E --> O --> A
Role Can open API it may call
executive Overview, SLA, Analytics, Report /api/summary, /api/series, /api/gpu/*
operator + Calls + /api/calls, /api/calls/{id}
auditor + Audit + /api/calls/{id}/body, /api/verify-chain, /api/access-log

Routes: / Overview · /sla · /analytics · /report · /calls · /audit.

4. Shared controls (top of most views)

  • Range picker — Last hour, Last 24 hours, Last 7 days. This drives every tile and chart on the page. Bucket granularity auto-adjusts: 1-minute buckets for ≤24h, 1-hour buckets for 7d (chart titles say "per hour" vs "per minute" so a point is never mislabeled).
  • Time zone — all time-axis ticks and timestamps render in WIB (GMT+7, Asia/Jakarta), the same for every viewer, so two people reading the shared dashboard read the same clock.
  • Theme toggle — light/dark; the app and the charts flip together.

5. How to read the numbers

Before the panels, two things that recur everywhere.

Which direction is "good"?

Metric Good direction Why
Error rate ↓ lower Fewer failed calls.
Latency (avg, max, p50/p95/p99) ↓ lower Faster responses.
Queue wait ↓ lower Less time stuck waiting for a free GPU.
External calls in prod ↓ lower (ideally 0) Prod should stay on-prem.
Cache-hit rate ↑ higher More requests served from cache = cheaper + faster.
Throughput (calls served) ↑ higher More work getting through — but read it with latency, not alone.
Total calls / total tokens • neutral Volume and cost signals, not quality. High isn't good or bad by itself.
GPU utilization • context Higher means you're using the hardware you paid for, but sustained ~100% usually means saturation → rising queue wait. Healthy is "busy but not pinned."

What are p50, p95, p99? (latency percentiles)

A percentile answers "X% of calls finished faster than this number." They describe the distribution of response times, not just one summary point.

  • p50 (the median) — half of all calls were faster than this, half slower. This is the typical experience.
  • p95 — 95% of calls were faster; only the slowest 5% were worse. This is the "bad-but-not-rare" tail — the slowness real users occasionally feel and complain about.
  • p99 — 99% of calls were faster; the worst 1%. Catches rare stalls: a cold model, a queue spike, one giant 40-page document.

Why not just use the average? An average hides the tail. If thousands of calls take ~1s but a handful take 30s, the mean barely moves — yet those 30s calls are a real, painful experience for those users. Percentiles surface exactly that, which is why SLAs are written against p95/p99, not averages. Rule of thumb: read p50 for "typical," and p95/p99 for "worst realistic," and remember lower is always better.

This dashboard shows p50/p95/p99 only where the observatory computed them server-side (per engine, or window-wide). It never fakes them client-side, because a mean of p95s is not a p95 — so where a build doesn't provide them you'll see —, and the latency panels fall back to avg + max. max is just the single slowest call in the window: a useful ceiling, but noisy (one outlier moves it).

6. The sections, panel by panel

6.1 Overview (executive) — the headline BI

Four stat tiles + six charts, all from /api/summary and /api/series.

Stat tiles

Tile Direction Meaning
Total calls • All AI calls in the window.
Overall error rate ↓ error ÷ total. "Success" = status ok or completed (OCR jobs report completed); everything else counts as an error.
Total tokens (in + out) • Sum of tokens_in + tokens_out across engines — a usage/cost signal.
External calls (prod) ↓ Count of calls routed to an external upstream (e.g. cloud ext-dashscope-397b). On this dev host a nonzero value is expected (allow_external_upstreams=true); production refuses external upstreams, so a red tile here is not a live prod violation.
Latency p95 / p99 ↓ The slow-tail response times (see §5). Shown when the build provides percentiles; otherwise —.

Charts

  • Calls (per minute/hour) — total call volume, all engines & statuses combined. The ok/error split is not drawn here (the series API has no status dimension) — read it from the error-rate tile and "Calls by status".
  • Latency (avg + max, ms) — ↓ average is call-count-weighted per bucket; max is the true max. Only avg + max (no client-side percentiles).
  • Tokens — tokens_in vs tokens_out per bucket.
  • Calls by engine — volume per engine, summed over the range.
  • Calls by model (upstream) — grouped by upstream id (≈1:1 with the model it serves).
  • Calls by status — the ok-vs-error split, summed over the range.

6.2 SLA & Throughput (executive)

"Are we meeting service levels?" — the same call data as Overview, but framed around latency, errors, queueing, and the GPU fleet behind them.

Stat tiles

Tile Direction Meaning
Error rate (window) ↓ Share of all calls in the range that failed (anything outside ok/completed). Same definition as Overview's error rate, repeated here as the headline SLA number.
Fleet latency p95 / p99 ↓ Window-wide p95/p99 across all engines combined, taken from the API's exact overall aggregate — not a recombination of per-engine numbers (a mean of p95s isn't a p95). Reads — if the build doesn't serve percentiles.
GPU utilization now • context The latest utilization reading for each GPU from the DCGM sampler. Only samples ≤2 min old are shown; a dead exporter reads as "no data" rather than a frozen number.
Throughput ↑ Calls per minute over the window (total calls ÷ minutes in range). Read it with latency: high throughput at low latency is healthy; high throughput while queue wait climbs means the fleet is saturating.

Charts

  • Throughput (calls) by engine ↑ — one line per engine over time; shows who's driving load.
  • Error rate (%) ↓ — per-bucket failure rate (errors ÷ calls in each bucket), so you can see spikes, not just the window average.
  • Queue wait avg (ms) ↓ — average time calls spent waiting for a free GPU, call-weighted across engines per bucket (an avg-of-avgs would misweight a low-volume engine). Rising queue wait is the earliest warning of GPU saturation.
  • Latency p95 by engine (ms) ↓ — one p95 line per engine, each from its own server-computed p95. Engines whose build omits the field are simply not drawn, never faked.
  • GPU utilization (%) • — one line per GPU (avg_util_pct per bucket, from the DCGM sampler).
  • Table — Latency by engine and status — exact avg/max (and percentiles where served) broken out for every engine × status combination.

6.3 Analytics (executive)

Usage and efficiency: who's using the platform, for what, and how well caching is paying off.

Stat tiles

Tile Direction Meaning
Total calls (window) • Every AI call in the range — the raw usage volume.
Busiest operation • The single operation (chat, extract, classify, …) with the most calls in the range, plus its count. Tells you what the platform is mostly doing.
Cache-hit rate ↑ Share of calls served from cache (cache_hits ÷ calls) across the window. Higher = more requests answered without hitting a model = cheaper and faster. Reads — on builds that don't report cache hits (unknown, never a fake 0%).
Tokens in / out • Total prompt vs completion tokens over the range — the cost/throughput signal for LLM traffic.

Charts

  • Calls by operation — volume per logical operation, summed over the range.
  • Calls by model (upstream) — volume per upstream/model.
  • Tokens by engine — combined (in + out) tokens per engine over time.
  • Cache-hit rate (%) ↑ — per-bucket cache-hit rate; buckets predating the cache feature are left out, never plotted as a fake 0%.
  • Table — Per-engine rollup — per tenant/engine: calls, errors, error rate, tokens in/out, and cache hits/rate, sorted largest tenant first.

6.4 Report (executive)

A print/export-friendly Operational report: Key figures, Per-engine rollup, and Latency by engine and status — the same numbers as the other views, laid out for sharing and CSV export.

6.5 Calls (operator) — the call explorer

A searchable, keyset-paginated Datagrid of individual AI calls.

Filters: Engine (e.g. ahu-chatbot) · Upstream (e.g. qwen-35b) · Trace ID (paste a trace-… / X-Request-Id to find one exact request chain).

Columns: Time · Engine · Surface · Model · Status · Traffic · Queue · Total · Tokens (in/out).

Click a row to open the Call detail panel, which shows the full event: Event ID, Trace ID, Engine, Surface, User ID, Upstream, Upstream class, Model, Operation, Traffic class, Status, Queue, Total, Tokens (in/out), Pages, Doc hash, Job ID, Parent job ID, Config version, Body truncated.

The Fleet health card here (GPU queue depth, replica health, autoscaler state) is a labeled placeholder — "Available when GPU cluster management (P3) ships." It's intentionally empty, not broken.

6.6 Audit (auditor) — tamper-evidence & access trail

Three tools:

  • Chain verify — checks the hash-chained integrity of the audit log. Returns ok: true, or ok: false with the position of the first broken link (bad_chain_pos).
  • Access log — who looked at sensitive bodies. Columns: At · Actor (hash) · Role · Event ID · Action. The actor is a sha256(token) prefix, never a raw token or a human name.
  • Body viewer — the raw request/response payload for one call. Every body view is server-logged (it writes an access-log row); the UI shows a "this view is recorded" notice and cannot suppress that.

7. Field glossary

The raw per-call fields you'll see in the Calls grid and detail panel:

Field Meaning
event_id Unique id of this audit event (used to fetch its body).
ts When the call happened (rendered in WIB).
engine The tenant/engine slug that made the call (X-Tenant-Id), e.g. ahu-chatbot.
surface Which product surface triggered it (X-Surface).
user_id End user attributed to the call (X-User-Id).
trace_id Request correlation id (X-Request-Id) — ties a call to its logs across services.
upstream The specific model server the gateway routed to (e.g. qwen-35b, ext-dashscope-397b).
model Model name served by that upstream.
upstream_class on_prem or external_dev. Production refuses external_dev.
traffic_class Priority lane: interactive, batch, or system (X-Priority).
operation The logical operation (e.g. chat, extract, classify).
status ok / completed = success; anything else = error.
error_code Machine error code when the call failed (blank on success).
queue_ms Time spent waiting in the GPU queue before processing — ↓ lower is better (doesn't count toward the processing timeout).
upstream_ms Time the upstream model actually spent — ↓ lower is better.
total_ms End-to-end time for the call — ↓ lower is better.
tokens_in / tokens_out Prompt / completion token counts (null if not applicable, e.g. OCR).
doc_hash Hash of the processed document (OCR/classifier paths) — dedupe/trace without storing the doc.
pages Page count for document jobs.
job_id / parent_job_id Async job id and its parent (for the queued gateway job path).
config_version Gateway/engine config version in effect for the call.
body_ref / body_truncated Whether a stored request/response body exists, and whether it was truncated at ingest.

Status colors: green = ok/completed/success · red = error/failed · gray = unknown · purple = special (e.g. queued/system).

8. Honesty notes worth remembering

  1. — means "the API didn't serve this" (e.g. p95/p99 or cache-hit on older observatory builds), never a zero dressed up as data.
  2. No client-side percentiles. A mean of p95s isn't a p95, so per-group percentiles are shown only where the server computed them exactly.
  3. "External calls (prod)" being nonzero on the dev host is expected; it's a proof tile, not an alarm, unless you see it against production.
  4. GPU staleness: a dead DCGM exporter shows an empty GPU list (honest "no data"), never a frozen last-known number.
  5. Body views are always recorded — opening a payload in the auditor view leaves an access-log row you cannot erase from the UI.

Source: ahu-observatory-dashboard (React SPA) over ahu-ai-observatory (Go :8300 API). Fields verified against the shipped API client (src/lib/api.ts) and view components.