think
16px
820px
← all files

Reusable pattern: services pull secrets from OpenBao (AppRole)

Date: 2026-07-10
Status: established + proven end-to-end with Grafana. This is the recipe every other service (gateway, observatory, DBs) follows to get its secrets from OpenBao instead of hand-written env files.

The shape

OpenBao (ahu-ctrl-01:8250)                         service host (e.g. ahu-gov-01)
  ├─ auth: approle                                   render-env.sh (run at deploy time):
  ├─ policy: <svc>-ro  (read secret/data/<svc>/*)      1. AppRole login (role_id + secret_id)  --TLS-->  short-lived token
  ├─ role:   <svc>     (bound to <svc>-ro)             2. read secret/<svc>/*                    --TLS-->  values
  └─ kv:     secret/<svc>/...                          3. render <svc>.env (600), compose reads it via env_file

Every fetch is audited in OpenBao (HMAC'd), scoped to just that service's secrets, and revocable centrally. Rotating a secret = update it once in OpenBao + re-render (one place, not N hosts).

What's live (Grafana, the worked example)

  • OpenBao side (ctrl-01): approle auth enabled; policy monitoring-ro (read secret/data/monitoring/*); role grafana (token_ttl 20m, max 1h); secret at secret/monitoring/grafana (admin_password).
  • gov-01 side: OpenBao CA cert at ~/ahu-monitoring/bao-ca.crt; approle creds ~/ahu-monitoring/grafana-approle.env (600); ~/ahu-monitoring/render_env.sh renders grafana.env from OpenBao.
  • Firewall: gov-01 → ctrl-01:8250 opened (governance→control-plane secret fetch).
  • Verified: AppRole login ok → grafana.env rendered from OpenBao → Grafana auth returns HTTP 200 with the OpenBao-sourced password → OpenBao audit log shows the login + reads.

To onboard the next service (recipe)

On OpenBao (with an admin token):

bao policy write <svc>-ro -    # path "secret/data/<svc>/*" { capabilities=["read"] }
bao write auth/approle/role/<svc> token_policies=<svc>-ro token_ttl=20m token_max_ttl=1h
bao read  -field=role_id   auth/approle/role/<svc>/role-id      # -> deliver (not secret)
bao write -f -field=secret_id auth/approle/role/<svc>/secret-id # -> deliver securely (600)
bao kv put secret/<svc>/... key=value                           # store the actual secrets

On the service host: drop the CA cert + a render_env.sh (copy Grafana's, swap the role/paths), and call render_env.sh before docker compose up in the deploy script so every deploy re-pulls from OpenBao. For DBs, use the database/ engine so credentials are generated + rotated by OpenBao rather than static.

Known tradeoffs / hardening follow-ups

  • Secret-zero: the secret_id sits in a 600 file on the service host to bootstrap AppRole auth. That's still "a secret on the host," but it's scoped (reads only that service's secrets), centrally revocable, and every use is audited — a real improvement over the raw secret sitting there. Harden later with short-TTL secret_ids + response-wrapping, or an OpenBao Agent sidecar.
  • Rendered secret at rest: the rendered <svc>.env is plaintext on the host (compose reads it via env_file). OpenBao is now the source of truth, but at-rest isn't eliminated. Upgrade: render to a tmpfs mount, or an OpenBao Agent sidecar that renders to memory and auto-renews on rotation (this is the SOTA endpoint, worth it once the DBs with real rotation land).
  • Root token still used for admin. Setup currently uses OpenBao's root token (from the bootstrap init file). Create a proper admin policy/token and revoke root for day-to-day (part of the still-open "secure the init keys" action).