Upstream analysis — ahu-gpu-server CLOSE_WAIT / wedge

Companion to 2026-08-21-ocr-storage-and-gpu-server.md, which recorded the
recovery. This is the root cause and the fix, for the OCR engine repo
(ahu-ocr-akta-notaris-POC, gpu-server/app/main.py).

Root cause: a synchronous LLM call inside an `async def` handler

Two endpoints block the event loop:

endpoint	deployed line	upstream line
`POST /summarize`	448 / client 472 / call 479	395 / 419 / 426
`POST /cek-bukti`	763 / client 787 / call 794	710 / 734 / 741

Both are declared async def, and both call the synchronous OpenAI client:

@app.post("/summarize")
async def summarize(...):
    client = OpenAI(base_url=settings.vllm_base_url, api_key="not-needed", timeout=120.0)
    response = client.chat.completions.create(...)   # BLOCKS the event loop

uvicorn app.main:app runs one process, one event loop, no --workers. An
async def handler runs on that loop, so a blocking socket read inside it
stops everything: no other request is served, no queued response is written,
and no socket-close event is processed. With timeout=120.0 and vLLM under
load (a 27B model answering an 8192-token summary), one request can hold the
entire server for two minutes; a handful queued back-to-back holds it for far
longer.

Why this produces exactly what we saw

438 sockets in CLOSE_WAIT on :8000 — clients gave up and sent FIN; the
loop never ran the callback that would close() our side. CLOSE_WAIT is
by definition "the peer closed, the application has not".
Only 27 fds held by pid 1 — the sockets were orphaned kernel-side, not
leaked file handles. Consistent with a stalled loop, not fd exhaustion.
CPU 0.00% — blocked in a socket read waiting on vLLM, not spinning.
GET /health 200 OK in the log while every client timed out — those
lines are the backlog draining between stalls; the access log records
completion, and the next stall began before our probes could be served.
Restart clears it instantly — a fresh loop with an empty backlog.

/health itself is trivial and non-blocking; it was collateral damage. This is
why the Docker healthcheck was right to fail and why nothing in the app's own
logs looked like an error.

Fix (smallest correct change)

Drop async from both handlers. FastAPI/Starlette then runs a plain def
endpoint in an anyio worker thread, and the event loop stays free:

@app.post("/summarize")
def summarize(body: SummarizeRequest) -> dict[str, str]:   # was: async def

Verified safe: neither handler body contains a single await, so nothing
breaks in the conversion (grep -c "await " → 0 across both bodies).

Two follow-ons in the same edit:

Hoist the client. OpenAI(...) is constructed per request, so every
call builds a fresh httpx connection pool and TLS/TCP connection to vLLM.
Build it once at module scope and reuse it.
Cap the thread pool or the timeout. With def handlers the default
anyio pool is 40 threads; 40 concurrent 120s summaries would still queue.
Either lower timeout (120s is longer than any caller waits — the OCR
backend gives up first) or bound concurrency explicitly.

An alternative fix is AsyncOpenAI + await, which is more idiomatic but a
larger diff and changes error handling. The def conversion is one word per
handler and provably equivalent.

Not applied here — engine repo

gpu-server/ on the host is a deployed copy; the source of truth is
repo/gpu-server/ (ahu-ocr-akta-notaris-POC), an engine repo the workspace
rules keep hands-off, and sync-from-repo.sh would overwrite an edit made only
to the deployed copy.

The two also differ today: deployed main.py is 837 lines vs upstream 784,
and deployed carries two endpoints upstream lacks — POST /jual-beli/extract
and POST /pendirian-pp/surat-pernyataan/extract. Whoever applies the fix
should reconcile that drift first, or the sync will silently drop two live
endpoints.

Mitigation already in place