Task 2 report — Durable job store (Store interface + LocalStore)
Branch: feat/gateway-p2-jobs-ocr · New package: internal/jobs
(Note: this file previously held a stale report from an earlier plan's "Task 2 / registry"; overwritten for the gateway-P2 job store task.)
Files
internal/jobs/job.go—Jobstruct,JobStatustype +StatusQueued|Processing|Completed|Failedconsts,clone()helper.internal/jobs/store.go—Storeinterface,ErrNotFound,LocalStore(in-memory) +NewLocal().internal/jobs/store_test.go— namespace-safe tests (uniquet2-*ids/tenants, filtered assertions, no bare counts).
Design — mirrors the pool.Pool Local/Redis split
Store is the durable-state interface (Redis impl is Task 3, fan-out is Task 8 — not built here). LocalStore is mutex-guarded with three maps:
- byID map[string]*Job — canonical records.
- payloads map[string][]byte — the (possibly large) submit payload is kept OUT of the Job record and stored separately, retrievable via Payload().
- idem map[tenant]map[key]id — the idempotency index.
All accessors return clone()d copies (deep-copying Pages and Result) so callers can't mutate stored state outside the lock.
Method semantics
- Create(ctx, *Job, payload) — dedups on
(Tenant, IdemKey). If the key exists, returnsexisting=true, copies the found job back into the caller's*Job(sojob.IDpoints at the existing job), and does NOT insert a second record or overwrite the payload. Otherwise assignsID = audit.NewID()(ULID, monotonic-time so CreatedAt ordering is stable), setsStatus=queued,CreatedAt/UpdatedAt, stores a copy of the payload. Jobs with an emptyIdemKeyare never deduped (fan-out children). - Get / Payload — return copies;
ErrNotFoundon unknown id (API maps to404 JOB_UNKNOWN). - ClaimNext(ctx, class) — scans all queued jobs and picks the best by
(classRank, CreatedAt, ID)whereclassRank = interactive(0) < system(1) < batch(2)(matchespool.rankso admission and claiming agree). Transitionsqueued→processing, setsStartedAt+Heartbeat=now,Attempts++. Theclassarg is advisory (kept for Redis per-class-list parity); Local scans all classes in priority order so a second claim never re-hands a processing job. - Heartbeat — refreshes the liveness clock + UpdatedAt.
- UpdateStatus(id, fn) — applies the mutation under lock, then bumps UpdatedAt.
- SetResult(id, status, result, errorCode) — terminal write; copies the result slice.
- ListChildren(parentID) — returns only jobs whose
ParentID == parentID, sorted by CreatedAt then ID. - ReclaimStale(visibilityTimeout) — acks-late recovery: any
processingjob whoseHeartbeatis older than the timeout is set back toqueued(Attempts is NOT bumped here — it increments on the subsequent re-claim, per the plan's "Attempts already ++'d" wording). Returns the reclaim count.
RED → GREEN
RED (before impl): go test ./internal/jobs/ → build failed — undefined: Job / NewLocal / StatusQueued / StatusProcessing / ....
GREEN (after impl): all 6 tests pass —
- TestCreateDedup — resubmit same (tenant,key) → existing=true, same id, original payload preserved; cross-tenant same key = distinct job.
- TestClaimNextPriority — batch enqueued before interactive, interactive claimed first; second claim hands the batch job; third claim returns nothing.
- TestReclaimStale — forced-stale heartbeat re-queued and re-claimable; Attempts→2 after re-claim.
- TestListChildren — only the target parent's children; unrelated parent's child excluded.
- TestSetResultAndGet, TestGetUnknown.
Verification
go build ./...— OKgo vet ./...— OKgo test ./...— all packages pass (jobs 0.009s); additive, no existing package behavior changed.
Concerns / notes for later tasks
ClaimNext's full-map scan is O(n) — fine for Local (dev/small). Task 3's Redis impl uses per-class ready lists (BRPOPLPUSH) for O(1) claim + real cross-replica once-only semantics.- Attempts increments only at claim time (not at reclaim), so a job reclaimed but never re-claimed keeps its prior count — matches the plan text; confirm this matches the manager's audit
attemptsexpectation in Task 7.