KitePDF system architecture

This is not a README pasted into Benchlog. It explains the design problems KitePDF solves, the boundaries between components, and why the stack looks the way it does. Deployment commands and env file names live in later parts; here we focus on mental models you can reuse when you extend the product.

KitePDF turns PDF work into durable, billable jobs: upload once, process asynchronously, download when ready. That shape forces three non-negotiables: a public edge that stays thin, a policy layer that owns identity and credits, and stateless workers that can scale independently. Everything in the architecture serves one of those three.

We split identity into two stores on purpose. Better Auth in the Next.js app owns login sessions, email verification, 2FA, and admin plugins against the frontend database. The Go api-gateway never sees passwords—it only accepts cryptographically verifiable JWTs and maps them to rows in the backend database. That separation keeps the high-churn auth surface in one codebase while the job and credit domain stays in Go.

Better Auth handles email/password, verification, password reset, bearer support, optional two-factor, and admin capabilities. User records live in frontend Postgres. On signup (or guest conversion), a database hook calls the api-gateway internal API to create or link a backend user keyed by auth_id—the same identifier embedded in JWTs.

The Better Auth JWT plugin signs access tokens (payload includes id, email, name, role). The browser obtains a fresh JWT via getSession—the Set-Auth-Jwt response header—and sends Authorization: Bearer on api-gateway requests. The gateway does not share a static secret with the frontend; it bootstraps a JWKS client (AUTH_JWKS_URL, typically /api/auth/jwks on the app origin) and validates signature, issuer, and audience on every protected request.

Anonymous users can still run a subset of tools. POST /guest/session creates a backend anonymous user; a signed httpOnly cookie (and optional X-Guest-Token) proves guest identity on flex routes. Flex middleware tries JWT first (AuthenticateJWTOptional), then guest cookie (AuthenticateGuest), then RequirePrincipal ensures someone is present. Member-only routes stack RequireMemberJWT with RejectAnonymous so dashboards, purchases, and paid tools require a real account.

Authentication answers who is calling. Authorization answers what they may do. KitePDF layers policy in middleware rather than scattering checks in handlers.

Tool handlers are thin: validate input, enqueue work, return a job id. Workers own CPU and IO-heavy PDF operations. The gateway remains the system of record for job state; workers report outcomes through internal callbacks. S3 holds bytes; the gateway issues short-lived presigned URLs so browsers never receive long-lived object credentials.

Queue routing by tool type (Node vs Python) is a scaling knob: add consumers per queue without forking the API. The pattern is identical; only the worker implementation changes.

Frontend Postgres holds auth and UX-adjacent state. Backend Postgres holds jobs, credits, files metadata, and anonymous users. Neon or local containers are deployment details; the split is logical. Redis supports rate limiting and ephemeral coordination. This boundary lets you patch auth plugins without migrating job history, and vice versa.

Part 3 (planned) walks local setup. Part 6+ will cover horizontal worker scaling and observability. Keep this page as the map: edge → policy (authn/authz) → queue → workers → storage.