AI/ML Gateway
Long-running work — reports, AI calls, bulk jobs — runs in the background and notifies users live, so the core API never blocks.
In plain words
Some tasks, like generating a big report or calling an AI model, take a minute or more. Instead of making people stare at a spinner, the gateway takes the job, works on it in the background, and pings the user's screen the moment it's done.
Interactive preview
How AI/ML Gateway works
Interactive · illustrative sample data
The challenge
Holding an HTTP request open for 30–90 seconds risks load-balancer timeouts, duplicate work when users refresh, and a clogged API.
Our approach
Enqueue and notify: accept the job immediately, process it asynchronously from a Redis queue, then push the result to the right browser over WebSockets.
Step by step
From first click to finished job
- 1
User starts a long task
The main app enqueues it and returns a job ID immediately.
- 2
Worker processes the job
From the Redis queue, with correlation IDs for tracing.
- 3
Browser gets notified
A Socket.IO event delivers progress and completion live.
Capabilities
What it does
Redis job queues
Durable job storage with retries.
Socket.IO rooms
Per-user/session rooms so notifications never leak across accounts.
Idempotent handlers
Retries don't duplicate side effects.
Graceful degradation
If sockets drop, clients can poll job status.
Engineering
Key decisions
Respect request lifetimes
Heavy work never runs inline in the core API.
Architecture
Tech stack
Service
- Node.js
- Express
- TypeScript
Realtime & jobs
- Socket.IO
- Redis
- Sequelize
Outcome
What it delivers
- Core API stays responsive while heavy jobs run in the background
- Users see live progress instead of timeouts
Deep dives
Read the engineering notes
WebSockets and Redis Queues — Async UI Without Blocking the API
Building an event gateway with Socket.IO and Redis so long-running reports and bulk jobs notify users in real time instead of timing out the browser.
Prime Alley Microservices — One Product, Four Deployable Services
How we split billing, accounts, payments, and live updates into separate services while keeping a single login and navigation shell for end users.
Services behind it