Order ingestion rewrite for a regional logistics operator

The operator took orders from about forty enterprise customers, each in their own format, and everything landed in one nightly batch job. That meant dispatch planned the morning routes from a snapshot that was up to eighteen hours stale, and any order placed after the cutoff simply did not exist until the next day. When the batch failed — which it did roughly twice a month — nobody found out until the operations floor started calling. The business wanted orders visible within minutes of arriving, without asking forty customers to change how they send them.

Backend lead on a team of five, reporting to the platform manager. I owned the design and about half the implementation, and I ran the cutover. I was also the person on the phone at 4am during the first three switchovers, which turned out to be the useful part of the role.

I designed and built the ingestion service itself: the per-customer adapter layer, the normalisation step, the idempotency keys, and the dead-letter path for anything that could not be parsed. I wrote the replay tool that lets an operator re-run a single customer's day without touching anyone else's data, which is the piece the support team actually uses. I also owned the database work — the new order tables, the backfill of eleven months of history, and the indexes we needed once dispatch started querying live. Documentation and the runbook were mine as well.

The obvious move was a message broker with one topic per customer, and I argued against it. We were a five-person team with no broker experience on call, and the real requirement was minutes, not milliseconds — so I put the queue in PostgreSQL with SELECT ... FOR UPDATE SKIP LOCKED and kept the whole thing on infrastructure the team already knew how to debug at 3am. I also insisted every adapter write the raw payload to object storage before parsing, which the team thought was wasteful; it paid for itself the first week when a customer changed a field silently and we could replay three days from the originals. The one place I did add new infrastructure was Redis, purely for the dedupe window, because doing it in the database made the hot path noticeably slower.

No downtime window — the operator runs six days a week and the seventh is when the warehouse counts stock. We also could not ask customers to change their integrations, so every quirk had to be absorbed on our side, including one customer sending fixed-width text with a trailing checksum. Budget kept us on the existing managed database rather than anything new, and two of the five engineers were part-time on another product for the first two months.

Orders went from up to eighteen hours stale to visible in about ninety seconds at the median. Dispatch moved their planning cycle from 5am to a rolling one, which the operations manager said took roughly an hour out of every morning. Failed imports stopped being invisible: the dead-letter queue plus the replay tool meant the support team resolved most bad files themselves, and escalations to engineering dropped from a handful a week to maybe two a month. We ran both systems in parallel for six weeks and reconciled counts daily before turning the batch off.

Two backend engineers built adapters alongside me, and one of them ended up owning the whole adapter layer by the end, which was the goal. A data analyst on the customer side wrote the reconciliation queries we used to prove parity during the parallel run. Our operations manager sat with us for the format discovery and saved us weeks of guessing.