← All insights
Engineering·4 min·June 2, 2026

Holding a sub-100ms latency budget for two years

RDVC

Sub-100ms from price change to placed order reads like a single number. In production it is a sequence of hops that each take a share of it: receive and decode the market event, evaluate the strategy, run the checks that stop a duplicate order, serialize the request, and place it against a broker's API over a network you do not control. So the budget gets written down as an allocation before the code exists - this hop gets 8ms, that one gets 15, the external call gets what is left and a ceiling. The allocation is the part that does the work. It turns speed into a constraint with a number attached, and anyone proposing a new step on the path has to say which hop is giving up its milliseconds. Without it, keeping things fast is a preference, and preferences lose arguments to features.

The number on the dashboard is a percentile, not a mean. Averages are close to useless on a path like this, because slow events are not randomly distributed - they cluster in the conditions that matter most, when the market moves, the event rate spikes, a collection pause lands, or a lock is contended by the same burst that caused it. We track p99, and p99.9 where the volume supports it, measured from the timestamp on the event rather than from the moment our process picked it up. Measuring from your own ingress is the most common way a latency dashboard lies: it hides queueing, and queueing is usually where the time went.

Architecturally, the budget becomes a rule about what is allowed on the hot path. No synchronous call to anything we do not control. No lock held across I/O. No allocation-heavy work inside the loop, and no logging that can block on a disk. Everything else - persistence, analytics, notification, anything a human will eventually read - goes onto the messaging core and is handled by a process that is not in the way. That is what the messaging core earns its place for in a trading system: not throughput, but keeping work that is not on the critical path off the thread that is. Ordering belongs in the same conversation, because a guarantee that event N is handled before event N+1 is a queue, and a queue is head-of-line blocking waiting for a slow moment. Where ordering is genuinely required, scope it to the narrowest key that needs it - one instrument, one account - so a slow item blocks only its own key.

Correctness pulls the other way, and it wins. An order placed twice costs more than an order placed late, so the duplicate check sits on the critical path by necessity. That does not exempt it from the budget; it means it gets its own line in the allocation and has to be built to fit - an in-process guard keyed on the identity of the signal, backed by a durable record, rather than a round trip to a shared store before every order. The shape generalizes past trading. Making the fast version also the safe one is a design problem you solve at the start, while the check can still be cheap. Deciding it “must be fast” without a number is how it becomes 30ms two years later, by which point it is load-bearing.

Which is the actual difficulty: systems do not get slow in one commit. They get slow across two hundred commits that each cost half a millisecond and none of which was anyone's fault. The only thing that holds a budget over years is making that drift visible and expensive - a latency test in CI on the hot path with a threshold that fails the build, and production percentiles alerted against the same numbers, so a regression is caught in the week it shipped rather than the quarter someone noticed. That trading platform has been in continuous production development for over two years with a four-person team, and it is still inside the number. There is no heroics in that. Write the number down, measure the tail, and let it fail a build.

Want engineering like this?

Newsletter

Notes from the team, now and then.

Occasional, engineering-grade. No spam, unsubscribe anytime.

Start a project →