How we cut over a system that is not allowed to stop

The systems worth replacing are the ones a business runs on continuously, which is exactly what makes them hard to replace. A booking engine takes orders at 3 a.m. A settlement service has a cycle it cannot miss. There is no maintenance window worth the name, so the standard plan is a weekend: freeze writes, migrate, repoint, and hope. That plan works often enough to stay popular, and when it fails it fails at the worst moment available - four hours in, half the data moved, with a rollback path nobody has rehearsed. We do not take that bet on a system carrying money or bookings. We plan the cutover so that every step is reversible and the interesting failures happen while both systems are still up.

The first phase is the data, and it happens long before any traffic moves. Migrating 450,000+ historical bookings ahead of a cutover was not the migration - it was the rehearsal. We copy the corpus, then reconcile it: row counts per range, checksums per partition, field-level comparison on a sample plus on every record a business rule says matters, which in practice means anything with money attached. Mismatches get classified rather than counted - a character encoding, a timezone boundary, a rounding rule, or a real semantic difference between the old model and the new. Then we run the whole thing again. The final backfill happens under time pressure, and you want it to be the twentieth run rather than the first.

The second phase is running both systems against live traffic with only one of them holding authority. The tempting way to do that is a dual write from application code - save to the old store, save to the new one, carry on. It is also the one option that cannot be made correct: a crash between the two writes leaves the stores divergent with no coordinator to finish the job, and wrapping them in a distributed transaction buys atomicity at an availability cost a live system will not pay. So we keep a single writer and propagate. The system that owns the write records its change and its outgoing event in the same local transaction - a transactional outbox - and the second store follows that ordered stream. One place decides truth; the other catches up, and can be rebuilt from the log when it drifts.

With the follower in place, reads get shadowed. The new system answers the same query as the old one, responses are compared off the hot path, and differences are logged instead of served. That is where the real defects surface: the pagination that sorts differently on ties, the currency field that rounds at a different step, the status the old code inferred and the new code stores. Only once the difference stream goes quiet do we move traffic, and we move it by a stable key - tenant, account, market, partner - rather than by percentage, so a given customer sits consistently on one system and support can answer which one. Internal accounts first, then low volume, then the rest, each slice reversible with a config change.

The old path stays warm until the last slice has been through a full business cycle on the new system: a month end, a settlement run, a peak day. Decommissioning is a dated decision taken afterwards, not the same evening. All of this costs more than a weekend - duplicate infrastructure, a comparison harness someone maintains, a stretch where every change lands in two places - and it is still the cheaper option, because what you are buying is the absence of the hour where the business cannot take orders and you are restoring a backup. We ran the same shape decomposing a .NET Framework monolith into event-driven services on Kafka, moving one capability at a time while a million-plus users kept using it. The measure of a good cutover is that nobody outside engineering can tell you the date it happened.