← All posts

Offline-first sync: what happens when two phones edit the same record

Making a mobile app work offline is the easy half. The hard half starts when the phone reconnects and finds that someone else changed the same record in the meantime. The worst sync bugs are not network bugs. They are quiet data loss: an edit that was saved, shown as saved, and then overwritten without anyone noticing.

This article walks through the parts of an offline-first sync layer as we build it, and the decision behind each one: where writes go first, how retries stay safe, which conflict strategy to pick, why the phone’s clock cannot be trusted, and how deletes survive a device that was offline for a week.

The shape of an offline-first write is simple. The user’s edit goes into one local transaction that writes both the changed row and an outbox entry to the local SQLite database. When the phone is online, a sync worker drains the outbox in order and pushes each operation to the sync API, which checks the idempotency key and the record version. The app then pulls the changes made since its last cursor and merges them into the local database. The UI only ever talks to the local database; the network is a background job.

Write locally first, and keep an outbox

An offline-first app treats the local database as the source of truth for the screen. When the user taps save, we write the changed row and an outbox entry in the same local transaction. The outbox entry records the operation, not the whole object: “set field price on item 42 to 18”, with the base version the user was looking at. If the transaction commits, the change exists, whether or not the network does.

A background worker drains the outbox in order. On Android that is usually WorkManager, whose default initial backoff for retried work is 30 seconds according to the WorkRequest reference: https://developer.android.com/reference/androidx/work/WorkRequest. On iOS, background time is granted at the system’s discretion, so we also drain the outbox on app start and on every return to the foreground.

Retries must be idempotent

A mobile request can succeed on the server and still time out on the phone. The worker sees a failure and sends the same change again. Without protection, that is a second order, a second payment or a counter incremented twice.

The fix is an idempotency key: a random ID generated once when the outbox entry is created and sent with every attempt. The server stores the result under that key and returns it again on a retry. Stripe’s API documents the same pattern, recommends V4 UUIDs as keys, and allows keys to be pruned once they are at least 24 hours old (Stripe API reference: https://docs.stripe.com/api/idempotent_requests). For offline apps we keep keys far longer, because a phone can sit in a drawer for a week before it retries.

Here is how a timeout plays out with a key in place:

  • The phone pushes an operation with key K1, and the API saves the result under K1.

  • The response is lost on the way back, so the phone sees a failure.

  • The worker retries with backoff and sends the same operation with the same key K1.

  • The API finds K1 already stored and returns the same stored result.

  • The change is applied once and acknowledged twice, with no duplicate.

Pick a conflict strategy per entity

Two devices edit the same record while one of them is offline. Something has to decide what the record looks like afterwards. The default in many tools is last write wins, and it is a real default: Firestore’s offline documentation says plainly that for multiple changes to the same document, “it’s last write wins” (Firebase docs: https://firebase.google.com/docs/firestore/manage-data/enable-offline). Applied to whole records, it means one person’s edit disappears even when the two people changed different fields.

We choose a strategy per entity, not per app. The four we use most:

  • Last write wins on the whole record keeps the latest save and silently loses the other edit. It fits settings and drafts that only one person edits.

  • Field-level merge keeps edits to different fields, but clashes on the same field still need a rule. It fits profiles, tasks and catalogue items.

  • Server-authoritative versions never accept a stale write, at the cost that the client must retry or ask the user. It fits orders, stock and money.

  • A CRDT such as Automerge or Yjs keeps every concurrent edit, at the cost of history in storage and harder queries. It fits shared text, lists and whiteboards.

Server-authoritative versioning is our default for anything involving money or stock. Each record carries a version number; a push names the version it was based on, and the server rejects it if the record has moved on. The client then rebases its pending operation onto the new state or shows the user both values. For data edited by many devices without a central owner, version vectors (one counter per device) tell “this replaced that” apart from “these happened concurrently”.

CRDTs merge concurrent edits automatically and are the right tool for collaborative text. Their historic cost was memory. Automerge 3.0, released in July 2025, reports cutting memory use by more than 10x; in its own example, pasting the text of Moby Dick takes 700 MB in Automerge 2 and 1.3 MB in Automerge 3 (Automerge blog: https://automerge.org/blog/automerge-3/). That is the project’s own benchmark, so we measure with our real document shapes before committing.

The phone’s clock is not a timestamp you can trust

Last write wins needs to know which write was last. A phone’s wall clock cannot tell you. Android’s own SystemClock documentation (https://developer.android.com/reference/android/os/SystemClock) warns that the wall clock can be set by the user or the phone network, so it may jump backwards or forwards. Even Google’s Spanner, with GPS and atomic clocks in its datacentres, reported clock uncertainty of about 1 to 7 ms (Corbett et al., OSDI 2012, an older paper). A phone whose user set the clock by hand has no such bound at all.

When we need ordering, we use a hybrid logical clock. Described by Kulkarni and colleagues in 2014, an HLC stays close to physical time but never goes backwards, and it fits in a 64-bit timestamp. Each timestamp is a wall-clock part plus a small counter. On every event the clock takes the largest wall time it has seen, from its own clock or from an incoming message, and bumps the counter when the wall time has not moved.

The difference in short:

  • A device wall clock can be changed by the user or the network, can jump backwards, disagrees between devices, and a wrong order means a wrong winner.

  • A hybrid logical clock never goes backwards, respects cause and effect, stays close to real time and fits in 64 bits.

Deletes need tombstones

If a delete simply removes the row, a device that missed it will push its old copy back on the next sync and the record comes back to life. So a delete becomes a tombstone: a marker that says “item 42 was deleted at this HLC”, synced like any other change. Cassandra solves the same problem server side and keeps tombstones for a default of 864,000 seconds, ten days, so that a node that was down can still learn about the delete (Cassandra docs: https://cassandra.apache.org/doc/latest/cassandra/managing/operating/compaction/overview.html). On mobile we set the grace period from data: the longest time a real device stays offline, plus a margin. A device that returns after the grace period does a full resync instead of an incremental one.

Test sync by breaking it on purpose

Sync bugs hide in timing, so unit tests alone will not find them. We test in four layers:

  • Merge functions: pure tests that check the same inputs in any order give the same result.

  • Two simulated clients: scripted offline edits, then sync in both orders.

  • Fault injection: dropped responses, duplicated requests and clocks skewed by hours.

  • Real devices: airplane mode in the middle of a save, the app killed and restarted, then the data compared.

The property we check everywhere is convergence: after all devices have synced, they hold the same data, and no acknowledged edit is missing. A randomised test that replays thousands of interleavings against that property finds more bugs than any list of hand-written cases.

What to take from it

  • Write to the local database and the outbox in one transaction. Then a save is a save, online or not.

  • Send an idempotency key with every push. Keep keys longer than your longest offline period.

  • Choose a conflict strategy per entity. Last write wins for settings, server versions for money, CRDTs for shared text.

  • Never order writes by the device clock. Use a hybrid logical clock or server-assigned versions.

  • Turn deletes into tombstones. Size the grace period from real offline durations.

The short version: offline is easy. Agreeing on what happened while you were offline is the actual product.

If you are planning an app that has to work in a warehouse, on a farm or on the underground, we are happy to talk through the sync design: https://www.commonsaltx.com/contact