Anti-Entropy Sync

Anti-entropy is a background process that ensures all nodes in the mesh eventually converge to the same cache values. It runs on a timer without blocking normal operations.

---

How It Works

1. Periodically, a node selects a random peer.
2. The node sends its cache summary (keys + timestamps + vector clocks).
3. The peer compares the summary against its own cache.
4. The peer sends back any entries that the requesting node is missing or has outdated.
5. The requesting node applies the updates.

This process is based on a gossip-style epidemic protocol — each node only talks to one peer at a time, but over time every node learns about every change.

When Anti-Entropy Runs

• On a configurable timer (every 30 seconds by default)
• When a new node joins the mesh (initial sync)
• After a network partition heals

---

Anti-Entropy Entries

auto& cache = app.get_state()->of_cache();
auto entries = cache.get_anti_entropy_entries();

for (auto& entry : entries) {
    fmt::println("Key: {}, created: {}, TTL remaining: {}ms",
        entry.key, entry.created_at, entry.ttl_remaining_ms);
}

Each anti_entropy_entry contains:

Field	Description
key	Cache key
created_at	Creation timestamp (nanoseconds since epoch)
ttl_remaining_ms	Remaining time-to-live in milliseconds
clock	Vector clock at creation time

---

Consistency Guarantees

These guarantees describe what you can expect from the distributed cache:

• Eventual consistency — All nodes converge to the same values, given enough time without new writes. In practice, convergence completes within a few anti-entropy cycles (seconds to minutes, depending on cluster size and network latency).
• Read-your-writes — A write is immediately visible on the writing node. Other nodes will see the value after the next anti-entropy cycle.
• Monotonic reads — Once you see a value on a node, you will never see an older version of that key on the same node. Reads are non-regressive.

---

Next: Jobs & Queues.