Write-Ahead Log (WAL)

The WAL provides durability for messages. Every message can be persisted to disk before processing, ensuring recovery after crashes or restarts.

When is the WAL Used?

The WAL is used internally by the framework for critical state that must survive crashes. Specifically:

• Job queue persistence — Dispatched jobs are written to the WAL so they are not lost if the node crashes before processing.
• Cache anti-entropy metadata — Internal cache state used for cross-node synchronization.

User-level messages (HTTP requests, WebSocket frames, Pub/Sub messages) are not automatically persisted. You must call wal.push() explicitly if you need custom durability for your application data.

---

Configuration

The WAL path is set via configuration:

using namespace framework;

config.wal_.enabled_.store(true);
config.wal_.path_.store(
    std::make_shared<const std::string>("storage/wal.log"));

Field	Default	Description
wal_.enabled_	true	Enable or disable the WAL.
wal_.path_	"storage/wal.log"	File path for the WAL log.

The WAL is initialized automatically from config.wal_.path_ at app startup. You do not need to call initialize_wal() manually unless you want to override the configured path.

---

How It Works

1. Push — A message arrives and is persisted to the WAL file via wal::push(tid, buffer).
2. Process — The message is processed by the application.
3. Ack — After processing, the message is acknowledged via wal::ack(tid).
4. Recovery — On restart, wal::recover() returns all unacknowledged messages for reprocessing.

---

WAL API

auto& wal = app.get_state()->of_wal();

push(tid, buffer) — Persist a message

bool success = wal.push(transaction_id, message_buffer);

Param	Type	Description
tid	boost::uuids::uuid	Transaction ID — used to acknowledge later.
buffer	shared_buffer	The message payload to persist.
Returns	bool	true if successfully written to disk.

Generate a boost::uuids::uuid with:

boost::uuids::uuid tid = boost::uuids::random_generator()();

ack(tid) — Acknowledge

Marks a message as processed. On recovery, acknowledged messages are skipped.

wal.ack(transaction_id);

recover() — Recovery

Returns all unacknowledged entries for reprocessing.

std::vector<wal::entry> entries = wal.recover();
for (auto& entry : entries) {
    // entry.tid — transaction ID
    // entry.payload — the raw message bytes
    reprocess(entry);
}

clear() — Clear

Removes all entries from the WAL (acknowledged and unacknowledged).

wal.clear();

---

Initialization

The WAL is initialized automatically from config.wal_.path_ at startup. To override the path at the code level:

state->initialize_wal("/custom/path/wal.log");

This overrides config.wal_.path_ for this session only.