Write-Ahead Logging (WAL)
The crash-recovery mechanism nearly every database relies on — whether it's built on a B+ Tree or an LSM Tree underneath.
Overview
Write-Ahead Logging is the crash-recovery mechanism nearly every database engine relies on, regardless of whether it uses a B+ Tree or an LSM Tree underneath — every change is first appended to a sequential log on disk before it's applied to the actual data structure, so a crash can never lose a write that was already acknowledged as committed.
Why It Exists
Applying a change directly to a B+ Tree page — or even to an LSM Tree's in-memory MemTable — isn't durable on its own. A crash before that page or in-memory state reaches stable storage would lose data the application was already told was safely committed. WAL exists to separate 'the change is durable' from 'the change is fully applied to its final structure' — the log alone is enough to guarantee durability and to replay the change after a crash, whatever the underlying engine actually is.
Real World Example
A transaction updates a row. Before that update ever touches the actual table page, the engine appends a compact log record describing the change to the WAL and flushes it to disk. Only then is the transaction considered committed and acknowledged back to the caller — this is the exact mechanism behind ACID's Durability guarantee from earlier in this path. If the server crashes the instant after, recovery on restart replays the WAL from the last checkpoint, reapplying any committed change that hadn't yet reached the actual data pages.
Example Data
A single UPDATE, from the caller's perspective vs. what actually happens on disk
| Step | What Happens | Durable Yet? |
|---|---|---|
| 1 | UPDATE arrives | No |
| 2 | Change appended to WAL, flushed to disk | Yes — commit acknowledged here |
| 3 | Change later applied to the actual data page | Already was, since step 2 |
How WAL Actually Works
Append, Don't Overwrite
A WAL is written purely sequentially — new records are always appended to the end. Sequential writes are dramatically cheaper than the random writes an in-place update to a data page would require, which is part of why WAL is fast enough to sit on every transaction's critical path.
The Log Is the Source of Truth Until Data Catches Up
Once a change is durably in the WAL, the transaction is safely committed — even if the actual data page hasn't been updated yet. The engine can apply that change to the real data structure at its own pace afterward, because the log guarantees it can always be redone.
Checkpointing — Trimming the Log
Periodically, the engine confirms that all changes up to a certain point have been fully applied to the actual data, and safely discards WAL entries older than that point — otherwise the log would grow forever and crash recovery would take longer and longer to replay.
Different Engines, Same Idea
A B+ Tree engine logs page changes before applying them in place. An LSM Tree engine logs writes before they land in the (volatile) MemTable. Both rely on exactly the same underlying principle — log first, apply after.
Diagram
The WAL flush is the actual durability boundary — everything after it can happen later
Write request arrives
Change appended to WAL
sequential write, flushed to disk
Commit acknowledged to the caller
durable as of this point
Change applied to the actual data structure
can happen slightly later
Checkpoint
old WAL entries safely discarded
Common Mistakes
Assuming the write to the main data file is what makes a commit durable
Why: It's specifically the WAL flush that provides durability — the actual data page can be updated later, since it can always be reconstructed by replaying the log from that point.
Fix: Recognize the WAL flush as the real durability boundary, not any other disk write that happens to occur around the same time.
Letting the WAL grow without bound
Why: Without periodic checkpointing, the log file grows indefinitely, costing disk space and lengthening crash-recovery time, since more log has to be replayed after a restart.
Fix: Understand and monitor your engine's checkpoint interval and behavior rather than assuming it's automatically fine at any write volume.
Assuming WAL protects against every form of data loss
Why: WAL protects specifically against a crash or power loss after a commit was acknowledged. It does nothing for a hardware failure that destroys the disk the WAL itself lives on, or for a logical error like an accidental DELETE that gets faithfully logged and replayed.
Fix: Pair WAL-based crash recovery with real backups and replication, which protect against disk loss and human error — WAL alone doesn't cover those cases.
Interview Questions
What problem does write-ahead logging solve?
It guarantees that a transaction acknowledged as committed will survive a crash, by making sure the change is durably logged before the commit is confirmed — even if the change hasn't been fully applied to its final data structure yet.
Why is appending to a WAL fast compared to updating the actual data page in place?
A WAL append is a purely sequential write to the end of a file, which disks (especially spinning disks, but even SSDs to a lesser degree) handle far more cheaply than a random write to an arbitrary page location — which is exactly what an in-place update to a B+ Tree page requires.
After a crash, how does an engine use its WAL to recover, and what happens to transactions that never committed?
On restart, the engine replays the WAL from the last checkpoint, reapplying every change belonging to a transaction that reached a logged COMMIT record — restoring exactly the state that was acknowledged before the crash. Any transaction whose COMMIT was never logged is treated as if it never happened, and its partial changes are discarded — this is Atomicity and Durability working together through the same log.
Production Best Practices
Do
✓Treat the WAL flush as the real durability boundary in any discussion of commit safety.
✓Monitor checkpoint behavior and interval, especially under heavy write load.
✓Pair WAL-based crash recovery with real backups and replication for disk-loss scenarios.
Don't
✗Don't assume durability comes from the main data file write instead of the WAL flush.
✗Don't let WAL growth go unmonitored — an unbounded log lengthens crash recovery.
✗Don't treat WAL as a substitute for backups — it protects against crashes, not disk loss or human error.
Comparison
| Crash Recovery | Write Pattern | Durability Guarantee | |
|---|---|---|---|
| Without WAL | Data since last full flush can be lost | Random writes to final structure | Weak — depends on OS/disk flush timing |
| With WAL | Replay the log, no acknowledged write lost | Sequential log append first | Strong — commit means logged and flushed |