The Recovery Notebook: Your Homelab’s Most Underrated Backup
Backups get most of the attention in a homelab, but recovery usually fails somewhere more mundane: nobody remembers the order of operations.
The password manager is available after the identity service starts. The identity service needs DNS. DNS runs as a virtual machine. The hypervisor credentials are stored in the password manager. Every component works, yet the system forms a neat little circle that cannot bootstrap itself.
A recovery notebook breaks that circle. It is a short, offline document containing the minimum information needed to move from a cold, unfamiliar machine to a functioning environment. It is not a complete copy of your documentation, and it should not contain every secret. Think of it as the index and ignition key for the rest of the recovery process.
What belongs in it
Start with the physical layer: which machine runs the hypervisor, where the router and switches are, which storage device contains backups, and what must be powered on first. Include labels that make sense to someone who has never seen the rack before.
Next, record the bootstrap path. Explain how to reach the hypervisor locally, restore essential networking, locate encrypted backup media, and regain access to the password manager. If a recovery code or sealed credential is required, document where it is stored rather than copying it into every version of the notebook.
Finally, list services in restore order. Infrastructure dependencies should come before applications: network, DNS, identity, storage, databases, then user-facing services. A simple dependency order is more useful during an outage than a beautiful architecture diagram with no starting point.
Write for the stressful version of yourself
Recovery documentation should use exact commands, addresses, filenames, and expected results where those details are stable. Avoid instructions such as “restore the usual snapshot” or “log in with the admin account.” Under pressure, “usual” is not actionable.
Add checkpoints: what should be reachable after each step, which log confirms success, and when it is safe to continue. These turn a long procedure into a sequence of small tests and make it easier to identify the first broken dependency.
Keep it offline, but maintain it
A printed copy in a sealed envelope is wonderfully boring. An encrypted copy on removable media is also useful, especially when commands or configuration snippets change often. Keeping both gives you readability during a total outage and easy updates during normal operation.
Review the notebook whenever you change the router, hypervisor, identity provider, backup target, or password-management setup. Then test it occasionally by pretending the normal control plane is unavailable. You do not need to destroy anything; simply begin from the notebook and verify that every referenced path still exists.
The point is not to predict every failure. It is to preserve the first few moves. Once networking, identity, and documentation are reachable again, ordinary troubleshooting can take over. Until then, the recovery notebook may be the most valuable backup in the room.