Green Check Marks and a Restore You Have Never Run. What the Rules Actually Ask For
Backup software reports success on the copy, not the recovery. What the applicable rules require, and what an experienced hand checks before believing any of it.
| Published | Section | Technology | |
|---|---|---|---|
| Author | Malcolm Greenhalgh | Reading time | 5 min |
The first thing I teach anyone new to this is that a backup job reporting success has told you one thing only: that data left one place and arrived somewhere else in a form the software recognized. It has not told you the file opens. It has not told you the database is transactionally consistent, that the encryption key is somewhere you can reach it, or that the machine you would restore onto still exists. Those are separate questions with separate answers, and every one of them has been the reason somebody lost a week.
What an apprentice learns in the first year, and what it leaves out
Year one is mechanical, and rightly so. You learn where the jobs are defined, how to read a log past the summary line, what a warning means as against an error, and how to spot the job that has been silently skipping a mounted volume since a reboot in March. You learn to check capacity before it bites, and to notice when a nightly window that used to close at two in the morning is now running until seven. All of that is real work. What it does not teach is whether the thing you are protecting can be brought back, because nothing in the daily routine ever asks that question out loud.
The gap shows up the first time somebody tries a recovery under pressure. The copy is fine. The credentials to the backup console were stored in the password manager that lived on the server now being restored. The virtual machine host is a version behind and will not import the image. The license key for the accounting package was emailed to a person who left in 2019. None of these are backup failures in any sense the software would report, and all of them are recovery failures, which is the only category the person paying you cares about.
The rules that already apply, whether or not anyone told you
Most small operations assume backup is a matter of good practice rather than obligation, and that is usually wrong. If you handle consumer financial information, the Federal Trade Commission's safeguards requirements expect a written security program with periodic testing of the controls in it, and recovery is a control. Medical practices, and their vendors, work under contingency planning obligations that include data backup, a disaster recovery plan and a procedure for testing and revising both. Anyone taking card payments inherits contractual requirements from the card brands. State breach notification laws add a clock: you cannot notify accurately about data you cannot reconstruct.
The National Institute of Standards and Technology is the body responsible for the federal information security standards that most of this vocabulary descends from, which is why terms like recovery time objective and recovery point objective turn up in contracts written by people who have never restored anything. Those two numbers deserve to be argued about before they are written down. Recovery point is how much work you are willing to redo, measured in hours. Recovery time is how long the business can be dark. Set them honestly and the rest of the design follows almost automatically.
Three copies, two kinds of media, one somewhere else, and one nobody can edit
The old rule of three copies on two types of media with one held off site survives because it addresses three different failure modes at once: the file goes bad, the device goes bad, the building goes bad. What it did not anticipate is an intruder with your administrator credentials who deletes the backups first and the production data second. That is the ordinary shape of a ransomware event now, and it is why the meaningful addition to the rule is a copy that cannot be altered or deleted within its retention window, by anyone, including you. Vendors sell this as immutability or object lock, and it is worth the line item.
Retention is the other half, and it is where regulation and practicality meet. Tax records, employment files and patient records all carry their own retention periods, and a backup scheme that keeps fourteen days of nightlies satisfies none of them. Work out what you are legally required to keep, keep monthly or annual copies to match, and keep them separately from the operational backups that exist to solve a Tuesday morning problem. The two purposes are frequently confused, and the confusion is expensive in both directions.
The restore drill, scheduled and priced like any other job
A test that costs nothing and takes ten minutes is not a test. What I schedule for clients is a quarterly restore of something that matters, onto hardware that is not the original, performed by whoever would actually be on shift rather than by the person who built the system. Time it from the moment the decision is made, not from the moment the software starts. Write down what was needed and where it came from: the console password, the encryption key, the license file, the network settings, the person who knew which database goes back first. That written record is the deliverable, more than the restored file is.
Annually, go further and restore something whole. A full server, a full workstation image, a database with the application in front of it, brought up in isolation and logged into by someone who uses it daily and will notice if the last three days are missing. Budget the hours the way you would budget a service call, because that is what it is. The first one of these usually runs long and turns up two or three surprises, which is the entire point, and every one afterward runs shorter because the surprises were fixed rather than discovered at the worst possible hour.
What only comes with time
Experience mostly shows up as sequence and dependency. You learn that domain controllers come back before file servers, that a virtual host with no running domain controller may not authenticate you into its own management console, and that the licensing server frequently has to be alive before the application it licenses will start. You learn to keep credentials and keys out of band, printed or in a separate vault held by a second person, because a recovery plan stored only inside the system being recovered is not a plan. You learn to distrust anything green until you have watched it come back.
Hire for that. Ask a candidate what the last thing they personally restored was, when, and what went wrong during it, and the answer will separate people who administer backups from people who have recovered from them. Both matter. Only one of them will be any use on the morning it counts.
