The perils of `rm -rf`

I came across this question on Stack Exchange about how to recover from an accidental rm -rf /*.

The questioner says he runs a small web-hosting service, and accidentally deleted all the files on his servers, and all the files on his backup drives. He asks how to recover. (The answer: it’s difficult.)

This made me smile, because I, like everyone who programs for a living, have an rm -rf story too.

I was running a shell script at the command line. The script had to be run as root, which means it had permission to do almost anything. In the shell script I made a very similar error to the one the questioner made. I told the computer to rm -rf ${FOO}/*, and I had accidentally set FOO to the empty string, so the expression evaluated to rm -rf /*. I pulled the plug when I started getting strange errors about directories and files that even as root I was not allowed to remove.

Fortunately for me, I was running on a virtual machine. There was no important data on the machine – we were using it for processing and storing the results elsewhere. We could just spin up a new copy of the virtual machine and trash the old one. But I was sheepish for days. Until I started hearing everyone else’s stories about rm -rf.

  • Toy Story 2 accidentally deleted the master copy of the movie.
  • Back in 1986, someone at Manchester accidentally deleted most of their VAX server, which was a shared machine with many users on it. Since rm works alphabetically, the deletions began with /bin, so they did not have access to common tools like ls. They got around this by using the built-in shell command echo *, which at least allowed them to examine what was still there and what was gone. They couldn’t log in remotely, because all the login information is stored under /etc, which is early in the alphabet. Fortunately, open login sessions were still there, and one of those sessions was logged in as root, so they could write some VAX assembly and sort themselves out.
  • In 2011, a game studio named Bumblebee accidentally included an extra space in their install script. Instead of rm -rf /usr/lib/nvidia-current/xorg/xorg, they included the line rm -rf /usr /lib/nvidia-current/xorg/xorg, which happily deleted the entire user directory hierarchy. As a side note – test your changes. If they had run this particular code before committing it, they would have immediately detected the problem. You can read the commit and all its comments – it is apparently the most commented upon commit on github.
  • In 2015, a game distribution platform named Steam introduced a bug into their install script. They had the line: rm -rf "$STEAMROOT/"*. If $STEAMROOT was not set, this would delete /*. And if the user removed a configuration file for Steam, then $STEAMROOT would not be set.

How to prevent this problem

  1. Consider installing safe-rm on your system, and aliasing rm to safe-rm. safe-rm is a wrapper around rm, but it keeps a blacklist of directory entries that are not allowed to be removed. By default, it has entries like /bin and /usr, and you can add additional items in a config file. safe-rm is available from the standard package managers (yum, apt, brew).

  2. Be super careful with using rm -rf in shell scripts – especially with variables. Consider what will happen if the variable evaluates to nothing. Better yet, use parameter substitution, like ${variable_name:?ERROR_MSG} in Bash, so that if the variable is not set, the script will print ERROR_MSG and abort.

  3. Test all your changes before you make them public. This won’t protect you, necessarily, but it might protect the public.

  4. If you do rm -rf something – don’t panic. Take a breath, walk away, calm down, and think before you do anything.

Leave a Reply

Your email address will not be published. Required fields are marked *