A man appears to have deleted his entire company with one mistaken piece of code. By accidentally telling his computer to delete everything in his servers, hosting provider Marco Marsala has seemingly removed all trace of his company and the websites that he looks after for his customers.
Marsala wrote on Server Fault - a forum for server experts - that he was now stuck after having accidentally run destructive code on his own computers. But far from advising them how to fix it, most experts informed him that he had just accidentally deleted the data of his company and its clients, and in so doing had probably destroyed his entire company with just one line of code.
The problem command was “rm -rf”: a basic piece of code that will delete everything it is told to. The “rm” tells the computer to remove; the r deletes everything within a given directory; and the f stands for “force”, telling the computer to ignore the usual warnings that come when deleting files. Together, the code deleted everything on the computer, including Masarla's customers' websites, he wrote.
Masarla runs a web hosting company, which looks after the servers and internet connections on which the files for websites are stored. That piece of code is so famously destructive that it has become a joke within some computing circles. Normally, that code would wipe out all of the specific parts of the computer that it was pointed at. But because of an error in the way it was written, the code didn't actually specify anywhere - and so removed everything on the computer.
“I run a small hosting provider with more or less 1 535 customers and I use Ansible to automate some operations to be run on all servers, “ wrote Marsala. “Last night I accidentally ran, on all servers, a Bash script with a rm -rf {foo}/{bar} with those variables undefined due to a bug in the code above this line.”
Marsala confirmed that the code had even deleted all of the backups that he had taken in case of catastrophe. Because the drives that were backing up the computers were mounted to it, the computer managed to wipe all of those too.
“All servers got deleted and the offsite backups too, because the remote storage was mounted just before by the same script (that is a backup maintenance script).”
Most users agreed that it was unlikely that Marsala would be able to recover any of the data. And as a result his company was almost certainly not going to recover, either.
“I feel sorry to say that your company is now essentially dead,” wrote a user called Sven. “You might have an extremely slim chance to recover from this if you turn off everything right now and hand your disks over to a reputable data recovery company.
“This will be extremely expensive and still extremely unlikely to really rescue you, and it will take a lot of time.”
Others agreed that perhaps Marsala was on the wrong forum.
“You're going out of business, “ wrote Michael Hampton. “You don't need technical advice, you need to call your lawyer. “
Many of the responses to Marsala's problem weren't especially helpful - pointing out that he could have taken steps to stop it happening before it did.
“Well, you should have been thinking about how to protect your customers'' data before nuking them,” wrote one person calling himself Massimo. “I won''t even begin enumerating how many errors are simultaneously required in order to be able to completely erase all your servers and all your backups in a single strike.
This is not bad luck: it's astonishingly bad design reinforced by complete carelessness.”
Marsala’s problem is far from the first time that someone has accidentally destroyed their own system by missing a mistake. Indeed, responses to his post mostly referenced a similar thread posted two years ago, with the headline “Monday morning mistake”.
That error saw someone accidentally lose access to their entire server, after they didn’t notice a stray space in the code.
The Independent