News & Blog
Anger as Websites Deleted by 123-Reg in Error
By Francis West on 25th April 2016
One of the clearest examples of why it is important to have an effective backup system and a disaster recovery plan in place came to light this week when it was reported that web hosting company 123-reg deleted an unspecified number of websites in error.
Judging by reports so far it seems fair to assume that the majority of deletions are likely to have been customers’ current websites.
How Could This Happen?
It is common practice for web hosts to store multiple websites on one server with each customer effectively renting an amount of space on what is known as a virtual private server (VPS).
In this incident, 123-reg is reported as saying that it used software with automated scripts to ‘clean up’ the servers, but that a coding error in that software resulted in the deletion of multiple customer websites. The software was intended to detect server activity but the automatic deletions were triggered when the script wrongly showed several VPSs as running no servers.
It is thought therefore that the lack of a human approval check in this automated process was a key reason why website deletions occurred.
123-reg is part of Host Europe Group (HEG), which is reported to have described itself as Europe’s largest privately owned hosting company. 123-reg has 800,000 customers in the UK where it hosts 1.7m sites.
According to 123-reg the customers affected by their “VPS Issues” were those whose websites were hosted on 67 of its 115,000 servers across Europe. The fault occurred on the morning of Saturday 16th April and an email was sent to customers explaining what had happened the following day.
Back Up If Paid For Or Backed Up Yourself
Reports indicate that the VPS service in this case was “unmanaged” and 123-reg did not have back ups of all the affected customer websites affected, unless those customers had also specifically purchased back up.
Customers who hadn’t purchased the back up aspect had therefore been responsible for backing up their websites themselves.
However. 123-reg is now reported to be using a data recovery specialist to "manage the process of restoration" although this will be on a on a case-by-case basis and therefore is likely to take considerably longer to resolve than those customers with backups who were able to be back online the next day.
Customer Reaction
Reaction was predictably angry and swift as customers took to social media like Twitter to voice their fury and frustration about how it could possibly happen, the levels of communication that they had received from the company about it, plus the impact that it would have on their businesses.
Lost sales (goods, services and tickets), loss of potential new business and funding, as well as potential loss of the business itself were all concerns raised by customers.
What Does This Mean For Your Business?
This incident indicates how important it is for your business to make sure that you and / or the company that hosts your website has a secure backup of your website as well as other critical business data.
Making this one of the key selection criteria for your host / hosting service could therefore save you from some serious problems in the future.
The incident also highlights how important the host selection process is in the first place and to carefully choose a host whose services and capacity closely match the specific requirements and scale of your business both now and in the foreseeable future.
Another important lesson to be learned here is that as part of IT governance in today’s business environment, and as part of your responsibility to your stakeholders it is necessary to have a disaster recovery process in place.
Anger as Websites Deleted by 123-Reg in Error
One of the clearest examples of why it is important to have an effective backup system and a disaster recovery plan in place came to light this week when it was reported that web hosting company 123-reg deleted an unspecified number of websites in error.
Judging by reports so far it seems fair to assume that the majority of deletions are likely to have been customers’ current websites.
How Could This Happen?
It is common practice for web hosts to store multiple websites on one server with each customer effectively renting an amount of space on what is known as a virtual private server (VPS).
In this incident, 123-reg is reported as saying that it used software with automated scripts to ‘clean up’ the servers, but that a coding error in that software resulted in the deletion of multiple customer websites. The software was intended to detect server activity but the automatic deletions were triggered when the script wrongly showed several VPSs as running no servers.
It is thought therefore that the lack of a human approval check in this automated process was a key reason why website deletions occurred.
123-reg is part of Host Europe Group (HEG), which is reported to have described itself as Europe’s largest privately owned hosting company. 123-reg has 800,000 customers in the UK where it hosts 1.7m sites.
According to 123-reg the customers affected by their “VPS Issues” were those whose websites were hosted on 67 of its 115,000 servers across Europe. The fault occurred on the morning of Saturday 16th April and an email was sent to customers explaining what had happened the following day.
Back Up If Paid For Or Backed Up Yourself
Reports indicate that the VPS service in this case was “unmanaged” and 123-reg did not have back ups of all the affected customer websites affected, unless those customers had also specifically purchased back up.
Customers who hadn’t purchased the back up aspect had therefore been responsible for backing up their websites themselves.
However. 123-reg is now reported to be using a data recovery specialist to "manage the process of restoration" although this will be on a on a case-by-case basis and therefore is likely to take considerably longer to resolve than those customers with backups who were able to be back online the next day.
Customer Reaction
Reaction was predictably angry and swift as customers took to social media like Twitter to voice their fury and frustration about how it could possibly happen, the levels of communication that they had received from the company about it, plus the impact that it would have on their businesses.
Lost sales (goods, services and tickets), loss of potential new business and funding, as well as potential loss of the business itself were all concerns raised by customers.
What Does This Mean For Your Business?
This incident indicates how important it is for your business to make sure that you and / or the company that hosts your website has a secure backup of your website as well as other critical business data.
Making this one of the key selection criteria for your host / hosting service could therefore save you from some serious problems in the future.
The incident also highlights how important the host selection process is in the first place and to carefully choose a host whose services and capacity closely match the specific requirements and scale of your business both now and in the foreseeable future.
Another important lesson to be learned here is that as part of IT governance in today’s business environment, and as part of your responsibility to your stakeholders it is necessary to have a disaster recovery process in place.
Comments