Most of us host part or maybe even all of our infrastructure at hosting providers. They provide you with floor space, rack space, or in cloud environments with platforms and software for you to use. As with all of these solutions there are pros and cons to having your hardware hosted. In cloud environments the hardware and often software typically belongs to the provider and only the data belongs to you. What could go wrong?
As security professionals we get to discuss the risks of these kinds of arrangements and most of us will raise the risk of the provider going south or the data being unavailable for other reasons. The answer we often get is along the lines of “oh that never happens and we have backups”. Unfortunately that doesn’t always help and losing data isn’t the only issue as has been aptly demonstrated this week when a number of datacentres Belgium and the Netherlands closed up shop.
In a nutshell the provider was declared bankrupt, the doors closed and connections were cut. As the articles state customers were denied access to their servers whilst the bankruptcy processes were established. In a number of cases connectivity to servers was cut, denying access to the data. So what risks are there when a hosting provider goes bust?
Denied access to physical servers – In many hosting situations the line between who owns what is difficult and often physical access will be denied until ownership can be demonstrated. In the mean time you may have expensive equipment sitting in a datacentre that you can no longer access.
Denied access to data (internet) – This can happen a number of ways. The Internet connection may be removed which obviously cuts your and anyone else’s access. Sometimes machines are shut down. Whilst many administrators may decide to keep things running, after all earning some money is better than none, to cut cost supporting services may be reduced and if something breaks it is unlikely to get fixed.
Denied access to data (Local) – You may decide to go pick up your data, but getting access may not be that easy. So unless you can retrieve it remotely you may have to kiss that good bye.
Backups – Any backups taken by the hosting provider are unlikely to be accessible. Depending on the systems used to manage backups it may be quite a task to get them. Even if you get the physical tapes (if used) you are unlikely to get the backup catalogue, so retrieving data will be difficult.
Disclosure of data – Physical access usually trumps most of the controls many of us place on our hosted environment and in the cloud we do not have any control. So it is quite likely that you will not be able to deny access of third parties to your data.
Least of all you will be left with the cost of moving operations to an alternate location and as most of use who have been involved with datacentre moves know that is not a trivial task.
It would be mean to just leave it there, so what can be done about this to mitigate the risks?
Denied Access – If denied access to your servers or data a DR environment is probably your best bet. Being able to run up services elsewhere provides processing capabilities whilst lawyers sort out getting physical assets back. However tempting it may be it is probably not a great idea to have the production environment and your DR environment hosted by the same organisation.
Backups – Make your own. Do not rely on the hosting provider to do all the backups. Alternatively make sure that backups are stored elsewhere, including the catalogue so you can readily identify the data on the tapes, if needed.
Disclosure of data – This is probably the most difficult one and makes you wish that the mission impossible slogan (this message will self-destruct in …) is an actuality. Not many of us are in the habit of full disk encryption on servers, but that may be the only way and won’t help in a PAAS or SAAS situation.
Some of you may have been in this situation and others can no doubt learn from your experience so if you are able to I’d love to see your experiences or additional risks and controls I may have missed.
(c) SANS Internet Storm Center. http://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License.