Hetzner Cloud – Data loss incident (redux)
Dear Customer
We regret to inform you that there has been a loss of snapshots from your following Storage Box that were created in the period from 15 May 2023 11:00 CEST and 12 July 2023 13:00 CEST. Snapshots created before or after this period and the contents of your Storage Box are not affected!
On 25 July 2023, we discovered that due to a software error, snapshots created for Storage Boxes were partially deleted. This affected both the snapshots created manually (via web interface or Robot Web Service) and those created automatically as part of snapshot plans. Unfortunately, we were unable to restore the snapshots deleted during the above-mentioned period.
The protection of your data is important to us. We take this matter very seriously and have already rectified the problem and immediately reported the incident to the Bavarian State Office for Data Protection Supervision.
We apologise for this inconvenience and ask for your understanding.
If you have any questions, please do not hesitate to contact us. Please use the contact form in your administration interface under the menu item “Support”.
Kind regards
Hetzner Online
----
* Previous incident, 15 months ago: https://news.ycombinator.com/item?id=31015840 Data loss is terrifying. I am building a tool to help in data transfers. I always assume my data is more secure in S3 than my on prem NAS. I also assume that 0.00001% accounts for anything from one file to an entire data center worth of bytes. This is bad, but storage boxes are already a backup target and the content of the backup target (actual files stored in in box) are not affected... "only" snapshots. Almost a month of snapshots. Yikes. you mean almost two months… Apparently only if you changed the config and also only snapshots that are older than 14 days. Still Bad but less impact than described in their first email. (Asked support) Eek! Even worse! And now I know I can't count!