May 13/14 incident (resolved)

by Daniel Heitz | Last updated on May 14, 2020

At 07.54am UTC on May 13 during routine maintenance we discovered an issue in our main data storage.

Because customer data is of critical importance to us, we paused the automatic GitHub repository daily backup until we could resolve the issue. We wanted to ensure no further problem would be created by writing data before we had a resolution.

We have been updating the incident status on our Twitter channel.

The issue has been fixed and BackHub service is online. We are monitoring things closely and will soon complete our testing and analysis.

We anticipate resuming GitHub backups normally as of 12.00am PDT on May 15. Once service is fully restored, everything should appear as expected.

How does this affect you? You can access backups as well as restore any backups prior to May 13, as well as after we resume normal service. In other words, this incident only affected the backup process for the two days May 13-14. After resolving the issue, we can confirm this is definitely the case. 

We apologize for any inconvenience and will provide a post-incident report on our blog. 

Profile picture of <a class=Daniel Heitz">