Backups are Important, Recovery is Essential

On January 31, 2017 Gitlab experienced a massive database outage. They suffered hours of downtime, which they have detailed here. The transparency in that link is breathtaking for a couple reasons. First, any company willing to details their failures and challenges in such public terms gets my kudos. It’s not easy to put yourself on display, warts and all. Second, there are so many self inflicted wounds and unforced errors in the story.