Yesterday was a doozy. A lot of you had a really stressful day, (for those of you who are unaware, we had a real bummer of an outage) and we’re very sorry. While sorry doesn’t quite cut it, we want you to know exactly what happened.
At around 1 o’clock, 2 of our 3 database servers broke down. This immediately sent our engineers and support staff into a complete all-hands-on-deck fight to get things fixed, while trying desperately to keep you informed. Downtime is unfortunately an unavoidable part of doing business online, and we do our very best to make sure it’s a rare occurrence, but it still breaks our collective heart when it does happen.
We know how important that time was to you, missing out on possible sales, wondering what that looks like for your brand. Being artists ourselves, we feel that pain. More important, though, is the trust you have in us. We deal with your livelihood, or at very least, that important extra income you use to make your life a little bit better. That trust is very important to us and we’re deeply sorry for the downtime yesterday. We’ve got some techy answers for those of you who want the dirty details, but for the rest of you, know that with each of these issues, we learn volumes, tighten our ship, and become a closer and more efficient team.
That may sound a bit like a silver lining, but we’d actually like to propose another: you! Even in the midst of a hectic situation, which was obviously very frustrating, you came through with words of encouragement, charm, and even a little bit of humor to keep us going. You’re an amazing community and we’ll gladly take this opportunity to sincerely thank you for your support.
In the span of 2.5 hours, we sent over 400 apology emails asking for patience, 158 tweets pointing to a live status page, and fully recovered 2 naughty database servers (who—we can assure you—went to bed without supper and are grounded the ENTIRE summer).