Dealing With Downtime

Big Cartel has been a frequent target of so-called Denial of Service (DoS for short) attacks for years. The last week has been no different. What has sadly been different is how much these attacks have impacted your business. I want to take a moment to talk about what's changed and what we're doing about it.

Unlike attacks on other businesses you may have heard about, we have no reason to believe that the perpetrators were after, or were able to obtain, any sensitive account information. In contrast, the goal of these kinds of attacks is simply to overwhelm our servers so that they are unable to respond to legitimate customer requests. Why we end up in the crosshairs of these attacks is a bit of a mystery, but we wear it as a badge of honor. Maybe one of you started up a hot new clothing brand and one of your competitors orders a contract hit on your store. Or perhaps you posted some art that offended an extremist group. Whatever the case, we're proud of the stores that put their trust in us and we do our best to have their backs.

Over the years, we've gotten rather good at dealing with these kinds of attacks and up until very recently our uptime has reflected that. These latest attacks have been some of the largest and most persistent we've seen.

As we've adapted, so have the attackers, changing methods each time we succeed in blocking them. Traditionally these attacks have been easy to identify - they're mostly meaningless traffic intended to "clog the tubes" and saturate our network. The most recent ones have complemented this garbage traffic with a huge amount of "bots" masquerading as real users that are much more challenging to filter. Unfortunately we were not as prepared for these kinds of attacks and it took longer than it should have to identify and block the bots. At the same time, one of our data center providers had issues dialing in a new set of defensive measures. They've been incredibly helpful in mitigating past attacks, but this time some legitimate traffic was blocked as well.

Having to explain downtime sucks. I'd much rather you be reading about more effective ways to promote your business right now. As a result of these issues we've already made some important behind the scenes changes that will help us handle attacks more gracefully. As an added bonus, these changes also put us in a great position to speed up your store, particularly for customers who live outside the United States. We're working with some of the biggest names in the industry to build a super reliable platform and we won't stop until these kinds of outages are a thing of the past.

Thanks for hanging in there with us.

Lee Jensen is Big Cartel's Engineering Director. He was one of our first employees, and focuses on making life easier for our artists and shop owners.