Website Down Monday

Hi

I note that our website was down on Monday night, our monitoring software has notified us that it received a Http 200 OK result yesterday after 26 hours, can you confirm what the issue was?

I look forward to hearing from you.

Many thanks

Jim

Hi Jim,

Our server clusters in both the US and the UK came under a sophisticated distributed denial of service (DDoS) attack late on Monday afternoon. The initial attack took both centres offline for an hour, before mitigation could be put in place by the network security team at our upstream provider (Rackspace).

Approximately an hour later, the attackers changed both their approach and the servers being targeted, resulting in a further outage of approximately one hour.

At that point, a new DDoS mitigation solution was put in place and, while the attack continued for many hours afterwards, the sites were fully available from then on.

The mitigation solution was technically “on” for 24 hours from that point, and then was set to “auto trigger” when it was clear that the attack had stopped.

While the mitigation solution was on, some legitimate TCP requests were being dropped and forced to retry. Had you visited the site during that period, which I presume you did, you would have seen that the site was fully responsive to end users, perhaps sometimes requiring the browser to automatically refresh to see pages.

This mitigation approach meant that most uptime checkers failed to properly detect the status of the sites, so they were incorrectly showing them as unavailable. They had particular trouble negotiating SSL connections. However, shoppers using regular browsers did not.

You can see from our own third party uptime checking service that there was a total of just over 2 hours of downtime: NitroSell Platform Uptime

Obviously, while we do make extensive efforts to protect the sites, it is impossible to fully protect against denial of service attacks. We did get the sites back up as quickly as we could, and we are prepared for these types of attacks in future.

Note that at no point was site security compromised, nor was any data breached; this was purely a denial of service attack.

Regards,
Donogh

Hi Donogh

Thank you for your reply, it sounds like it was a nightmare! Do you know where the attacks originated from?

Regards

Jim

Hi Jim,

It wasn’t the most fun we’ve had of a Monday! (And it was a bank holiday in Ireland, too.)

So far, we are still investigating. It’s quite likely that the attackers were targeting the entire infrastructure to try to take down just one site.

Unfortunately, with these types of attacks it’s difficult to figure out the source – the addresses from which the attack originated were spoofed.

We are currently investigating and will let you know if we find anything useful.

We’re also looking into reporting it to the authorities, particularly the FBI, via their Internet Crime Complaint Center (IC3), once we have gathered enough data. That’s probably our best chance of getting some satisfaction or justice.

Regards,
Donogh

I do not know if it’s related but since the end of September, google analytics report us that we receive 23% of our traffic from robots in India (we sell only in north america), i had to exclude India from our statistics because it distorted everything (like the bounce rate at 100%) . I do not know if this attack is still going on.

Regards

Hi Damien,

I’m afraid that’s unrelated. The DDOS traffic won’t even show up in Google Analytics because the attackers weren’t actually getting to the point of even requesting specific pages.

If you can give us more info about the Indian bots, we can look into blocking them. Best to do that by ticket please.

By the way, the attack has stopped already.

Regards,
Donogh

Not sure if anyone else received a message like this, but at the time the DDoS attack I received this message thru the website.

Full Name : Hanzo Tanaka
Company Name :
E-mail Address : KuroiRyuu@tutanota.com

Query : We are a Hacker org called Kuroi Ryuu Inc
We are going to be shutting down your website/services again soon
We understand this can be frustrating and hurt the business reputation which might not be recoverable
So to avoid all of this we make an offer of 10XMR, this equates to about 950eur/1,100usd, to be sent to the following monero wallet address:

45tgJ9NbfWZKvXnuxheKwLD1VoyMisL3uRpMnB1Ah7bMcL9ATqF3CNoifRxm5jmKo1Jzm8hdYXcAjCtDfXyJszLY1KoJ1VA

We will immediately cease everything on your servers and you will not hear from us again. Do not think you are being specifically targeted.
We are targeting other businesses as well. We are only here for Monero.

To verify it was you that made the payment to us. Send us a transaction id or screenshot.(note: THE QUICKEST WAY IS TO
1)use wallet.btc.com/#/setup to create online BTC wallet
2)use https://buybitcoin.shapeshift.io/ or https://www.xcoins.com/ to Buy 0.17 bitcoin with credit card/debit card and send to your BTC wallet address
3)Use https://www.morphtoken.com/ to convert Bitcoin to Monero and send to our Monero address provided above

Use this website How to buy Monero if you are not familiar with Monero
and how to send monero or you can use google.

The longer you wait; the more your business will take financial loss
Respond back to us on KuroiRyuu@tutanota.com

-Kuroi Ryuu Inc

I’ve been a Nitrosell customer for about 12 years now and this is only the second outage I’ve ever experienced so kudos to you guys first off. That said… how do we get information if this happens again? The portal was down, the forum was down, my website was down… I tried tweeting to nitrosell but didn’t get a response (I assumed everyone was busy working on the issue).

Hi Molly,

Thanks for the feedback. Our own web site, using either live chat or the contact form, or phoning us are the best options in that case.

Particularly, when phoning, if you spot an outage, it’s best to use the emergency option.

Regards,
Donogh

I guess I didn’t check nitrosell.com but portal.nitrosell.com was down so I couldn’t get any contact info from there. I don’t have you guys on speed dial :wink:

Sorry about that Molly! Yes, the main NitroSell web site was up. We do have a (slightly) catchy freephone number you could save: 888 9 06 06 39 :slight_smile:

1 Like