Service Restored December 01, 2014 21:44
Hello everyone! We’ve got a quick announcement for you on today’s downtime.
Our DNS provider, DNSimple, experienced a massive denial-of-service attack today, which caused their site and services to go down for several hours. As a result, while our site and apps were operational, DNS services failed and both our site and app admins were inaccessible to visitors today. If you’re located outside of the US, it probably took longer for this to resolve.
So what does this mean? First, our apps were still operational. No worries there :) . For example, if you had Product Customizer options created, Happy Ending “Thank you” page changes made, or BlogFeeder feeds set up, they continued to function as expected. So to be perfectly clear: our services were not interrupted, and any orders placed with Product Customizer will have your options correctly applied. The actual code behind our apps is hosted with a different service, so your Cyber Monday sales were unaffected.
However, you couldn’t access our site. DNS (Domain Name Servers) are like internet phone books – they route visitors to your website IP address. Without them, while the site is still operational, the visitor’s browser doesn’t know your address and can’t find you. This meant that, while our apps functioned, you couldn’t change or add Product Customizer options, add new feeds, or change your “Thank you” page, as our site (which hosts the app control panel) wasn’t available to visitors.
The bigger issue today is that emails have probably been lost. As email is tied to DNS hosting, we’re sure the DNS monster must have gobbled up some of our emails while our site couldn’t be found. If you submitted a support request today and it hasn’t been answered by tonight, please resend it. We’re making sure we’re all hands on deck for any support tickets that slipped through this issue, and we’ll be answering them as swiftly as possible.
So what do you need to do moving forward? Nothing. Our apps are still running smoothly and continued to do so while our site was down. The only thing you may need to do is resend an email, or continue editing and making changes to our apps as you were before our site outage.
Moving forward, we’re investigating strategies to deal with DNS issues in the future in order to mitigate the risk that our site will be down for an extended period of time. We’ve got a few ideas that our team will be working on over the next week to implement this.
If you have any questions at all, please let us know in the comments or shoot us an email! We’d be happy to help out, and we thank you all for your patience today.