Brightwurks Domain Failure

Today we lost the Brightwurks.com domain for about 10 hours. It expired early this morning and somehow we were never notified. By the time we got it renewed the nameservers were switched and it takes a while to get them switched back. This caused chaos we are very unhappy to report.

Domains are a single point of failure. If the domain goes down there isn't a backup or way to minimize damage in most cases. We made a terrible mistake and have moved all our company domains to a place where this will never happen again. It shouldn't have happened today and we're very sorry for the inconvenience.

Help Scout 

We lost any incoming emails between 12pm-10pm EST today. Any customer who's message was lost received a bounce message informing them that their email was not delivered. Emails hit Help Scout and immediately bounced because our mail servers use brightwurks.com in the host name.

Feed My Inbox

Realtime and 6-hour emails that should have been delivered during the 12pm-10pm EST window were not delivered. Any entries posted during that window were not emailed out properly and thus were lost. We are migrating over some stability improvements we already have in Help Scout that will prevent data loss in this case if it were to ever happen again (which it won't). 

If you have any questions or want further explanation from us, please email help (at) helpscout.net or help (at) feedmyinbox.com. We promise it won't bounce!

Status blog first post

It's always best to plan for the worst before it happens, so that's why we are launching the Brightwurks Status site. This site is hosted with Posterous, so if any of our websites ever go down, it shouldn't effect this site at all.

Here we'll announce scheduled maintenance, address outages/hosting issues and constantly monitor the uptime of our web apps. We're hoping that the action here is very seldom, but it's nice having it around "just in case".