Squarespace System Status

Success.

Within the past 30 minutes, we successfully transferred over to our street-level generator, ran for 20 minutes, then failed back to our building generator. Our building generator, street generator, and battery backups worked properly during this test.

We are confident to report that we now have redundant on-site power while we wait for our building to formally reconnect to the Manhattan power grid.

We have received word that the redundant street side generator has been connected, and a load transfer test will be performed tonight. Wednesday, November 7, 2012 from 8:00pm ET to 12:00am ET.  

If this load transfer is successful, we will have 2 confirmed working power sources on site, and will be able to install a redundant fuel filter in our main generator tomorrow, further bolstering our resiliency.

We do not anticipate any impact from the Nor’easter blizzard/storm that is currently descending upon New York (though we certainly aren’t thrilled by it!)

Roll up generator for redundant power supply is in NY at facility. Electricians are almost done with wiring. No significant progress on restoring the street power for the building due to water damage.

We are still operating off of generator power and are awaiting the arrival of the redundant truck-mounted generator, which we expect to come this evening.  We will update again when redundant power has been restored.

Squarespace office infrastructure in Soho is now completely restored, and we are ready to develop again and get rolling on Monday.  We will continue to provide updates related to the Peer1 facility as more news emerges.  The situation remains stable at this time.

We continue to operate sustainably via the building generators. Power has not been completely restored to the Wall St area, and we do not have an up to date report on the building’s mechanical state. We have received word that a street side generator will arrive by Monday to restore truly redundant power while we wait for the building to be repaired. We have systems staff on site at the datacenter 24/7 to monitor the situation until we are back to normal. Power to our office has been restored, and we are bringing development systems back online now.

We continue to successfully run via the connected pump.  

We do not have additional infrastructure updates at this time, but major personnel support from Peer1 are on site.  ConEdison is stating that power will be restored to lower Manhattan tonight, but we do not know how that will help with our situation immediately, as our building infrastructure is still damaged, but is obviously another step in the right direction.

We have confirmed that the fuel pump is now working. We can now operate off of fuel indefinitely and there are continuous deliveries scheduled.

Additional issues remain, the largest of which being that our generator will need to be taken offline at some point to clean the fuel filter. We can avoid that downtime if Peer1 can provide a street level generator. If this generator failure does occur, we expect the level of downtime to be on the order of an hour, instead of the multi-day outage we were facing at the onset of this crisis.

We remain in a difficult situation, but the most difficult challenge we were presented with is now behind us. As of this writing, Squarespace has miraculously avoided all downtime related to Sandy, but we still have many challenges ahead with respect to getting back to anything normal.