[M5Hosting] Emergency Maintenance and Initial Network/Power Event Overview

Michael J McCafferty mike at m5computersecurity.com
Wed Mar 18 21:00:08 PDT 2009


Dear M5Hosting Customer,
	As you may be aware, your server was rebooted today due to a  
momentary power loss at the data center facility (American Internet  
Services) from which M5Hosting provides your service. The root cause  
was a failed power system component during a scheduled generator test.
	After the power was restored, M5Hosting also had some network issues  
on our "PremiumNet" due to some redundant network components behaving  
in an unexpected manner.
	The failed power system components have been replaced and power will  
be taken off generator and put back on the "street" power tonight  
during an emergency maintenance window declared by the data center  
facility. This maintenance window will begin at 8:30pm US/Pacific  
Time. While the window has been declared to be 2hrs, the actual  
cut-over will be quite short. The window will allow for testing. This  
work is not expected to impact service at all.
	I have pasted the most recent announcement from the data center  
facility below. We will also forward any subsequent announcements from  
them regarding this incident.
	While the network and facility are fully functional and stable at  
this time, there is some work to do return all systems back to full  
redundancy and normal operating status. The maintenance discussed  
below will be the first step. This will return the power system to  
it's normal operating status. Our network work will take place during  
a late night maintenance window to be announced later and scheduled to  
mitigate any risks of an impact to service.

	There will be more detailed follow up explanation of the events today.

	If you are still experiencing any issues with your server or other  
services which we provide for you at this time, please contact us at  
877-344-4678 or submit a support ticket by email to  
support(at)m5hosting.com, so we take care of your needs immediately.

Sincerely,
Mike

The most recent Data Center facility announcement from AIS:

[quote]
To Our Valued Customers,

Following this morning's facility issue at the 9725 Scranton Data  
Center, AIS Facilities and Engineering teams are announcing a 2 hour  
emergency maintenance window which will begin at 17:15 and which will  
conclude at or before 20:15.

During this window, main power for the facility will be returned from  
generator to normal operating mode.  Throughout the day, engineering  
and mechanical teams have taken every precaution and performed  
considerable diagnostics to ensure that all systems are once again  
operating at 100% effectiveness.   Even following these precautions,  
all maintenance windows involving critical power systems still carry  
the risk of causing of a power event during the maintenance operation,  
thus, we are bringing this important issue to your attention at this  
time.   While maintaining a cautious posture, we at AIS do not  
anticipate any issues during this window.

During the maintenance window, the facility will remain fully staffed  
by 18 customer service engineers, 3 different mechanical engineering  
contractors, and AIS senior management personnel.

We are in the process of producing a detailed post incident report  
which will be made available during business hours tomorrow, Thursday  
March 19th.

A follow up communication will be sent upon completion of the  
maintenance window.  We apologize for any inconvenience this issue may  
have caused.

AIS Support Services
[/quote]

-- 
************************************************************
Michael J. McCafferty
Principal, Engineer
M5 Hosting
877-344-HOST x501 Voice
http://www.m5hosting.com
************************************************************



More information about the M5Hosting mailing list