Raid Failures in Server Rooms.

Raid Failures in Server Rooms.

We were recently asked by a major media service provider to comment as to RAID failures in their server room.   This subsequent to IGFSS deployment and the release of fire suppressant gas.

Server Room Environment.

The general circumstances were that their servers were located in a dedicated server room with all environmental systems correctly installed and functional. There was no evidence of ambient temperature events or gradients outside of acceptable limits. Inspection of each rack, server, communication, power and control equipment showed no sign of local overheating, burning or fire. We noted however that server racks were operated without the front panel covers.

Failed RAID configured Hard Disk Drives.

We were presented with and examined a sample of hard disk drives from the failed RAID 6 configured servers. On inspection each of the hard disk drives was fully functional and fit for purpose. We concluded that the RAID hard disk drives had not suffered permanent  electronic, mechanical component or platter damage subsequent to IGFSS deployment. This was as expected,  IGFSS comprise inert gas that presents no danger to electronics , hardware or storage media.

Rapid Suppressant Gas Release into Server Rooms.

IGFSS  are designed to extinguish a fire by quickly flooding equipment rooms with inert gas in order to rapidly reduce oxygen levels.  The gas is stored under high-pressure in cylinders for rapid deployment to fire affected areas. Cylinder pressures are circa 2,900 PSI, and discharge of circa 1,000 PSI. We noted that the system  exhaust nozzles were reported to be pointed towards the racks and that covers were not fitted. We concluded that the rapid release of the gas directly on to or near the hard disk drives had resulted in high levels of acoustic noise or transient air displacement pressure variations that may have disrupted the read/write functions of the RAID configured server hard disk drives.

Prior Research and Testing.

General available research into the effects of suppressant Gas release events is scarce and what there is has been under controlled conditions. What we do know is that real world reported events are relatively rare but have resulted in simultaneous multiple server failures. Trials conducted in the US have shown that the worst disruption to RAID Server hard disk drive operation was evident during the first 60 seconds of the IGFSS being deployed. The pressure variations or noise reaching the HDDs causing individual hard disk drive read/write heads to go “off track” (generally HDDs can tolerate less than 1/1,000,000 of an inch offset from the centre of the data track, anything in excess will disrupt read/ write events).

Hard Drives and Server Room Environments

As far as the wider implications as to which models of HDD are more or less affected by IGFSS being deployed is almost imponderable. Each enterprise class hard disk drive make and model having a unique set of sensitivities and design specifics.

How to avoid RAID Server Failure.

Power Down.

Most IGFSS events will be preceded by an alarm. If possible and without risk to the person, safely and gracefully shut down the RAID Server systems to militate against potential data loss and damage that may result from a rapid gas release event.

Fit Smaller Gas Exhaust Nozzles

Smaller nozzles generate less noise and have less impact on HDDs., where possible fit the small variant of these nozzles. Although there is little evidence to support this:- pneumatic sirens may affect HDD operation  so install these outside the actual server rooms.

Ensure RACK Front Covers are fitted

Fit covers to the front of the racks as these will act as noise baffles and shield the impact of direct gas release.

Oh and don’t smoke in the server room !

pip pip