• Blade enclosure caught fire. A few power supplies melted at the ends. The fire alarms went off in the data center and the whole building was evacuated. That type of failure was unheardof by both us and the manufacturer.

    Luckily for VMWare and our second blade enclosure in a nearby rack, most systems failedover properly and immediately. The rest were back up and we were fully functional in under 90 minutes. We had to 'migrate' a few of the least singed blades from the smoking enclosure to the working one.

    Finding a replacement blade enclosure (still under warranty from manufacturer) took about a week. Apparently they don't keep any extras lying around and had to divert someone else's shipment.