djackson 22568 (9/29/2016)

---

jasona.work (9/29/2016)

---

djackson 22568 (9/29/2016)

---

jasona.work (9/29/2016)

---

The thing to remember about the MTBF numbers is, it's a Mean, not an absolute. So you could have (using Daves' RAID-5 example) 3 SSDs in a RAID-5 array.

Drive A might last 9 years until the server is put in the trash

Drive B might fail after 3 months

Drive C might last 3 years before failing

I know some time back there were people concerned about how drive (HDD) capacity was getting to the point where even with a RAID-6 array and hot spares, the possibility of having a drive fail, then during the rebuild process another drive failing causing the array to no longer have enough drives to be able to be rebuilt. The concern was that in the time it took to rebuild one drive, another would fail (and as the drives were often from the same batch, in theory they might all fail around the same time)

I'm sure it's out there somewhere, it would be interesting to see a comparison of rebuild times of two identical drive arrays (same controller, same capacity) with one using HDDs and one SSDs.

As for the silence of a server room, I've had that happen at a previous job when we lost power. The UPS units were barely able to hang on long enough for me to go the couple hundred feet from my desk to the server room to start shutting things down (yes, I didn't have a UPS on my desktop, neither did anyone else, cheap boss) Even more fun was doing the shutdowns in the dark, no emergency lights in the server room (which was part of a waiting area, with a glass wall between the racks and the waiting room seats.)

What I am trying to say is this: Spinning hard drives (IMO) tend to fail at random times. So if I purchase 20 drives and install them all at once, in the same array of disks, my expectation is that each drive would fail at a different time. Now Steve pointed out that it is possible there is a correlation between batches. Maybe a batch of drives would all fail at the same time. However, when we initially build a server the drives might be from a single batch, but as we replace them they probably are not. Since I have never heard of an entire RAID array failing at once, this has never concerned me. Maybe it should.

Now when it comes to SSDs there is a particular number of writes per cell. When SSDs first came out they were designed to be written to sequentially. So if there were 100 cells, it would start at cell 1, move forward to cell 100, and then start over at the first available cell (deleted data). If you think about how RAID works, then I think it is correct to assume that most, if not all, of the drives in the array would write to the same cell each time. EG as each drive gets to cell 100, the others would as well. Deleted data is likely to be present on the same cells. I am ignoring parity - and since I am not a RAID expert, I might be wrong about how this works. If my assumptions are correct (big if!) then I am forced to conclude that there is a higher chance of multiple failures at once.

Hence my question - we have had what I believe to be sufficient time for servers to have had this occur. Since I am not seeing any news articles about it, and since I haven't seen anyone respond to my question saying it happened to them, I am starting to believe that there is something I am unaware of. Maybe manufacturers changed their write algorithms, maybe it works differently than I believe, maybe there are additional factors I don't know about.

So going back to what you posted - I believe your example is valid for spinning disks. I am under the impression that SSDs are likely to fail in much closer proximity to each other. I would love to hear that I am wrong. Backups are great, but nobody wants to rely on those. We all prefer to not have failures, especially something as bad as this.

Now I see where you were going.

I don't think the drives would all write to the same cells on the same drives at the same time. But, like you, I'm not a RAID expert. If what I remember from when I did work on servers correctly, if you've got a small file (say, something that fits in one allocation unit) than that file only exists on one of the drives (while the parity information exists on another.) So now your drives aren't "aligned" such as what you're picturing, and thus are less likely to fail together in a short span of time.

Which, of course, isn't to say that they *can't* all fail in a short span of time.

It depends on which RAID level. RAID 1 is mirroring, RAID 5 is striping. So for 1, everything is duplicated. For 5, you have N number of disks, and I believe the information is striped evenly across N-1 disks. The last disk is, I believe, for parity. I am not sure if parity is always on one drive, I think it changes with each write or a similar algorithm. So therefore N-1 disks would have an equal portion of the data written to them, but not sure on parity...

I could of course read up on it again, but the point is that I am looking for real world experiences rather than theory. I don't know that reading a RAID article is going to give me that.