linux - Running ZFS as MediaStorage with non ECC-Ram
2014-04
Thomas Nordquist
First to the Setup:
I'm planning to use 8-11 USB3 Disks to form a MediaStorage
The idea is to use 2 USB3 Hubs to balance the load between two seperate USB3 Ports (on different hubs)
Now from what I've read so far it seems a terrible idea to use any kind of software Raid when you only got non-ECC RAM
https://pthree.org/2013/12/10/zfs-administration-appendix-c-why-you-should-use-ecc-ram/
Studys from google say you get bitflipping approximatly 8% per DIMM per year, with 4 DIMMs its just 32% that you got one bitflip per year if the server is running 24/7
Since I only store big media files where i don't care if i loose a bit or byte a year, and any degredation of this kind seems not to pose a serious threat
Since i already got 16GB of ram and I don't want to buy new ones the quesion is, is the threat so great or is there a better alternative to ZFS ? Raid6 would be nice =)
Mike Fitzpatrick
If, as you said, you're not too worried about a potentially small failure rate then give ZFS with non-ECC a try. Set up the pool(s) with some form of redundancy (copies=2, raidz or raidz2) and check their integrity regularly with zpool scrub. Any bit errors will be picked up during the scrub and should be automatically repaired if the pool has some form of redundancy.
I use ECC memory and back up to external USB HDDs with ZFS. I have on one occasion detected an error on the backup during a scrub so, even with ECC, the process is not error-free. Bit errors can occur at the disk level, during read/write or the media itself.
Using ECC helps prevent bit errors while the data is in RAM. ZFS checksums and redundant pools help prevent bit errors on disk.
teabot
Which non-Apple drives work reliably with the latest Mac Pro RAID card (early 2009 Nehalem)? I 'm interested in drives that are >= 1TB in size.
Mark Henderson
I'll let you in on a little (not so) secret.
An "Apple" drive is just a drive from another brand, with an Apple sticker stuck over the top. So as far as reliability goes, just choose one from a recognised manufacturer (Western Digital, Seagate, Maxtor, etc) and you won't have any issues.
Personally I'm using WD Green drives.
The only thing to be wary of is that some controllers (I don't know about apple in this case, but it wouldn't surprise me given their history) will refuse to work with off-brand drives. I had a RAID controller from an old dell that would not work with anything except Dell drives, even when the other drives were identical, just with a different part number (same manufacturer, but non-dell branded).
That said, there's only one way to find out.
Apple Hardware Tech
Alexander, search the various forums for people using the Mega RAID cards supplied by Apple for their XServe G5s. I believe people were still reporting issues even with the RAID card for the early 2006/2007 MacPros as well. Some people can get them to work and others cannot. We had them working for a while and then the RAID settings kept disappearing from the RAID card. We tried other Mega RAID cards, motherboards, and hard drives, etc. We found only the Apple branded drives would work correctly all the time. We even used identical drives minus the Apple branding and those drives did not work. Only difference was the Apple firmware on a drive. Up until that time we would install non-Apple drives on Apple RAID cards. Now we need to do extensive testing and research before doing so anymore.
It seems the 2008 MacPro Apple RAID card may work fine with non-Apple drives. We've been running a machine with them for a few months without seeing problems yet.
Always check the manufacturer's recommendations and always search forums to see if anyone is having issues with a particular configuration is not covered by the manufacturer's recommendations. Then make sure you perform your own testing before relying on it for a critical system. Just because it unofficially worked in the past does not mean it will work on newer hardware.
MacLemon
Even though Apple uses major brand drives in their systems, they may contain custom Apple firmware. I personally would not use non-Apple drives on any Apple hardware raid. You don't want to risk an intermittent issue cropping up because of a firmware incompatiblity. I've seen some Apple hardware RAID cards work with non-Apple drives and then suddenly have intermittent failures after a bit of use. If you want to use non-Apple drives, then invest in a third party RAID card that does support standard drives.
While many hardware RAID vendors do limit compatibility with some drives, I feel it is wrong of Apple to limit it to only their outrageously expensive ones.
Old Pro
I've been using the Apple Raid Card (first release version) since 2008. Used it very successfully with Hitachi Ultrastar A7K100 750GB drives (3 SATA drives in a RAID 5 configuration) and 1 Seagate Cheetah 15K.6 300GB SAS drive. Put the OS on the SAS drive and data files (digitized feature film video) on the RAID. Except for one failed battery (requiring the card to be replaced under warranty), really no problems with it. This is even though Apple says the card needs all SAS or all SATA drives.
However, about 5 months ago I upgraded from the 750GB drives to 2TB Hitachi 7K3000 Deskstar drives, which have flaked out, at a rate of about once every 2 months. The RAID card simply loses contact with a drive, declares it missing, kicks it out of the RAID array, and then almost immediately reconnects to it, but because it has been kicked out, it is treated as a new drive and has to be reassigned to the array, forcing a complete rebuild. Which, now that the array is made up of 2TB drives, takes about 36 hours. Kind of scary because the flakes seem random, and if one of the "good" drives fails during the rebuild, the whole array is lost. And at 3TB, I don't have full backups for the array. I've also been feeling like there are performance issues with these 2TB drives that I didn't have before, but I can't be sure.
I'm seriously considering upgrading to the SAS Ultrastar version of the drives (Hitachi Ultrastar 7K3000 HUS723020ALS640 2 TB). Just I need to buy 4 to have a spare on hand, and they're currently close to $300 each.
BTW, the reason I'm getting 2TB drives instead of bigger ones as there are serious questions about whether or not the card can handle bigger drives.
Also, while I've never bought an Apple Drive except for the ones that came with the machine (and have found those to wear out sooner than the Hitachi drives), Apple does include custom firmware on their drives and I do believe that makes a difference with the RAID card. Personally I would not attempt to use the card with drives other than from Apple, Hitachi, or Seagate. For sure I would not try using "green" drives that slow down their spindle speed to conserve power. That's asking for performance nightmares.