HD suggestions for ZFS?

D

devilmouse

n00b
Joined
Mar 28, 2010
Messages
10
I've got a Norco 4224 sitting next to me with a bunch of empty HD trays (read: they're all empty) and whenever the motherboard arrives (an X8SI6), I'm going to be putting it together and throwing Solaris or some derivation thereof on it. I don't have an immediate need for all 24 drives, but I do need at least a few to start with. =p

So a two part question: Assuming I want 2 TB drives, which brand/model do I go with (Hitachis seem all the rage. As for Samsung's F4s, it feels too early for 4k sector drives, but I'm happily convinced otherwise. WD Greens?)? And then what raidz configuration do I use? My initial thought is to buy them in sets of 8 and set them up as raidz2 arrays, expanding the drive sizes as I buy them over the years (8x 2 TB now, maybe 8x 3TB in a year, etc), and adding them to the pool.

Thoughts? Thanks!
 
My preference from best to worst:
Samsung F4 (666GB platter)
WD EARS 3-platter (666GB platter)
512-byte 500GB platter disk (5400/7200rpm)
WD EARS 4-platter (500GB platter)

If you use 4K sector disks, you may want to use my ZFSguru distribution to create the pool, since Solaris cannot change the 'ashift' value of your pool i think. After you created a pool with my distro and set sectorsize override to 4096 bytes, the 'ashift' value of your disks/pool will be set to 12; ideal for 4K sector disks. Now you can go back to Solaris and you should have very good performance.

Samsung F4 still does 140MB/s max at 5400rpm, at very low cost. Lowest ratings are 70 euro in my area. This makes it the best performance/capacity ratio i think.
 
Oh! I didn't realize once the pool was created with ashift, Solaris could still use it. Good to know, thanks.

Any thoughts on disk config?
 
Note: my sectorsize override feature should NOT be confused with the ashift-patch, that FORCES ashift=12 even on existing pools; that patch is dangerous and should NOT be used for real data.

My procedure is completely safe, cross-platform and persistent across reboots and even across OS! If you need higher ZFS version than 14 or 15, then you can upgrade it on Solaris when you import it. I recommend not to do so, however, to keep the pool cross-platform compatible to FreeBSD in particular which has lower ZFS versions for now.

As for disk config:

5-disk RAID-Z or 6-disk RAID-Z2 is your best bet. Multiple arrays in the same pool is good, though you will add extra redundancy (parity disks) this would maintain both performance and redundancy level as you expand.

Other good performing combinations:
9-disk RAID-Z (not recommended; too low redundancy)
10-disk RAID-Z2 (decent redundancy level)

You can also consider mirroring, adding arrays in pairs to your pool. You can start with say 4 disks (2 vdevs of 2 disks in a mirror config) so you would have:

pool: tank
vdev1: mirror (disk1, disk2)
vdev2: mirror (disk3, disk4)

Then you can add as many disks you want in pairs, making expansion more easy; just buy 2 more disks each time. This also escapes any 'variable stripesize' problem, and gives you a new feature: degrading a mirror to single disk to 'steal a disk' from the mirror, and upgrading a single disk to a mirror again. This allows you to shuffle with disks and so on more easily.

There is one problem with the mirror: it's not THAT safe. Even with 50% redundancy level, you could run into problems with 1 failed disk already, and you start a rebuild, and one of the remaining disks encounters BER and cannot read a sector. ZFS won't mind if its metadata (ditto copies) but if it's your data and you didn't use copies=2 on it, then that file will be inaccessible/corrupted. You would see the name of the file in the zpool status -v output. The ZFS filesystem itself should never get corrupted, since all metadata uses ditto blocks, even on single disk pools.
 
sub.mesa said:
pool: tank
vdev1: mirror (disk1, disk2)
vdev2: mirror (disk3, disk4)

Then you can add as many disks you want in pairs, making expansion more easy; just buy 2 more disks each time. This also escapes any 'variable stripesize' problem, and gives you a new feature: degrading a mirror to single disk to 'steal a disk' from the mirror, and upgrading a single disk to a mirror again. This allows you to shuffle with disks and so on more easily.
Click to expand...

Huh - I'd never thought of that. That's a pretty swanky trick, though I'm not sure if it's for me. I'll probably go with the 6-disk raidz2s, so I can add 4 sets of vdevs over time to fill out the case.

Thanks for the input, sub.mesa!
 
With my -4020, I'm going to do 3 6-drive RAID-Z2s + 2 hot spares to be shared among the vdevs. Pretty good mix of redundancy and IOPS that way. Have a 30GB Agility 1 as the OS drive, and I need to cram in another 2 drives in a mirror for VMs and the such.

I'm looking at the Seagate 5900rpm drives; the 7K2000s haven't dropped in price in the past few months, and their SmartAlign tech seems to take care of any 4k weirdness. I'll likely patch ZFS to allow me to set ashift, and then go back to a stable release once my pool is created.
 
I would still recommend the Hitachi 7200 RPM 2 TB drives. They are fast, cheap, reliable, and you don't have to worry about the 512 to 4K sector conversion issues that can bite you in the bottom if you aren't careful. Of course, we have Fry's nearby which often discounts them.

Sub.mesa has made dealing with the overhead of emulation a lot easier in his distro, which is freeBSD based, but the CIFS in-kernel support in opensolaris derived distros provides much faster performance in windows filesharing than SAMBA in my experience. Maybe this will change in the future, but in kernel CIFS is a big draw for a ZFS server that can deliver high performance.
 
mikesm said:
I would still recommend the Hitachi 7200 RPM 2 TB drives. They are fast, cheap, reliable, and you don't have to worry about the 512 to 4K sector conversion issues that can bite you in the bottom if you aren't careful. Of course, we have Fry's nearby which often discounts them.

Sub.mesa has made dealing with the overhead of emulation a lot easier in his distro, which is freeBSD based, but the CIFS in-kernel support in opensolaris derived distros provides much faster performance in windows filesharing than SAMBA in my experience. Maybe this will change in the future, but in kernel CIFS is a big draw for a ZFS server that can deliver high performance.
Click to expand...

I really appreciate the work sub.mesa is doing; spreading the gospel of ZFS is always good, even if it's with BSD ;) Right now, I'm on OpenSolaris, planned migration to OpenIndiana; I struggled in mid-late 2007 with learning Solaris well enough to get a ZFS fileserver up and running, so now that I've got the knowledge, it isn't as bad (and I've got *rc files up the wazoo that make it feel more like Linux).

I agree, the 7K2000 is definitely the easiest way to do this, no need to worry about advanced format at all. Sadly, in MI, there are no Fry's, and the lowest I've seen 7K2000s drop is around $110 on the 'egg. I'm willing to chance the Seagates though; going to get 2 from the Cyber Monday deal and abuse them as a little 2-disk pool to see what happens. Though, even though the 7K2000s draw more power, they are rated for "heavier" operation I think.

If they turn out to be unsuitable, well, my parents' Popcorn Hour needs a bigger drive, and so does their desktop :) And I'll be sure to blog about it/post about it. Wish I had (or we could get some sponsors) a budget to do ZFS server builds with each 2TB model (Barracuda, Caviar, SpinPoint, DeskStar, etc) and run 'em through benchmarks.
 
You must log in or register to reply here.
Back
Top