ZFS Fragmentation

M

m1abram

2[H]4U
Joined
Mar 15, 2002
Messages
3,175
So I have been playing with ZFS and just trying to learn about it. Came across this article thought I would share it here for people who may not have seen it.
http://wildness.espix.org/index.php?post/2011/06/09/ZFS-Fragmentation-issue-examining-the-ZIL

From that writeup it appears having a dedicated ZIL device is also useful to prevent fragmentation. I did not see in the writeup the percentage of pool usage they were running under and how that would effect the rate of fragmentation. I suspect the pool utilization was getting high and that maintaining enough free space in the pool could also help reduce fragmentation.
 
really not fragmentation per se. interesting though as i thought the ZFS intent log was always in RAM, never on disk unless a specific device (zil/slog) was specified.
 
No, if no slog is specified, zil space is used on the pool itself. It would make no sense to have the zil in ram since it would vaporize if you crash, which would obviate the whole point of a transaction log.
 
yes, i would also expect a higher disk fragmentation on sync writes without a dedicated ZIL.
With sync=disabled or sync writes with a dedicated ZIL, all writes goes to RAM where they are collected for about 5s and then written as one large sequential write.

This improves write performance especially with small writes and you do not have the situation that every single small write request must be witten to disk and commited until the next one occur with a massive speed degration and a higher fragmentation.
 
danswartz said:
No, if no slog is specified, zil space is used on the pool itself. It would make no sense to have the zil in ram since it would vaporize if you crash, which would obviate the whole point of a transaction log.
Click to expand...
not really no. remember the previous 127 uber blocks are retained on disk, losing the intent log in a hard down scenario isn't the end of the world, necessarily.

i always use slogs so i must have confused myself somewhere along the line into thinking ZIL is always in RAM, for me it is.
 
I think we have a terminology disconnect here. If you have an slog in volatile storage, and you crash,you've lost the last N seconds of writes, which may or may not be a bad thing. If that doesn't matter to you, just set sync=disabled, no?
 
danswartz said:
I think we have a terminology disconnect here. If you have an slog in volatile storage, and you crash,you've lost the last N seconds of writes, which may or may not be a bad thing. If that doesn't matter to you, just set sync=disabled, no?
Click to expand...

I think losing any amount of writes is always a bad thing. The level of that risk versus and the downside to lost data is what you weigh against the advantage of sync=disable.

In my case I have sync=disable on my VM store because the lack of write speed really kills it. I try to handle the data loss by having a reliable UPS that will properly shutdown my VMs and ESXi host if power failure happens. Also since I only store OS and Programs on that store the data loss would most likely just be a corrupted OS file that can be restored either from a snap or just reinstall. Any "created" data I do not have sync=disabled.
 
keep in mind also that there a difference between filesystem corruption and losing writes to data files. the former should never happen with apps writing to files on a zfs filesystem. with an nfs datastore it's a whole different thing, since the guest filesystems *may* be vulnerable to corruption. i don't worry and run sync=disabled since it is an all-in-one and i have a large ups.
 
Last edited:
You must log in or register to reply here.
Back
Top