I think you first need to do 'zpool remove' to remove the spare, and then do 'zpool replace' to use it to replace the removed drive?
![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)
- Thread starter _Gea
- Start date
I think you first need to do 'zpool remove' to remove the spare, and then do 'zpool replace' to use it to replace the removed drive?
Thanks for your reply
Unfortunately, I've already did that. I tried again with no luck.
If I connect myself with a user of admin group, with the computer management I've only "Everyone" and "Root" user. Despite that, I can see my users in computer management (Solaris) -> system tools -> Local user and group -> users
If I use root username, then folder -> propreties -> security, there is only "Everyone" user name.
I don't know what I missed
connect your share as user root, right klick on a folder or file and select property-security
then edit -> add -> advanced -> search
It should now display a list of Solaris users and SMB-groups
Does this happen?
Gea
Hi,
I use 7 home premium x64 and 7 ultimate x64 but I've heard there was a problem with W7 home & ultimate with ACL. So I installed a virtuel xp sp3 pro for testing. But finaly it works with W7 HP & U
and not with XpI did this on Xp, it displayed no one of my users, so I tried on W7 HP as advocated by Obscurax and it works
I'm able to see my users and add them, same thing on W7 ultimate, it's weird but it works Thank you very much both of you
Hi,
I use 7 home premium x64 and 7 ultimate x64 but I've heard there was a problem with W7 home & ultimate with ACL. So I installed a virtuel xp sp3 pro for testing. But finaly it works with W7 HP & U
I did this on Xp, it displayed no one of my users, so I tried on W7 HP as advocated by Obscurax and it works
Thank you very much both of you
i am currently working on a ACL-extension to set Share-, ZFS-folder- and other file/ folder-ACL with napp-it.
I will include ACL settings of a ZFS-folder with local user/ groups in the free version.
AD-User, Share-Level ACL and other files and folders-ACL are a nonfree extension (annual donation required)
planned first beta: end 06.2011
Gea
Solaris 11e throws these errors
bnx0 failed to initialize default msix isr handle block (-4)
bnx0 reverting to fixed level interrupts
The ML350 G5 I'm using isn't on the HCL, but this Broadcom 5708 is. The broadcom NIC driver I looked up on HPs site says the following as a Enhancement "BRCMbnx to revert to fixed interrupts when MSI-X interrupt allocation fails. This eliminates the need to manually edit bnx.conf file to use fixed interrupts when MSI-X interrupt allocation fails."
So I'm assuming that it is ok that the error is thrown? A feature maybe? Gets thrown on boot. That driver is for Solaris 10 on a different HP server but same NICs on it.
The main worry that I have about another error thrown, it states that e1000g0, e1000g1, e1000g2 are being removed from the kernel. These are the Intel PRO/1000 MT Server Dual nics, and are on the Solaris 11 HCL. They're also extremely well known supported cards. I'm not sure what being removed from the kernel exactly means for a NIC. But the nics appear to be functioning, though I can't be sure if poorly.
bnx0 failed to initialize default msix isr handle block (-4)
bnx0 reverting to fixed level interrupts
The ML350 G5 I'm using isn't on the HCL, but this Broadcom 5708 is. The broadcom NIC driver I looked up on HPs site says the following as a Enhancement "BRCMbnx to revert to fixed interrupts when MSI-X interrupt allocation fails. This eliminates the need to manually edit bnx.conf file to use fixed interrupts when MSI-X interrupt allocation fails."
So I'm assuming that it is ok that the error is thrown? A feature maybe? Gets thrown on boot. That driver is for Solaris 10 on a different HP server but same NICs on it.
The main worry that I have about another error thrown, it states that e1000g0, e1000g1, e1000g2 are being removed from the kernel. These are the Intel PRO/1000 MT Server Dual nics, and are on the Solaris 11 HCL. They're also extremely well known supported cards. I'm not sure what being removed from the kernel exactly means for a NIC. But the nics appear to be functioning, though I can't be sure if poorly.
ihave updated my changelog to be more informative:
Timeouts -> you must recreate replication jobs with this version to use this feature
Recreate jobs if you get timeout errors or if you want to have a timeout (or jobs with network interrupts may hang endless and must be cancelled manually)
>> basically a timeout is added to some netcat send/receive actions
Gea
Last edited:
I removed the spare "zpool remove fs1 c2t1d0".
Then I tried to replace the removed drive with c2t1d0 (in the napp-it GUI).
I got this error
Code:
"cannot replace c5t7d0 with c2t1d0: devices have different sector alignment"What the heck does that mean? Shouldnt it just copy data over to the new disk?
Umm, there are no drives that have 4K sectors outside of what we know. As far as any computer is concerned, all drives are still 512K afaik.
I'm having problems getting my disk pinned down.
It works but the auto service is interfering with the device-treshold.
Now for me it is spindown or auto service enabled, as my drives are polled every 15min.
Actually it makes sense, but is there a way around it?
It would be nice to have an option to run things once a day or something, I don't know I don't understand the mechanics behind it.
It would be nice if both could be combined.
It works but the auto service is interfering with the device-treshold.
Now for me it is spindown or auto service enabled, as my drives are polled every 15min.
Actually it makes sense, but is there a way around it?
It would be nice to have an option to run things once a day or something, I don't know I don't understand the mechanics behind it.
It would be nice if both could be combined.
Maybe there's still some formatting or partitioning on the drive that is confusing ZFS? Did you try zero-wiping the first and last megabyte or so of the drive with dd?
Just curious, how did you end up with a drive in the REMOVED state? Did you do a zpool remove or just physically pull the drive? If you did a zpool remove, maybe 'zpool add' is the right command to use instead of 'zpool replace'? If you physically pulled the drive without warning, ZFS should have automatically gone to the hot spare, otherwise something is very broken in ZFS.
That is my original worry (with ZFS) it seems to be less reduntant then expected, as you can read earlier in the thread. The drive basically died, not wrong sectors, which would produce errors, it just died (let's assume the electronics died, so like it was pulled) apparently this triggered ZFS to assume it was a administrative remove (which it was not). So now I am in the issue of having one of the vdevs -1 redundancy, and no way to replace that.
If anyone has any hints what to do, go for it
DJ, if the drive totally died, how can ZFS tell the difference vs you pulling the cable? Secondly, did you actually try googling for that error message (as I did)? There do in fact seem to be drives with 4096 sectors, although they emulate 512 or somesuch (forgive my sloppy wording, just going by what I have read.) The point being: there is something about that drive which is incompatible with the others in the pool. Have you tried a different drive?
Check this:
http://sigtar.com/2010/06/06/4k-sector-hard-drives-and-zfs/
If the drive in question IS a 4096 sector drive, possibly it can be jumpered to emulate 512b?
http://sigtar.com/2010/06/06/4k-sector-hard-drives-and-zfs/
If the drive in question IS a 4096 sector drive, possibly it can be jumpered to emulate 512b?
Alot of the drives are 4K in that pool. But they dont tell the computer that. We only know that because WD/Seagate/Samsung told us so. To the computer, they are all 512B Drives. Eventually, they will make models that disclose that they are 4K. but they dont. Just like SSDs, all SSDs are 4K, but all computers think SSDs are 512B.
To go further, there are exactly the same drives in the pool already. I havent tried a different drive. I may. But it's not a random test drives till I find a lucky drive that works that I am looking for. I was asking for the reasoning as to why it would complain.
To go further, there are exactly the same drives in the pool already. I havent tried a different drive. I may. But it's not a random test drives till I find a lucky drive that works that I am looking for. I was asking for the reasoning as to why it would complain.
As I understand, if it is a 4K drive, it can either present the disk as 4k or emulated 512b mode. Depending on which it is, the drive needs to be partitioned compatibly. If you go into the 'format' command, and select one of the already used drives, and then type 'partition', and then 'print', what does it say? Then repeat that for the spare drive... Actually what is better is to do 'prtvtoc /dev/rdsk/xxxxx', where 'xxxxx' is the disk name - shows more useful info...
I don't know what to tell you. You say there is no such thing - google shows me dozens of hits talking about this. Maybe we are using different terminology or something? Did you mean 'natively? Whether this is it or not, it's still possible that drive is partitioned with different alignment than the others...
All the 4k drives out there still report via ATA as 512 drives (in the spec sheet this will be listed as 512e - for emulated). They are internally 4k sector size, but handle the 512 (external) <-> 4k (internal) conversion via firmware.
When you created the zpool did you create it with gnop/hacked binary/etc? Is it possible you are getting the error because the zpool/vdev is expecting 4k drives and your drive is reporting 512byte?
When you created the zpool did you create it with gnop/hacked binary/etc? Is it possible you are getting the error because the zpool/vdev is expecting 4k drives and your drive is reporting 512byte?
It seems to me that if a drive mysteriously disappears for any reason, ZFS should assume a drive failure and immediately fall back to the spare. Pulling the cable counts as a drive failure in my book, since if any writes occur to the array while the drive is disconnected, it will be out of sync and probably will have to be rebuilt anyway. So I can't see much practical difference between the two events. Either way your data is at risk, so the spare should start resilvering at that point (maybe after waiting a few minutes to see if the drive comes back).
Here are the two drives (that I am trying to run a replace on):
Code:
admin@shiva:~# prtvtoc /dev/rdsk/c2t1d0
* /dev/rdsk/c2t1d0 partition map
*
* Dimensions:
* 512 bytes/sector
* 3907029168 sectors
* 3907029101 accessible sectors
*
* Flags:
* 1: unmountable
* 10: read-only
*
* Unallocated space:
* First Sector Last
* Sector Count Sector
* 34 222 255
*
* First Sector Last
* Partition Tag Flags Sector Count Sector Mount Directory
0 4 00 256 3907012495 3907012750
8 11 00 3907012751 16384 3907029134
Code:
admin@shiva:~# prtvtoc /dev/rdsk/c5t7d0
* /dev/rdsk/c5t7d0 partition map
*
* Dimensions:
* 512 bytes/sector
* 3907029168 sectors
* 3907029101 accessible sectors
*
* Flags:
* 1: unmountable
* 10: read-only
*
* Unallocated space:
* First Sector Last
* Sector Count Sector
* 34 222 255
*
* First Sector Last
* Partition Tag Flags Sector Count Sector Mount Directory
0 4 00 256 3907012495 3907012750
8 11 00 3907012751 16384 3907029134I didnt mess with the default ZFS config in any way. It's a pure ZFS ver 28, Openindiana build 148b
That is odd - they look identical. Couple of things to try:
1. Try using the '-f' flag to force it.
2. Power down, unplug the dead drive, plug the replacement into that slot and reboot. If it doesn't start using it automatically, try 'zpool replace' with just the single device name.
1. Try using the '-f' flag to force it.
2. Power down, unplug the dead drive, plug the replacement into that slot and reboot. If it doesn't start using it automatically, try 'zpool replace' with just the single device name.
While I'm at it, I've asked about this on a mailing list I frequent... Also, if you do a straight up install of OI build 148, once it is installed, if you do 'pkg image-update', it will find a bunch of updates and fixes. No idea if that might help or not.
Here is another interesting question:
Imagine you have one Zpool thats nearing capacity, its mainly static data. Now you add a new vdev to expand the pool size. Is there a way to balance the usage of the vdevs in the pool, so that the static data gets rewritten to all the drives in a balanced manner, like one does when one expands a RAID array ?
Imagine you have one Zpool thats nearing capacity, its mainly static data. Now you add a new vdev to expand the pool size. Is there a way to balance the usage of the vdevs in the pool, so that the static data gets rewritten to all the drives in a balanced manner, like one does when one expands a RAID array ?
wingfat
Weaksauce
- Joined
- May 13, 2010
- Messages
- 115
The only way to balance your vdev's is to back your data up and re-write to your new expanded pool. There is no "defrag" funtion in zfs.
I was actually thinking if there is no "smart" way of doing it, just making a copy of half the data, then deleting the old copy of the same data should do the trick, no ? (in a world where you expanded the pool by 50%, that is.)
I have a question about AD.
If i make a VM running Win 2008 AD that is located on the OpenIndiana Napp-ti VM, Can i then start the Napp-it VM first to make the store for AD vm available? Or do i have to run the AD VM before the Napp-it VM starts? therefor move it out of the NAS and to the local disk like Napp-it?
I hope you all understand what i mean
If i make a VM running Win 2008 AD that is located on the OpenIndiana Napp-ti VM, Can i then start the Napp-it VM first to make the store for AD vm available? Or do i have to run the AD VM before the Napp-it VM starts? therefor move it out of the NAS and to the local disk like Napp-it?
I hope you all understand what i mean