Napp-it cs web-gui for (m)any ZFS server or servergroups

G

_Gea

Supreme [H]ardness
Joined
Dec 5, 2010
Messages
4,234
About

Napp-it is a webbased management tool for a ZFS server since the early days of ZFS on Sun Solaris or OpenSolaris. Now ZFS is available on BSD, Illumos (Solaris fork), OSX and Windows where the first release candidate is available now. This gives a unique combination. The original Windows SMB server is not only quite the fastest one, it is also the most Windows compatible one especially regarding ACL permissions where only the Solaris kernelbased SMB server comes close. Combined with ZFS you have a dream team.

Main advantages of napp-it cs (Client-Server edition):
It is a mobile „Copy and Run“ application, no installation required
On Windows, simply download and start. Setup, Update or downgrade is a simple copy action.
It is intended that you can manage any ZFS server on any platform or replicate between any.

Requirements:
Napp-it cs needs a simple webserver with Perl and cgi capability. I decided to use Xampp portable as it offers a whole webserver suite with Apache. You can minimize size by deleting service folders beside Apache and Perl.

Current state:
First beta with basic Disk, Pool, vdev and snap management.


Setup

1. Download Open-ZFS for Windows
https://github.com/openzfsonwindows/openzfs/releases
2. Download Frontend web-gui Xampp.zip with napp-it (portable app, no installation required, just copy and run)
uncompress to c:\xampp

3. Download Backend Server cs_server.zip
uncompress and copy to any location of your BSD, Illumos, Linux, OSX, Solaris or Windows server

4. Open a Browser on Windows with adress http://localhost
or from a remote client with the ip adress of Windows.

more
https://www.napp-it.org/downloads/windows_en.html

Not convinced?
Uninstall Open-ZFS and delete c:\xampp

Discuss napp-it cs for ZFS on Windows:
https://forums.servethehome.com/index.php?forums/solaris-nexenta-openindiana-and-napp-it.26/
Discuss Open-ZFS on Windows
https://github.com/openzfsonwindows/openzfs/discussions

Current state of napp-it cs for ZFS on Windows:
https://www.napp-it.de/downloads/windows_en.html

I'll keep this initial thread up to date
 
Last edited:
newest nightly:
- download, unzip folder napp-it
- delete c.\xampp\napp-it and copy new napp-it folder to c:\xampp

Jan. 22: Menu ZFS filesystem with smb share enable/ disable
Jan 23. Option to edit driveletter
https://www.napp-it.org/doc/downloads/napp-it-win-jan-23.zip

shares.PNG



After a Windows reboot you may need to import pools
 
Last edited:
Last edited:
A Windows SMB filer with ZFS, why and when?

There was a discussion about this and the answer was:
Windows SMB direct with RDMA can be a powerful option with high performance needs ex for (multiuser) 4k+ video editing
and large (Petabyte) storage that is more secure, feature rich and faster than ntfs/ReFS (but be aware, current Open-ZFS 2.2 for Windows is release candidate/beta)

When it comes to performance over security:
https://www.borncity.com/blog/2023/...-in-windows-10-und-11-optimierung-ist-mglich/
https://www.borncity.com/blog/2023/...fer-leistung-hohe-windows-11-cpu-last-teil-1/
https://www.borncity.com/blog/2023/...sfer-leistung-und-cpu-last-optimieren-teil-2/
(in German, use Chrome to translate on the fly)

SMB Performance tuning
https://github.com/MicrosoftDocs/wi...ce-tuning/role/file-server/smb-file-server.md
 
Last edited:
Napp-it is now portable regarding path and name of napp-it folder. This will allow a online update/downgrade in future similar to the Solaris version. You can now keep versions like /xampp/napp-it and /xampp/napp-it2 or /xampp/web-gui/data,data2,.. Start the version with /xampp/napp-it[n]/data/start_web-gui_as_admin.bat, A Job background task will allow to start tasks without any modifications in Windows (under development, not ready). Menu Pools shows now state of dedup and special vdevs.

special.PNG
 
Last edited:
newest beta
jobs now working (scrub, snap and other)
new menu System with kstat and registry infos

Pool > Expand Raid-Z (raid-z expansion)
Pool > Create: draid

todo
other jobs
performance ex phys blocksize of ZFS under Windows (512B -> 4K)

To update to newest, download
https://napp-it.org/doc/downloads/napp-it_nightly.zip
 
Last edited:
extend napp-it with private menues
If you want to manage Xampp or other services via private napp-it menues (update/downgrade safe):

- create a private menu folder 1nn_xx in /xampp/[napp-it]/_my/menues for example 102_my_first_menue
- place a file action.pl with your menu actions within, see other examples under _my/menues

Typical structure of a napp-it menu script.
1. collect all relevant infos into a Perl hash ex %current
you can display hash state via &print_hash(%current);
like i do in menu ZFS Filesystems > data zfs
2. interact with user to manage settings
3. execute commands via &exe("powershell command"); with admin permissions.

To debug a script, use &mess($x), &print_var($x); &print_array(@x) or &print_hash(%x);

To edit Perl scripts, use the DZsoft Perl editor (now free)
DzSoft's Order Page

https://www.perl.com/pub/2000/10/begperl1.html/
 
Last edited:
groups.png


Napp-it for ZFS on Windows now supports groups to manage several Windows ZFS servers. This will be extended later to non-Windows servers like OmniOS or Linux but BSD, OSX or SmartOS are aldo possible options

Download (unzip and replace /xampp/[nappit]/[data] ([names] can be modified)
To update to newest, download
https://napp-it.org/doc/downloads/napp-it_nightly.zip

update 21.2.
Socketserver
I have added code to detect client OS (BSD,Linux,OSX)
napp-it now displays os:hostname as header ex W10:hostname and not only hostname

Tests on OmniOS,. I have included Perl modules not preinstalled on OmniOS
On OmniOS the socketserver often stops after requests, need time to investigate
 
Last edited:
ZFS on Windows ! I wasn't expecting that. At all. After many years on OpenIndiana with napp-it I just installed ZFS on Linux for the first time.
 
Aesma said:
ZFS on Windows ! I wasn't expecting that. At all. After many years on OpenIndiana with napp-it I just installed ZFS on Linux for the first time.
Click to expand...

Open-ZFS 2.2 on Linux or Open-ZFS 2.2 on Windows?

Open-ZFS is the same, it just uses Windows 10/11/Server as base OS with the main advantage that SMB direct with rdma and ntfs ACL features are best on native Windows. Based on first performance tests, ZFS is not as fast as ntfs (to be expected, CoW and checksum security have a price) but faster than ReFS that you had to use in the past if you wanted state of the art data security on Windows, https://github.com/openzfsonwindows/openzfs/discussions/352

Solaris/Illumos with NFS4 ACL and local SMB groups come near and is superiour to Posix ACL that you usually have on Linux + SAMBA.
 
Last edited:
On Linux as it's running on proxmox, with a samba share so that the SSD is available to the VMs, all on a ultra small Aliexpress PC. But I will consider a ZFS mirror on my main Windows PC.

I think it might have some future in enterprise settings where windows servers are needed to run some apps. If it works well and windows admins hear about it.
 
It really excites me when I sit in front of a Windows desktop and manage my OmniOS ZFS servers with same performance like on a local napp-it GUI or via Putty. A simple select and napp-it switches to another ZFS server.

Works like a charm beside large files. A zfs get all produces around 1 MB data and this seems too much for the socketserver in sync mode or even filetransfer mode with tmp files. Will need to check async socket connections. ZFS GUI remote control with zfs/zpool functions or cli commands via napp-it cmd form works on OmniOS and propably on BSD/Linux/OSX. Disk detection must be next per OS family.

New name for napp-it for ZFS on Windows is napp-it cs (client/server edition) as it should be able to manage (m)any ZFS server.
 
napp-it cs can now manage OmniOS, Linux and Windows ZFS servers

I have tested with Proxmox but any Linux ZFS server or appliance should basicall work
You only need Perl and ssh root access.

triple.PNG


Infos: napp-it // webbased ZFS NAS/SAN appliance for OmniOS, OpenIndiana and Solaris , ZFS Server for Windows
Proxmox: napp-it // webbased ZFS NAS/SAN appliance for OmniOS, OpenIndiana and Solaris : Linux

current nightly and Xampp2 are up to date

There remains a lot of work,
the basic problems are solved.
 
OK, I can confirm that ZFS on OSX can be managed by napp-it cs as a group member.
As I have no real Mac hardware, I can only test the software, not disk detection
or reading smartvalues. But it looks very promising.

The last one that is missing now is SmartOS
where a Perl would be needed on persistent storage.

https://www.napp-it.org/downloads/osx.html
 
There is a new Open-ZFS 2.2 rc.13 for Windows

Jorgen Lundman: " I threw in some code that looks up ashift to use instead of 512. If you can, check out OpenZFSOnWindows-debug-2.2.99-13-gfddfb6aeb5.exe. I did the most obvious places, but there are quite a few ways to query that.

*** Please update, the trim bug might be corrupting pools! ***

Now trim is disabled by default, to check it works (on test pools right?) change
HLM/System/ControlSet001/Services/OpenZFS
windows_enable_trim to 1. "

Care: ashift is not a pool but a vdev property. Different ashift in a pool is bad but can happen if you add vdevs without forcing ashift manually.

https://github.com/openzfsonwindows/openzfs/releases
 
Last edited:
Open-ZFS for BSD, Linux and OSX (rc) is now at 2.2.3 with improvements around trim and many bugfixes,
https://github.com/openzfs/zfs/releases/tag/zfs-2.2.3

Open-ZFS for Windows is 2.2.2 but as far as I know Jorgen Lundman
2.2.3 on Windows (rc) will be out soon

OpenZFS on Illumos is seperate as it has its own repository with integration
of Open-ZFS improvements after an additional review
 
Besterino is best..

Tests with Window SMB direct rdma/ vhdx/iscsi vs local performance:
SMB performance 11GB/s read and 5GB/s write on 100G nic, quite the same as local performance with lowest CPU load.

This makes Windows superiour for any high performance workload (ex 4k+ video editing)
with ntfs, ReFS or ZFS on Windows

https://www.hardwareluxx.de/communi...er-v-stammtisch.1114189/page-24#post-30305264
(use Chrome to translate on the fly)
 
Last edited:
New release of ZFS on Windows-2.2.3rc1, it is fairly close to upstream OpenZFS-2.2.3
https://github.com/openzfsonwindows/openzfs/releases
  • Redone CPUID probe, please verify it is correct
  • Tunable: "zfs_bclone_wait_dirty" set to wait instead of error from clonefile.
  • DDT work (memory reduction, speedup)
  • zfs list: add '-t fs' and '-t vol' options
  • Add slow disk diagnosis to ZED
  • Add 'zpool status -e' flag to see unhealthy vdevs
  • BRT: Fix FICLONE/FICLONERANGE shortened copy
  • (zpool: Add slot power control, print power status) not on macOS
  • spa: make read/write queues configurable
  • Allow block cloning across encrypted datasets
  • Add libfetch for https://
 
Small update of the remote_cs_server software as replication from any to any server requires more work
https://www.napp-it.org/doc/downloads/remote_cs_server.zip
This howto requires server software up from Mar 16. 2024
########################################################

Client /server web-gui for (m)any ZFS Server

is a client server configuration with a frontend web-gui (currently Windows web application on Apache) and a
server backend that can run on nearly any ZFS server or appliance (BSD, Linux, Illumos, OSX, Solaris or Windows)

This folder "remote_cs_server" contains the server backend with the startup script "start_server_as_admin.pl" and "cs_backend.pl" with the server part and server.auth with the authorisation key.

cs_backend.pl is a background task that runs with admin permissions to process and return data ex for zfs and zpool commands

client.pl
this script is an example to request data from the server if you want to add your own addition or menu

Client/server setup should work on any server or appliance with Perl installed and access to a root console
based on *BSD, *Linux (Debian/Proxmox), OSX, Solaris/Illumos and Windows 10/11/Server

Server Setup
Upload folder remote_cs_server to any location of a ZFS server
(ex /pool/filesystem or desktop or a system folder like /var/web-gui)
via wget, SMB, WinSCP or any other method

Optionally
edit cs_backend.pl (WinSCP, midnight commander or other editor) and
adjust allowed_ip (restrict clients), timeout (client timeout), max_connections (concurrent number of clients)

Start server backend from console as root or admin
perl /path_to_this_folder/start_server_as_admin.pl or with a Windows ZFS server
/xampp/perl/bin/perl.exe /path_to_this_folder/start_server_as_admin.pl

If you use a Putty remote console to start the server:
When Putty disconnects the server stopps, use only for tests.
 
I have uploaded a new nightly of the client/server ZFS GUI napp-it cs (Mar. 20)

- Allows large zfs list or get all (up to several MB)
- Client/ Frontend web-ui app (Windows): Copy and Run
- Server/ Backend software (BSD, Linux, OSX/ Solaris/Illumos, Windows): Copy and Run from any location like /var/web-gui, desktop or ZFS pool
- Jobs like snap or scrub run remote, replication any source /any destination memberserver is next.
- Gui performance: I would say very good especially as there is no local ZFS database to allow CLI modifications

see napp-it // webbased ZFS NAS/SAN appliance for OmniOS, OpenIndiana and Solaris , ZFS Server for Windows
 
There are new release candidates of Open-ZFS 2.2.3 for OSX and Windows

https://openzfsonosx.org/forum/viewtopic.php?f=20&t=3878
https://github.com/openzfsonwindows/openzfs/releases

btw
the more people are testing Open-ZFS 2.2.3 on OSX or Windows and report problems,
the faster remaining installer, driver or integration problems get fixed.

ZFS 2.2.3 on OSX or Windows is quite identical to Upstream Open-ZFS 2.2.3 so data security
should be already as good or bad as ZFS on Linux

https://github.com/openzfsonosx/openzfs/issues
https://github.com/openzfsonwindows/openzfs/issues
 
Last edited:
TrueNAS as a servergroup member

- Enable SSH, allow root (sharing options) or SMB
- Copy napp-it cs_server to a filesystem dataset ex tank/data (/mnt/tank/data)
- open a root shell and enter:
perl /mnt/tank/data/cs_server/start_server_as_admin.pl
Add Truenas to your servergroup (ZFS Servergroup -> add)

truenas.png

Anyone with a Qnap ZFS Box?
Can you verify a similar setup
 
napp-it cs beta, current state (apr.05)

Server groups with remote web-management: (BSD, Illumos, Linux, OSX, Windows): ok
ZFS (pool,filesystem,snap management): ok on all platforms
Jobs (snap, scrub, replication from any source to any destination): ok beside Windows as source or destination
(Windows as source works with nmap/netcat on Windows)
 
How much RAM do I need for napp-it cs

RAM for a ZFS filer has no relation to pool or storage size (beside dedup)!

Calculate 2 GB for a 64bit OS, add 1-2 GB for a Solaris based filer and 3-4 GB for a BSD/Linux/OSX/Windows based filer for minimal read/write caching or ZFS can be really slow. RAM above depends on web-gui, numer of users or files, data volatility or wanted storage performance. Add more RAM for diskbased pools than SSD pools for a good performance.

With napp-it cs I suggest 8 GB for the Windows machine where the frontend web-gui is running, 16GB if you additionally use ZFS on Windows on that machine.

For the ZFS filers that you want to manage with napp-it cs there is no additional RAM requirement for the server app what means that napp-it cs can manage a Solaris/Illumos based ZFS filer with 2-3 GB RAM and a BSD/Linux/OSX/Windows ZFS filer with 4-6 GB RAM what may allow to manage even a small ARM filerboard with ZFS like a Raspbee up from 2GB remotely with a ZFS web-gui (you only need ZFS and Perl on it).

If you use ZFS on such a board, you may try and report.
 
Raspberry 4 can be managed remotely by the napp-it cs web-gui
Just upload folder cs_server and start via
perl /path_to_cs_server/start_server_as_admin.pl

# uname -a raspberry4~192.168.2.89
Linux phoscon 5.10.103-v7l+ #1529 SMP Tue Mar 8 12:24:00 GMT 2022 armv7l GNU/Linux

ps aux

pi 30136 0.0 0.2 16540 10104 pts/0 S 15:23 0:00 perl /tmp/cs_server/server.pl
pi 30138 0.0 0.1 11192 4880 pts/0 S 15:23 0:00 perl /tmp/cs_server/monitor.pl

Resource needs are ultra low by the two napp-it cs processes on the pi.

Not the perfect tiny ZFS server with USB but should work with self powered USB disks.
Cnances are high that other Arm boards like Odroid M1 with M.2/ Sata work as well
 
Last edited:
You must log in or register to reply here.
Back
Top