Fileserver w/ file-integrity as #1 priority... on a budget

E

Elledan

[H]ard|DCer of the Month - April 2010
Joined
Oct 18, 2001
Messages
15,913
With an ever increasing number of computers in my possession, most of them actively used for a variety of tasks, I've finally come to the realization that it's time I start centralizing the data which is now stored on those individual systems. Hence I'm looking at setting up a fileserver which has the following characteristics:

- Runs Linux (Slackware w/ kernel 2.6)
- Provides those services: NFS, SMB, DHCP & TFTP (for diskless systems)
- RAID

The LAN to which this system will be connected consists out of around 8 systems, not all of which will be accessing the data stored on the fileserver continuously (like the (diskless) router).

The only potential problem is the budget for this server. If possible I would like to use one of my spare systems (mostly P1) and outfit it with an IDE-controller to which the HDDs (probably PATA) will be connected. Software RAID would then have to provide the required level of redundancy to protect the data stored on those HDDs.

Questions:

- Which RAID level should I use? Just 1? Or is 5 a realistic option?
- How much RAM should I have in this system? 64 is pushing it, isn't it?
- Which controller card is a good choice in this case?

Any comments are welcome :)
 
Software RAID5 will clobber a P1. A simple mirror with a hot spare may do what you need it to and create far less overhead. I'd also highly suggest upping the memory to 128MB or more if you can. For a few years I've been using a P200 non-mmx as an NFS/SMB/DNS/HTTP/SSH server for my parents' house. It's worked well with FreeBSD 4.x and my only real complaint is that the nic & IDE controller in it suck, so sustained xfer rates are low. Buildworlds are also slow, but I scripted those processes to run at 1am, so it's no longer such a pain. It has 160MB of memory, most of which is used supporting two users.

Lastly, make damn sure you make regular backups. Centralized data stores attract gremlins, gremlins eat data, managers without data eat IT guys. RAID is not a backup, a tape system where you update twice daily and have a full daily stored off-site is.
 
Let's see:

- P1 166 non-MMX
- 64 MB EDO SIMMs (I'll try to get my hands on some cheap 32 MB sticks)
- RealTek 8139C-based NIC (well, it's cheap and it works...)
- Advance 29134 UDMA/133 controller card (2 channels)
- 3x Seagate 120 GB Barracuda 7200.7

And a Linux CD of course :)


No need to worry, I'm well aware that RAID is only for Redundancy, not for backup. The only thing I worry about is that I lose data before I get the chance to back it up.

Tape is a bit over my budget, though, so I'm stuck with CD-R(W)s and ZIP-drives for now.
 
It won't be that bad. Try running a lab of 30 X11 thin clients off four shared 10MBit lines to 100MHz HPUX servers that get files off a 75MHz system sitting in another room on a heavily populated 100MBit line.
 
Originally posted by p0rt21
I guess speed isn't an issue :eek:
Click to expand...
The speed of the CPU isn't really important here (except for the software RAID), but the I/O-bandwidth is. That's why I use a seperate controller card for the IDE drives, namely so that I can use ATA-100/133 instead of being limited to ATA-33 or 66.

One more question: should I put the OS on a seperate HDD, or install it on the RAID-array? I think I should, but I'm not 100% certain.
 
How much downtime can you afford? If you're worried, put it on a mirror. I wouldn't put it on the same disks as the data, though.

On the side, samba can consume a significant amount of CPU power. SMB isn't the greatest protocol, and if you transfer lots of little files this will be dog slow, since CPU and network usage will spike on that per-file overhead.
 
Originally posted by Snugglebear
How much downtime can you afford? If you're worried, put it on a mirror. I wouldn't put it on the same disks as the data, though.
Click to expand...
Well, this is for my own network, so I'll be the only person affected by any downtime. However, downtime is a bad thing as far as I'm concerned, so I would like to avoid it. The downtime associated with installing the image created of the OS disk (w/ the image stored on the RAID array, of course) should be minimal, provided I've got a spare HDD to install it on.
I guess I'll use a seperate (non-mirrored) OS disk and see how it goes.

By the way, I've also considered using a 'shadow' fileserver, which would basically be a mirror of the primary fileserver. An advantage of this setup is that even severe hardware failures (exploding PSU, dying mainboard, etc.) of the primary fileserver wouldn't lead to downtime. This might be a bit tricky to pull off for me right now, though.

On the side, samba can consume a significant amount of CPU power. SMB isn't the greatest protocol, and if you transfer lots of little files this will be dog slow, since CPU and network usage will spike on that per-file overhead.
Click to expand...
I usually transfer relatively large (video/audio; 100+/4+ MB) files.

There's no way to add NFS support to Win2k/Win9x, or is there?
 
The problem with shadow fileservers is that they tend to mirror things like accidental or malicious deletions, data corruption, etc. You can get around that by having a fixed snapshot in addition to the regular/ongoing backups, but that eats up a lot of space.

For the files you're moving around, yeah, SMB isn't going to be the fastest, especially on that box. NFS would be better, but there aren't any freeware clients for Win32 that I'm aware of (cygwin has an NFS server for Windows, but that's it). Plenty of payware Win32 NFS clients, though.

Might as well give it a try and see what happens. If it's below your expectations, start opening up the bottlenecks.
 
Originally posted by Snugglebear
The problem with shadow fileservers is that they tend to mirror things like accidental or malicious deletions, data corruption, etc. You can get around that by having a fixed snapshot in addition to the regular/ongoing backups, but that eats up a lot of space.
Click to expand...
Indeed. Just keeping the shadow synchronized with the main server is not difficult to pull off, but to set up a really useful shadow is far more difficult. I'll just have to stick with regular backups for now.

For the files you're moving around, yeah, SMB isn't going to be the fastest, especially on that box. NFS would be better, but there aren't any freeware clients for Win32 that I'm aware of (cygwin has an NFS server for Windows, but that's it). Plenty of payware Win32 NFS clients, though.
Click to expand...
Hmm... I'm actually in the process of moving all of my systems to Linux so I could in theory just use NFS and an FTP server or something like that. SMB sounds like it might be too much trouble to be worth it.

Might as well give it a try and see what happens. If it's below your expectations, start opening up the bottlenecks.
Click to expand...
If I drop SMB and use hardware RAID 1, at least more resources will be available. I could upgrade the RAM to 128 MB (if the mainboard supports it) as well. Otherwise I'll just have to use another one out of the pile of old systems I got here.

Thanks for your help so far! :)
 
That depends what you mean by hardware RAID1. Most of those controllers aren't, so the CPU ends up doing the work anyway. Fortunately it's not a whole lot of work. NFS is definitely a faster protocol if you can use it, but those damn windows machines have to be taken care of. Since mapped network drives are almost essential to keep things transparent, SMB winds up getting used. It's certainly not great, but gets the job done.
 
Life would be so much easier without those pesky Windows machines ;)

Anyway, so I wouldn't gain much from using hardware RAID 1 instead of software RAID 1? In that case I'll just have to focus on getting more RAM in this system and somehow deal with this SMB-issue.


By the way, 72-pin EDO SIMMs do not have to be used in pairs, right?
 
You must log in or register to reply here.
Back
Top