New ESX white box

M

Medieval

[H]ard|Gawd
Joined
Apr 17, 2001
Messages
1,303
Posted this (http://www.hardforum.com/showthread.php?t=1387826) over in General Hardware:

Haven't posted to HardForums in years, but I know this is the place to go. I am looking to build a whitebox ESX beast. I have parts picked out, which will allow a good initial start and allow for lots of expansion. Here is what I have in mind:


Motherboard - ASUS DSEB-D16/SAS
CPU - 2.33 Ghz Harperton Quad-core Xeon (E5410)
RAM - 16 Gb (4x4Gb Kingston FBDimm, DDR2 667)
Powersupply - Thermaltake E0131RU 850W (EPS12V)
Hard Drives - two 1 Tb Western Digital SATA (16mb cache)
Case - Chieftec Bravo BA-01B-B-B Full Tower case
Dual 120 mm fans exhaust
Maybe 120 input form the front (using a fan adapter for 5.25 drives)
multiple 90mm fans for cool air in


So what I am thinking is, do I need another quad core CPU? OR should I wait til I have mroe RAM? Should I use the onbaord RAID, or buy a Zero Channel RAID card or a full blown RAID card for my SATA drives. The eventual plan is fully populated RAM of 64 Gb and two Quad core CPUs, as well as 8x 1 Tb SATA drives all configured in mirrored pairs. However, until I get a raise or a bonus, this is going to be a future build. So, if I start off with a pair of mirrored 1 Tb drives, a quad CPU and 16 Gb of RAM, is that a good start for a home build of ESX server? Any suggestions? Am I crazy? (oh wait, yes I am).

As a certain side note, if I get this server qith one or two Quads, I plan to use it to help out Distributed Computing team, which I have not done in the past year or so..... So, maybe going all out with the CPUs is a good idea? Thanks, I look forward to any comments!


Medieval
 
The really nice thing about this is you can always upgrade. We have no idea what you want to run on it, (Number or variant of VM's) and really, just a single CPU with 16gigs ram is more then enough to start out on.
 
I pretty much will be VM'ing Exchange 2007, Windows 2008 and a whole bevy of Windows products. I really want to set up multiple CCR clusters with 03 and 08 OS's. The goal is ultimately do 16-20 VMs. Since I know RAM is my limitation, I will be doing 8 - 10 max with the 16 Gb.
 
Actually, your limitation is most likely disk I/O long before you hit RAM limitations.

I assume you are talking ESXi, not ESX?

It's a bit tough to give good advice without knowing what the load will be. If you have no load on your clusters you can likely set up any number of them. If you put load on them, then "multiple" clusters will not work due to disk I/O.

Virtualization is often misunderstood as some magic that allows you to take 20 servers and run them all on one server, saving tons of money and space. That's partially true when it comes to CPU and RAM utilization, but it's not true when it comes to I/O.

If you want to run 20 VMs, you need to provision storage for 20 servers, not just for one server. That's really the bottom line.
 
Thuleman is right, your true bottleneck is not your memory, but your storage.

Yes you have plenty of space, but you don't have plenty of spindles.
 
I was thinking about going with 500 Gb drives anyways. Now I am going to start with 2 mirrored pairs off the motherboard using one of those ZCR (Zero Channel RAID) cards and then add another 8 drives over time in mirrored pairs. I will add a second RAID card in the middle so I can split all the spindles between two controllers. That should make this run a bit better I would think. Thanks for the idea.
 
Medieval said:
Now I am going to start with 2 mirrored pairs off the motherboard
Click to expand...
That won't work.

Medieval said:
using one of those ZCR (Zero Channel RAID) cards and then add another 8 drives over time in mirrored pairs. I will add a second RAID card in the middle so I can split all the spindles between two controllers.
Click to expand...
That may not work either.

ESX(i) is extremely choosy when it comes to RAID hardware. Motherboard based fake-raid isn't supported at all. Best have a look at the HCL and grab a cheapo 4-port controller that's actually listed.
 
Thuleman said:
ESX(i) is extremely choosy when it comes to RAID hardware. Motherboard based fake-raid isn't supported at all. Best have a look at the HCL and grab a cheapo 4-port controller that's actually listed.
Click to expand...

That motherboard has an LSI 1068 SAS controller, which is supported by ESXi.

http://www.vm-help.com/esx/esx3i/Hardware_support.php

I'd grab the zero channel controller and go that route personally.
 
I've seen on the various forums on ESX people that are using this motherboard, so I am not too woried about ESX or ESXi. The key thing will be do I get mutliple RAID cards. What are your feelings on any speed benefit? I know that I am limited to 6 drives using the motherboard. So I figure I will just add another RAID card to keep building up the spindle count on the server. This should help isolate the data and make things run smoother. Thanks for the imput guys.
 
You have 8 SAS ports from the LSI 1068 in addition to the 6 Sata ports from 6321ESB...
 
ND40oz said:
That motherboard has an LSI 1068 SAS controller, which is supported by ESXi.

http://www.vm-help.com/esx/esx3i/Hardware_support.php

I'd grab the zero channel controller and go that route personally.
Click to expand...

Sorry my bad, didn't pay attention to the board listed in the first post.

I had a look at that board and it's actually quite nice indeed. Anyone know whether there's a management card for it, so that one can access the server remotely even if it's powered off?
 
Yes there is, I cannot confirm if you can actually buy it or not. The card is called a "ASMB3/iKVM card" and it fits in the boards SODIMM socket. There is a RJ45 port on the board to hook up it to a switch for iKVM. I believe its like a GUI interface that HP servers use called iLO. There is also a card called a ASMB3-SOL that is used for remote monitoring and text based access to the server, this card also fits in the SODIMM slot. There is a port for that as well on the motherboard. Hope that helps.


Medieval
 
Thanks, yeah the ASMB3/iKVM was just what I was looking for.
Though the issue resolved itself as I just ordered another Dell 2950 III instead of salvaging a 2U enclosure I still had around, they have a $1,200 off promo through their SMB division and I just couldn't pass that up.
 
You must log in or register to reply here.
Back
Top