Need a motherboard that supports 256gb or RAM or more

H

HDHaggis

n00b
Joined
Sep 2, 2020
Messages
11
I’m planning on filling up a server rack of severs that will be doing at times GPU-focused work using RTX 4090s. At times there will be intensive CPU work as well, so these nodes will have 32 or more threads for each server.

Any suggestions? I already built one using an AMD 7950 CPU with an rtx 4090, but it is maxed out at 128gb of RAM.

Whether it’s Intel or AMD for the CPU, that doesn’t matter as long as the performance is good. I’d like to keep each node under $6000, not including the server rack/case.

Thoughts?
 
How many 4090s do you need per node?

Initial thought is secondhand lenovo server motherboard, secondhand xeons
 
vegeta535 said:
I think you will have to go with thread ripper or Intels new Sapphire lake CPUs. None of the main stream CPUs officially supports more then 128gb.
Click to expand...
All mainstream ddr5 boards technically support 256 gig of ram using dimms like these. But if you need that many cores you need to go Server or hedt. (Short of the desktop 16/24 core chips) Thing is, those high core count chips need watercooling/aios to really get working right

I say look for a server liquidator and get a second hand pair of 16 or 20 core chips (Xeon gold or something), and some ram to suit
 
Last edited:
Keljian said:
All mainstream ddr5 boards technically support 256 gig of ram using dimms like these. But if you need that many cores you need to go Server or hedt. (Short of the desktop 16/24 core chips) Thing is, those high core count chips need watercooling/aios to really get working right

I say look for a server liquidator and get a second hand pair of 16 or 20 core chips (Xeon gold or something), and some ram to suit
Click to expand...
They do but people are having issues running more then 2 dimm on both Intel and AMD.
 
What's your application? One does not casually "fill up a server rack of servers", you're talking $25K+ of hardware pulling 10KW or more. Does the application scale across cores? Does it need single-threaded performance? What OS will you be running?

The only officially supported solution (in-production CPU, board) is Sapphire Rapids. AMD offers Genoa but if you were looking at the 7950X Genoa isn't a comparable processor (it has many more cores at much lower clock speeds), and Threadripper Pro is currently one generation behind. If you're crafty you can do much better with used parts, but if you knew how to do that you wouldn't be asking...
 
Used EPYC systems unless you need very high single core speed.

Registered RAM DDR4 is dirt cheap used, you can really knock yourself out.

You'll stay way below $6000 is you want.
 
Application is primarily deep learning (Tensorflow/Keras), with some Houdini/Blender/Unreal work.
 
This is where used Dell/HP servers fit the bill since you can load these up with 1TB+ of memory on the cheap. You may not have as much single thread performance, but with copious amounts of ram and the work staying there, I think you'll get the same amount of work done cheaper than even a faster single thread setup with less ram. But you know your use case better, so you'll know what you need. But when you can pick up used servers for $100/ea in quantity (or less), the economies of scale of this may work really well for your use case.
 
On the latest mainstream DDR5 desktop platforms like Intel 12/13th gen and Ryzen 7000, you are stuck with a max of 192gb across 4x48gb sticks. moving to an HEDT platform with Intel W790 gets you up to 2TB or 256gb with Intel x299 and AMD TRX40
 
https://twitter.com/kevinmitnick/status/1649421434899275778

via

https://www.ekfluidworks.com/ :D

FuPr4FAX0AE3q_Q?format=jpg&name=4096x4096.jpg
 
  • Like
Reactions: uOpt
like this
You must log in or register to reply here.
Back
Top