196

16484 readers

1740 users here now

Be sure to follow the rule before you head out.

Rule: You must post before you leave.

^other^ ^rules^

founded 1 year ago

MODERATORS

[email protected]

655

The Rule (lemmy.ml)

submitted 3 months ago by [email protected] to c/[email protected]

63 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 28 points 3 months ago* (last edited 3 months ago) (2 children)

In terms of RAM it's not impossible, my current little server has 192GB of RAM installed.

Pic from TrueNAS

The VRAM would be quite the hurdle though, I'm curious on it's requirements for VRAM

Edit: Moving data in anticipation of a hardware migration ATM so basically none of the services are running.

[–] [email protected] 6 points 3 months ago (1 children)

VRAM would be 810Gb/403Gb/203Gb for FP16/FP8/INT4 for interferrence, according to their website.

[–] [email protected] 4 points 3 months ago

Hot damn that's a lot! They ain't messing around with that requirement.

My current server has 32 MB of VRAM. Yes, MB not GB. Once I finish the hardware migration it's going to 8GB but that's not even a drop in the bucket compared to that requirement.

[–] [email protected] 14 points 3 months ago (2 children)

That's not a little server.

[–] [email protected] 3 points 3 months ago (2 children)

You can have that much RAM with consumer ddr5.

[–] [email protected] 1 points 3 months ago

4x64gb udimms would cost over $1000.

[–] [email protected] 4 points 3 months ago

Yes but you can't call it a little amount.

[–] [email protected] 11 points 3 months ago (1 children)

It's pretty old hardware to say the least, it's also really proprietary. (Old Dell PowerEdge T610)

My hardware migration I'm currently in the midst of is going to bring it more in line with my typical use case for it.

Basically taking it down from 192 GB of ECC DDR3 to around 32 GB (maybe 64 GB) of DDR4 RAM. Also down to a single CPU rather than dual socket.

[–] [email protected] 3 points 3 months ago (1 children)

Old Epyc boards are super cheap on eBay. 8 channels of ddr4 and 80-100 lanes of pcie for nvme on an ATX mobo. You pay for the idle power consumption, but it's pretty cheap overall.

[–] [email protected] 2 points 3 months ago

I'm just going with a Ryzen 1600x system because I have one on hand

My current system has a pair of 12 thread Xeon CPUs and I really don't need them, plus I'm wanting to go with normal consumer hardware for the new system for repairability reasons