this post was submitted on 25 Jul 2024
654 points (100.0% liked)
196
16190 readers
2140 users here now
Be sure to follow the rule before you head out.
Rule: You must post before you leave.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I doubt this person actually had a computer than could run the 405b model. You need over 200gb of ram, let alone having enough vram to run it with gpu acceleration.
In terms of RAM it's not impossible, my current little server has 192GB of RAM installed.
Pic from TrueNAS
The VRAM would be quite the hurdle though, I'm curious on it's requirements for VRAM
Edit: Moving data in anticipation of a hardware migration ATM so basically none of the services are running.
That's not a little server.
It's pretty old hardware to say the least, it's also really proprietary. (Old Dell PowerEdge T610)
My hardware migration I'm currently in the midst of is going to bring it more in line with my typical use case for it.
Basically taking it down from 192 GB of ECC DDR3 to around 32 GB (maybe 64 GB) of DDR4 RAM. Also down to a single CPU rather than dual socket.
Old Epyc boards are super cheap on eBay. 8 channels of ddr4 and 80-100 lanes of pcie for nvme on an ATX mobo. You pay for the idle power consumption, but it's pretty cheap overall.
I'm just going with a Ryzen 1600x system because I have one on hand
My current system has a pair of 12 thread Xeon CPUs and I really don't need them, plus I'm wanting to go with normal consumer hardware for the new system for repairability reasons