this post was submitted on 25 Jul 2024
655 points (100.0% liked)

196

16484 readers
1777 users here now

Be sure to follow the rule before you head out.

Rule: You must post before you leave.

^other^ ^rules^

founded 1 year ago
MODERATORS
 
you are viewing a single comment's thread
view the rest of the comments
[โ€“] [email protected] 6 points 3 months ago (1 children)

VRAM would be 810Gb/403Gb/203Gb for FP16/FP8/INT4 for interferrence, according to their website.

[โ€“] [email protected] 4 points 3 months ago

Hot damn that's a lot! They ain't messing around with that requirement.

My current server has 32 MB of VRAM. Yes, MB not GB. Once I finish the hardware migration it's going to 8GB but that's not even a drop in the bucket compared to that requirement.