281GB
That's huge, I'm guessing we'll need to use a giant swap file?
this post was submitted on 10 Apr 2024
1 points (100.0% liked)
LocalLLaMA
2249 readers
1 users here now
Community to discuss about LLaMA, the large language model created by Meta AI.
This is intended to be a replacement for r/LocalLLaMA on Reddit.
founded 1 year ago
MODERATORS
You're right, but the model is also not quantized so is likely to be in 16bit floats. If you quantize it you can get substantially smaller models which run faster though may be somewhat less accurate.
Knowing that the 4 bit quantized 8x7B model gets downscaled to 4.1GB, this might be roughly 3 times larger? So maybe 12GB? Let's see.
It doesnt even have a license yet, or a blog post, anouncement, etc. To me it currently looks like someone just leaked their model, time will tell ...
They've done it that way for the previous models, too. I suppose it's to add a bit of "mystery" around it and give people some riddle to solve.
Likely they’re trying to get in before Llama 3 drops, because I suspect that’s all people will talk about for a fair bit.
Probably not. Since they're doing exactly this (drop a magnet link out of the blue) for the third time or so in a row... It's more likely a scheme than related to current happenings.