this post was submitted on 11 Jun 2024
1 points (100.0% liked)
AI
4142 readers
1 users here now
Artificial intelligence (AI) is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. The distinction between the former and the latter categories is often revealed by the acronym chosen.
founded 3 years ago
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Being able to run 7B quality models on your phone would be wild. It would also make it possible to run those models on my server (which is just a mini pc), so I could connect it to my Home Assistant voice assistant, which would be really cool.
Something similar to this already kinda exists on HF with the 1.58 bit quantisation which seem to get very similar performance to the original Llama 3 8B model. That's essentially a two bit quanitsation with reasonable performance!
That's really interesting, gonna try out how well it runs