this post was submitted on 01 Feb 2024
1 points (100.0% liked)

Free Open-Source Artificial Intelligence

2886 readers
1 users here now

Welcome to Free Open-Source Artificial Intelligence!

We are a community dedicated to forwarding the availability and access to:

Free Open Source Artificial Intelligence (F.O.S.A.I.)

More AI Communities

LLM Leaderboards

Developer Resources

GitHub Projects

FOSAI Time Capsule

founded 1 year ago
MODERATORS
all 14 comments
sorted by: hot top controversial new old
[–] [email protected] 0 points 9 months ago

Note that he did not confirm is was mistral-medium. He says that's a retrained llama2-70B model, but hints that it is not the fully trained one. Sounds a bit like damage control but is not a 100% confirmation of the claim.

[–] [email protected] 0 points 9 months ago (1 children)

How do you "leak" something that's open-source?

[–] [email protected] 0 points 9 months ago

This model was not yet openly available

[–] [email protected] 0 points 9 months ago (3 children)

I have tried the version available at hugginchat and it's almost gpt4 level for programming tasks but not so much for general knowledge

[–] [email protected] 0 points 9 months ago (2 children)

Do you know if there are any plans to quantize it? I'd love to test it, but my 3090 can't handle 70b models without quantization, unfortunately.

[–] [email protected] 0 points 9 months ago

Only quantized versions of the model were leaked. If you see any unquantized version of it then it's something which was recreated from these, and not the original model. People have also requanted it from GGUF to EXL2 and probably other formats too.

[–] [email protected] 0 points 9 months ago

There are quantized versions on hugging face. There's a q2 version, but idk how well that performs

[–] [email protected] 0 points 9 months ago (1 children)

I can't find miqu-1-70b in huggingchat. Do I need to enable anything special anywhere?

[–] [email protected] 0 points 9 months ago (1 children)
[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (1 children)

Thats the model, but how do I get it into huggingchat? https://huggingface.co/chat/

Edit: thanks for the suggestion though. 🙂

[–] [email protected] 0 points 9 months ago

Yeah only have the 7b it seems on huggingchat, that's the one I was talking about.

Given how good the 7b I wouldn't be surprised if the 70b is better than gpt4 for programming-related chat.