Late to the party, I never got FOSAI working until I found LMStudio, but I have 2 questions:
-
Is there any way I could utilize my GPU, a Radeon RX6800M (12GB VRAM)? I got Mistral-7B doing 5 tokens/s but it's all running on the CPU.
-
Is there any model specifically for programming questions? This could be of immense help to my projects without having to ask ChatGPT.