this post was submitted on 19 May 2025
19 points (88.0% liked)

Free Open-Source Artificial Intelligence

3417 readers
1 users here now

Welcome to Free Open-Source Artificial Intelligence!

We are a community dedicated to forwarding the availability and access to:

Free Open Source Artificial Intelligence (F.O.S.A.I.)

More AI Communities

LLM Leaderboards

Developer Resources

GitHub Projects

FOSAI Time Capsule

founded 2 years ago
MODERATORS
 

I don't have many specific requirements, and GPT4All is working mostly well for me so far. That said, my latest use case for GPT4All is to help me plan a new Python-based project with examples as code snippets, and it lacks a specific quality of life feature, that is the "Copy Code" button.

There is an open issue on GPT4All's GitHub, but as there is no guarantee that feature will ever be implemented, I thought I'd take this opportunity to explore if there are any other tools out there like GPT4All that offer a ChatGPT-like experience in the local environment. I'm neither a professional developer nor a sysadmin, so a lot of self hosting guides go over my head, which is what drew me to GPT4All in the first place, as it's very accessible to non-developers like myself. That said, I'm open to suggestions and willing to learn new skills if that's what it takes.

I'm running on Linux w/ AMD hardware: Ryzen 7 5800X3D processor + Radeon RX 6750 XT.

Any suggestions? Thanks in advance!

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 2 points 2 days ago

You can squeeze a lot more performance out with a newer framework and a model tailored for your GPU and task.

I'd recommend:

Kobold.cpp rocm, follow the quick-install guide here: https://github.com/YellowRoseCx/koboldcpp-rocm/?tab=readme-ov-file#quick-linux-install

Download this quantization, which fits in your VRAM pool nicely and is specifically tuned for coding and planning, select it in kobold.cpp: https://huggingface.co/mradermacher/Qwen3-14B-Esper3-i1-GGUF/blob/main/Qwen3-14B-Esper3.i1-IQ4_NL.gguf

Use the "corporate" UI in kobold.cpp in your browser. If that doesn't work well, kobold.cpp also works as a generic OpenAI endpoint, which you can access from pretty much any app, like https://openwebui.com/