Free Open-Source Artificial Intelligence

You can squeeze a lot more performance out with a newer framework and a model tailored for your GPU and task.

I'd recommend:

Kobold.cpp rocm, follow the quick-install guide here: https://github.com/YellowRoseCx/koboldcpp-rocm/?tab=readme-ov-file#quick-linux-install

Download this quantization, which fits in your VRAM pool nicely and is specifically tuned for coding and planning, select it in kobold.cpp: https://huggingface.co/mradermacher/Qwen3-14B-Esper3-i1-GGUF/blob/main/Qwen3-14B-Esper3.i1-IQ4_NL.gguf

Use the "corporate" UI in kobold.cpp in your browser. If that doesn't work well, kobold.cpp also works as a generic OpenAI endpoint, which you can access from pretty much any app, like https://openwebui.com/

If you're looking for a web UI and a simple way to host one yourself, nothing beats the "llama.cpp" project. They include a "llama-server" program which hosts a simple web server (with a chat webapp) and OpenAI-compatible API endpoint. It now also supports multimodality (for models that support multimodality), meaning you can for example upload an image and ask the assistant to describe the image. An example command to set up such a web server would be:

$ llama-server --threads 6 -m /path/to/model.gguf

Or, for multimodality support (like asking an AI to describe an image), use:

$ llama-server --threads 6 --mmproj /path/to/model/mmproj-F16.gguf -m /path/to/model/model.gguf

VSCode with the open source Cline extension. Easily the best open option, works everywhere. Excellent coding and planning agent, I use it for everything

Koboldcpp

Page assist, runs in your browser and interfaces with ollama.

LM Studio although i've never tried the linux version.

OpenWebUI is a superb front-end and supports just about any backend that you think of (including Ollama for locally hosted LLMs) and has some really nice features like pipelines that can extend out its functionality however you might need. Definitely has the “copy code” feature built-in and outputs markdown for regular documentation purposes.