LocalLLaMA

2882 readers

27 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

founded 2 years ago

MODERATORS

[email protected]

How do I get started with RAG (ideally with llama.cpp)? (lemmy.dbzer0.com)

submitted 22 hours ago by [email protected] to c/[email protected]

1 comments fedilink hide all child comments

I would like my model to know the code libraries I use and help me write code with them. I use llama.cpp's server and web UI for inference, but I have no clue how to get started with RAG, since it seems it is not natively supported with llama.cpp's server implementation. It almost looks like I would need to code my own agent.

I am not interested in commercial offerings or APIs. If you use RAG, how do you do it?

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 4 points 19 hours ago

You can use something like Anything LLM for RAG:

https://github.com/Mintplex-Labs/anything-llm

It works with local models.

https://docs.anythingllm.com/agent/usage#what-is-rag-search-and-how-to-use-it