RAG is basically like telling an LLM “look here for more info before you answer” so it can check out local documents to give an answer that is more relevant to you.

You just search “open web ui rag” and find plenty kf explanations and tutorials

Have you tried RAG? I believe that they are actually pretty good for searching and compiling content from RAG.

So in theory you could have it connect to all of you local documents and use it for quick questions. Or maybe connected to your signal/whatsapp/sms chat history to ask questions about past conversations

Yeah, personally I just looked for second hand high vram gpus and waited. I got 2 titan Xp (12gb vram) for only $180 each.

I know there is all of that AI hate, which i’m all for. But taking models to run locally does not benefit the AI companies. If anything this is the way to make something that is actually good out of that hot mess.

You could use an llm with an mcp to the local filesystem and hope it can do it for you

Not the best per se but I really liked Severance

What is this weird ad post? F u

About as much as I trust other drivers on the road.

As in I give it the benefit of the doubt but if something seems off I take precautions while monitoring and if it seems dangerous I do my best to avoid it.

In reality it means that I rarely check it but if anything seems off I remove it and if I have the time and energy I further check the actual code.

My general approach is minimalism, so I don’t use that many unknown/small projects to begin with.

Oh no! Not the business model!