RAG: connecting an LLM to your internal documents without leaks

July 15, 2026 · 3 min read · SOVALYX Technologies

Retrieval-augmented generation (RAG) lets a language model answer from your internal documents, without retraining the model and without exposing your data to a public AI. It is the most sensible first enterprise AI use case — provided you deal with two prerequisites that are too often ignored: access rights and document quality.

RAG explained without jargon

The principle fits in one sentence: instead of asking the model what it "knows", you hand it, at question time, the extracts of your documents needed to answer. Concretely, your documents are split into passages, converted into numerical representations and stored in a search index. When a user asks a question, the system retrieves the most relevant passages, attaches them to the question, and the model generates an answer grounded in those extracts — ideally citing its sources.

The most important consequence is widely misunderstood: the model does not learn your documents. The knowledge stays in the index, next to the model. Update a document and the answers change immediately; remove it and it drops out of scope. Unlike retraining, the setup is reversible, auditable and governable — you always know where an answer came from.

Why it is the right first AI project

RAG ticks every box of a good pilot. The value is immediate and concrete: internal support, procedures, technical documentation, recurring HR or legal questions, pre-sales preparation. Answers are sourced, therefore verifiable — the user can open the cited document, which limits silent hallucination. It requires neither costly retraining nor a technological bet: the scope grows progressively, corpus by corpus. And unlike generic SaaS AI assistants, it answers in your context, with your vocabulary and your rules.

There is a side benefit nobody advertises: RAG reveals the true state of your documentation. Duplicates, contradictory versions and obsolete procedures become visible from the very first questions. Many organisations discover that the documentation clean-up was the real project.

Two prerequisites before the technology

Access rights first

The main risk of an internal RAG is not leakage to the outside: it is internal leakage. An index built with a privileged account and queryable by everyone gives every employee search access to payroll, HR files or confidential projects. The rule is simple: retrieval must filter by the rights of the user asking the question, not those of the account that did the indexing. Source permissions must be propagated into the index and verified with targeted tests, and every question and answer must be logged.

Document quality second

A RAG answers with whatever you feed it. If the corpus mixes three versions of the same procedure, it will cite the wrong one with the confidence of an expert. Minimal curation: identify authoritative sources, assign owners, date the documents, weed out drafts. Better to start with a small, clean corpus than with "the whole network share".

Private hosting: where the model and the index run

A RAG index concentrates your most useful content — often more readably than the source systems — and every question sent to the model carries extracts of internal documents. Sending those requests to a public API recreates exactly the problem you set out to avoid: confidential data leaves your perimeter, as we analyse in confidentiality and public AI. A private LLM hosted on your own infrastructure or on a trusted private cloud keeps questions, extracts and answers within a controlled perimeter — and today's open models are more than sufficient for this use case. That is the architecture SOVALYX implements: internal model, access-controlled index, no dependency on any public AI.

Checklist before launching a RAG pilot

1. A precise use case, identified users and a list of typical questions.
2. A small, up-to-date pilot corpus with designated owners.
3. Access rights replicated in the index and verified by leakage tests.
4. Full logging of questions and answers.
5. A model and an index hosted in a perimeter you control.
6. Success criteria set before rollout: useful-answer rate, adoption, time saved.
7. A corpus extension plan — and someone to frame the pilot with.

🧰 The companion tool: Are you ready for the AI Act? — free · 2 minutes.

Reviewed and optimised by AI.