Quick answer
To secure a RAG system, enforce the same permissions in retrieval that apply to the source documents: tag every chunk with its access rules and tenant, filter results by the authenticated user before they reach the model, keep internal and public content in separate indexes, and vet any user-submitted content before it is indexed.
Key takeaways
- Vector and Embedding Weaknesses is LLM08 in the OWASP Top 10 for LLM Applications.
- Permissions are usually lost at indexing time — carry them into chunk metadata.
- Filter retrieval by the user's identity before results reach the model, not after.
- Separate indexes for public, internal and per-tenant content are the simplest safe default.
RAG is how most business chatbots "know" about your products, policies and customers. It's also one of the easiest places to leak data, because a vector database does exactly what it's designed to do: return whatever is most similar to the question, from whatever you indexed.
Where RAG leaks happen
- Lost permissions: a SharePoint folder restricted to HR becomes a set of chunks with no access metadata.
- Mixed indexes: public help articles and internal runbooks share one collection.
- Multi-tenant bleed: one customer's uploaded files are retrievable by another customer's questions.
- Poisoned content: user-submitted content (reviews, tickets, forum posts) is indexed and carries indirect prompt injection.
- Stale content: documents deleted from the source system remain in the index.
How to secure your RAG pipeline
1. Carry permissions into metadata
At indexing time, attach the source document's access rules to every chunk: tenant ID, visibility (public/internal), groups or roles allowed. No metadata, no indexing.
2. Filter at query time, using the session
Apply a metadata filter on every search based on the authenticated user — never on values supplied by the model or the user's message. Restricted chunks should never enter the prompt.
3. Separate what should never mix
The simplest safe design: a public index for anonymous visitors, and separate internal or per-tenant indexes for authenticated use. Your public website chatbot should not be able to query internal content at all.
4. Vet what gets indexed
Review or sanitize user-generated content before indexing, and strip hidden text and instructions from HTML and documents.
5. Keep the index in sync
Propagate deletions and permission changes from source systems, and re-index on a schedule.
Quick test: As an anonymous visitor, ask your website chatbot about internal-sounding topics — "What's the on-call escalation process?" or "Summarize the latest board deck." Any detailed answer means internal content is reachable.
How CertiSafe AI tests for it
Vector and embedding weaknesses are a required control in our audit criteria. We probe retrieval across anonymous, authenticated and cross-tenant contexts and check whether any restricted content reaches the model's answers.
Frequently asked questions
What is RAG in AI?
Retrieval-augmented generation (RAG) is a technique where an AI system searches a knowledge base — usually a vector database of document chunks — and adds the most relevant results to the model's prompt so it can answer using your own content.
Why do RAG systems leak data?
Because documents are split into chunks and embedded without their original access controls. Semantic search then returns whatever is most similar to the question, regardless of who is asking — including internal or other customers' documents.
Can you filter RAG results after the model answers?
Post-answer filtering is unreliable because the model may already have summarized or paraphrased restricted content. Permission filtering should happen at retrieval time, so restricted chunks never enter the model's context.
Get certified against the leaks covered in this article
CertiSafe AI tests your live AI features every quarter and gives you a verifiable stamp for your website.