A secure RAG implementation needs seven controls, built in this order: an authenticated vector store, sanitized and attested ingestion, document-level authorization enforced at query time, injection scanning of retrieved chunks, prompt assembly that marks retrieved text as untrusted, output grounding and citation checks, and least-privilege tool access. Skip the authorization filter and nothing downstream compensates.
The order matters because each layer assumes the one before it held, and none is sufficient alone. OWASP’s LLM01:2025 Prompt Injection entry notes that RAG does not fully mitigate prompt injection, and LLM08:2025 Vector and Embedding Weaknesses adds the retrieval-specific failures: cross-tenant leakage, embedding inversion and corpus poisoning. The PoisonedRAG paper reports a 90% attack success rate from five malicious texts per target question in a knowledge database of millions of texts. For the threat model, see secure RAG architecture best practices; what follows is the build sequence.
Step 1: Lock down the vector store before loading data
Start with the store, because a writable index undercuts every later control. An attacker who can write chunks can also write their ACL metadata. Self-hosted defaults can be open: Qdrant’s security documentation states that “by default, all self-deployed Qdrant instances are not secure.”
The pattern is credential separation. Only the ingestion service holds a write-capable key. The query path, which is the code an injected prompt can influence, gets a read-only key. Qdrant supports a separate read_only_api_key for this, and its JWT-based access control can scope tokens to specific collections with an expiry. Enable TLS, keep the database off the public internet, and route every query through an application tier that knows who the user is. Then verify it: from the inference service’s network position and credentials, attempt an upsert. It should fail.
Step 2: Sanitize and attest every document at ingestion
Ingestion is where indirect prompt injection enters the corpus. OWASP’s LLM08 entry gives the canonical case: a resume with instructions hidden in white-on-white text that a RAG screening system reads and follows. Its fix is text extraction that ignores formatting and detects hidden content, plus validation of every document before it enters the knowledge base.
A hardened ingestion job does four things:
- Extracts text with a parser that flags hidden text and metadata fields, then strips or quarantines them per source.
- Attaches provenance to every chunk: source URI, content hash, ingesting principal, timestamp, classification and the source system’s ACL.
- Generates embeddings server-side. Accepting client-supplied vectors lets the caller position content anywhere in the embedding space.
- Runs an injection classifier once per chunk, where latency is cheap, and quarantines hits for review.
Treat the vectors as sensitive too. In Text Embeddings Reveal (Almost) As Much As Text, Morris et al. recover 92% of 32-token inputs exactly from their embeddings. Embeddings of confidential documents need the same encryption, access control and retention as the documents.
Step 3: Enforce authorization inside the retrieval query
Authorization has no downstream substitute. Once an unauthorized chunk enters the context window, output filtering can only hope to catch the leak. OWASP LLM08’s cross-group scenario describes exactly this: one group’s embeddings returned for another group’s queries in a shared vector database.
The control is a pre-filter on ACL metadata, applied by the retrieval service with principals resolved server-side from the authenticated session, never from request parameters or model output. Azure AI Search documents this as a security filter pattern: a filterable group_ids field on each document and a query filter such as group_ids/any(g:search.in(g, 'group_id1, group_id2')). Microsoft’s docs note that the principal is just a string in a filter expression, with no authentication behind it, so the correctness of the group list is entirely your application’s responsibility. Azure also offers built-in document-level ACL support for sources that can use it.
For multi-tenant stores, Qdrant’s multitenancy guide recommends one shared collection with a tenant payload field indexed with is_tenant: true, reserving separate collections for the limited set of tenants that need strict isolation.
Plan for ACL drift. When permissions change in the source system, indexed chunks keep the old ACL until re-indexed, so run permission sync on a shorter schedule than content refresh.
Step 4: Scan retrieved chunks before they reach the prompt
Ingestion scanning misses documents indexed before a classifier update and content from live sources such as web search or email. A second scan at retrieval time covers both. Azure AI Content Safety’s Prompt Shields has a document-attack mode for this; per Microsoft’s docs, the standalone API analyzes a user prompt and up to five documents per call, so a top-k of 10 needs two calls. NVIDIA NeMo Guardrails exposes the same hook as retrieval rails, which can filter or transform chunks before they are added to the prompt.
Check how your guardrail actually treats retrieved text. In AWS Bedrock Guardrails, content passed with the grounding_source qualifier is excluded from every other policy, including prompt attack detection, unless you also add guard_content. A team that wires up grounding checks can end up with retrieved chunks that are never scanned for injection. Detection is probabilistic regardless: prompt injection detection methods covers where classifiers miss, and building your own guardrail eval set shows how to measure false positives on your corpus before switching from annotate to block.
Step 5: Mark retrieved text as data in the prompt
OWASP LLM01 lists segregating and identifying external content as a core mitigation. One well-documented technique is Microsoft’s spotlighting research (Hines et al.), which compared three variants. Delimiting roughly halved attack success but can be forged once an attacker learns the delimiters. Datamarking, which interleaves a marker token through the retrieved text, cut attack success on GPT-3.5-Turbo summarization from about 50% to 3.10%. Base64 encoding was most effective but hurt task performance on weaker models, so the authors recommend datamarking as the minimum. Azure now offers spotlighting as a preview that base64-encodes documents, which raises token counts.
Pair marking with limits. Cap chunk count and size per query, keep retrieved content out of the system turn, and keep secrets out of the system prompt on the assumption it will leak. Indirect prompt injection explained covers why instruction hierarchy alone does not hold.
Step 6: Validate outputs against what was retrieved
Require the model to cite chunk IDs in a structured format, then check in code that every cited ID was in the retrieved set and that the user was authorized for it. Hallucinated or injected citations become a hard failure. This is the deterministic format validation OWASP LLM01 recommends.
For semantic grounding, Bedrock’s contextual grounding check scores responses for grounding and relevance against the source and query, with thresholds configurable from 0 to 0.99. Read the limits first: AWS caps grounding source at 100,000 characters, query at 1,000 and response at 5,000, does not support conversational QA or chatbot use cases, and warns that with streaming an irrelevant response can reach the user before it is flagged. That chatbot exclusion rules it out for many RAG assistants.
Then sanitize rendering. Strip or allowlist URLs and markdown images in output, because a rendered image link can carry retrieved data to an attacker’s server. Output filtering architecture covers filter placement and PII redaction.
Step 7: Strip privileges the model does not need
If the RAG system can call tools, a successful injection becomes an action rather than a bad answer. OWASP LLM01 recommends least privilege, handling functions in code rather than through the model, and human approval for high-risk operations. In a RAG agent that means a tool allowlist per use case, read-only tools by default, and confirmation gates on anything that writes, sends or spends once untrusted content is in context.
How do you test a secure RAG implementation?
Test it with canary documents in a staging corpus and assertions in CI. Seed documents carrying injected instructions, hidden text and restrictive ACLs, then assert that low-privilege identities never retrieve the restricted ones, that injected instructions never execute, and that ungrounded answers are blocked. Rerun on every change to retrieval, chunking or models.
Log enough to reconstruct an incident. OWASP LLM08 recommends immutable logs of retrieval activity; per-query chunk IDs and retrieval scores are the minimum, and LLM audit logging lays out a field scheme. Building an internal adversarial testing pipeline covers automating the attack side.
Residual risk remains. A poisoned document from an authorized source that states false facts with no imperative language passes every control above; that is the knowledge-corruption attack PoisonedRAG demonstrates. For high-impact corpora, the backstops are provenance review, approval gates on ingestion and human review of answers that drive decisions.
FAQ
is rag secure by default
No. RAG adds a retrieval corpus the model treats as trusted evidence, and common components ship open: Qdrant’s docs state self-deployed instances are not secure by default. OWASP’s 2025 LLM Top 10 also notes RAG does not fully mitigate prompt injection. Security comes from authentication, retrieval-time authorization, chunk scanning and output validation added deliberately.
should rag access control happen before or after retrieval
Before, inside the retrieval query itself. Filter on ACL metadata using groups resolved from the authenticated session, so unauthorized chunks never enter the context window. Filtering the generated answer afterward is not equivalent, because once restricted text is in context the model can paraphrase, summarize or leak it in ways an output filter will not reliably catch.
which owasp llm top 10 risks apply to rag
Two apply most directly: LLM01:2025 Prompt Injection, which covers instructions hidden in retrieved documents, and LLM08:2025 Vector and Embedding Weaknesses, which covers cross-tenant leakage, embedding inversion and poisoning of the knowledge base. Output validation and least-privilege tool access address the downstream impact when either one succeeds against your retrieval layer.
can a guardrail stop rag poisoning
Partly. Injection classifiers at ingestion and retrieval can flag poisoned chunks that carry instructions, but not documents that simply assert false facts. PoisonedRAG reached a 90% attack success rate with five malicious texts per target question, and its authors found the defenses they evaluated insufficient. Provenance controls and ingestion approval carry the rest of the load.
Related across the network
- RAG Pipeline Security Best Practices: A Checklist — aisecreviews.com
- AI Jailbreak Detection Tools: Six Options Worth Shortlisting — jailbreakdb.com
