RAG
What is RAG?
RAGRetrieval-Augmented Generation: an LLM pattern that fetches relevant documents from a knowledge store at query time and injects them into the prompt to ground responses.
RAG augments a Large Language Model with an external retrieval step. At inference, the user's query is embedded, a vector or keyword index returns the most relevant documents, and those documents are concatenated into the prompt so the LLM can cite or reason over them. RAG reduces hallucinations and lets models use private or fresh data without retraining. Security-wise it creates new attack surface: prompt injection from documents (indirect prompt injection), data poisoning of the corpus or vector store, exfiltration through model outputs, access-control mistakes when multiple tenants share an index, and embedding inversion attacks. Hardened RAG pipelines isolate untrusted content, enforce per-document access checks, sanitize inputs, monitor retrieved snippets, and apply output guardrails.
● Examples
- 01
An enterprise chatbot answers HR questions by retrieving policy PDFs from a vector store.
- 02
A malicious wiki page contains hidden instructions that hijack a RAG assistant via indirect prompt injection.
● Frequently asked questions
What is RAG?
Retrieval-Augmented Generation: an LLM pattern that fetches relevant documents from a knowledge store at query time and injects them into the prompt to ground responses. It belongs to the AI & ML Security category of cybersecurity.
What does RAG mean?
Retrieval-Augmented Generation: an LLM pattern that fetches relevant documents from a knowledge store at query time and injects them into the prompt to ground responses.
How does RAG work?
RAG augments a Large Language Model with an external retrieval step. At inference, the user's query is embedded, a vector or keyword index returns the most relevant documents, and those documents are concatenated into the prompt so the LLM can cite or reason over them. RAG reduces hallucinations and lets models use private or fresh data without retraining. Security-wise it creates new attack surface: prompt injection from documents (indirect prompt injection), data poisoning of the corpus or vector store, exfiltration through model outputs, access-control mistakes when multiple tenants share an index, and embedding inversion attacks. Hardened RAG pipelines isolate untrusted content, enforce per-document access checks, sanitize inputs, monitor retrieved snippets, and apply output guardrails.
How do you defend against RAG?
Defences for RAG typically combine technical controls and operational practices, as detailed in the full definition above.
What are other names for RAG?
Common alternative names include: Retrieval-Augmented Generation, Grounded generation.
● Related terms
- ai-security№ 1198
Vector Database Security
The set of controls that protect vector databases used by AI systems from data leakage, poisoning, tenant cross-talk, and supply-chain or operational compromise.
- ai-security№ 376
Embedding Attacks
A class of attacks against AI embedding vectors that recover, alter, or abuse the original input or its semantics, including embedding inversion and similarity-based poisoning.
- ai-security№ 866
Prompt Injection
An attack that overrides an LLM's original instructions by smuggling adversarial text into the prompt, causing the model to ignore safeguards or execute attacker-chosen actions.
- ai-security№ 031
AI Model Card
A standardised document, introduced by Margaret Mitchell and colleagues in 2018, that describes a machine-learning model's intended use, training data, performance, limitations, and ethical considerations.
- ai-security№ 281
Data Poisoning
An attack on a machine-learning system in which adversaries inject, alter, or relabel training data so the resulting model behaves incorrectly or contains hidden backdoors.