RAG Security Detection

Retrieval-Augmented Generation (RAG) applications face unique security challenges. Oculum detects vulnerabilities in RAG pipelines including data exfiltration, poisoning, and PII exposure.

RAG Security Risks

RAG applications combine retrieved documents with LLM prompts, creating attack surfaces:

• Data Exfiltration — Sensitive data leaked via AI responses
• Document Poisoning — Malicious content in the knowledge base
• PII Exposure — Personal data in embeddings or responses
• Query Injection — Manipulated searches returning wrong documents
• Embedding Leakage — Reconstructing data from embedding vectors

---

Detectors

rag_data_exfiltration

Severity: Critical

Detects patterns that could leak sensitive data through RAG responses.

Triggers on:

• Retrieved documents exposed in raw form
• No filtering between retrieval and response
• System prompts visible in output
• Internal document IDs exposed

Example Vulnerable Code:

// VULNERABLE: Raw documents in response
const docs = await vectorStore.similaritySearch(query);
const response = await llm.complete(`
  Based on these documents: ${JSON.stringify(docs)}
  Answer: ${userQuestion}
`);
return response; // May contain sensitive doc content

---

rag_poisoning

Severity: High

Detects insufficient validation of documents in the knowledge base.

Triggers on:

• User-uploaded documents without scanning
• External sources ingested without validation
• No content filtering on indexed documents

Example Vulnerable Code:

// VULNERABLE: Unvalidated document ingestion
async function ingestDocument(doc: Document) {
  const chunks = await splitter.splitDocument(doc);
  const embeddings = await embedder.embed(chunks);
  await vectorStore.add(embeddings); // No validation!
}

---

rag_pii_exposure

Severity: High

Detects potential PII leakage in RAG pipelines.

Triggers on:

• No PII filtering before embedding
• Personal data in retrieved context
• Missing anonymization in responses

Example Vulnerable Code:

// VULNERABLE: PII in knowledge base
const docs = await loadCustomerEmails(); // Contains PII
await vectorStore.addDocuments(docs); // No redaction

---

rag_query_injection

Severity: Medium-High

Detects query manipulation vulnerabilities.

Triggers on:

• User input directly in similarity search
• No query sanitization
• Metadata filters from user input

Example Vulnerable Code:

// VULNERABLE: Direct user input in query
const results = await vectorStore.similaritySearch(userQuery, {
  filter: { department: userSelectedDepartment } // Unvalidated
});

---

rag_embedding_security

Severity: Medium

Detects embedding-related security issues.

Triggers on:

• Embeddings stored without encryption
• Embedding model from untrusted source
• No access control on vector database

---

rag_chunk_boundary

Severity: Medium

Detects issues with document chunking that could expose sensitive data.

Triggers on:

• Sensitive data split across chunks
• No chunk-level access control
• Inconsistent chunking revealing structure

---

Remediation

Prevent Data Exfiltration

// SAFE: Filter and sanitize before response
async function ragQuery(question: string) {
  const docs = await vectorStore.similaritySearch(question);

  // Filter sensitive content
  const sanitizedDocs = docs.map(doc => ({
    content: redactSensitiveInfo(doc.content),
    metadata: filterMetadata(doc.metadata)
  }));

  const response = await llm.complete({
    context: sanitizedDocs,
    question: question,
    systemPrompt: "Never reveal document sources or internal data."
  });

  // Post-filter the response
  return filterResponse(response);
}

Prevent Document Poisoning

// SAFE: Validate before ingestion
async function ingestDocument(doc: Document, source: string) {
  // Validate source
  if (!trustedSources.includes(source)) {
    throw new Error('Untrusted source');
  }

  // Scan content
  const scanResult = await contentScanner.scan(doc.content);
  if (scanResult.hasInjectionPatterns) {
    throw new Error('Potential injection detected');
  }

  // Sanitize
  const sanitized = sanitizeDocument(doc);

  // Then ingest
  await vectorStore.addDocument(sanitized);
}

Protect PII

// SAFE: Redact PII before embedding
import { PiiRedactor } from './pii';

async function embedDocument(doc: Document) {
  // Detect and redact PII
  const redacted = await PiiRedactor.redact(doc.content, {
    types: ['email', 'phone', 'ssn', 'name', 'address']
  });

  // Store mapping for authorized retrieval
  await piiStore.storeMapping(doc.id, redacted.mappings);

  // Embed redacted content
  return embedder.embed(redacted.content);
}

Secure Query Handling

// SAFE: Validate and sanitize queries
async function search(userQuery: string, userFilters: object) {
  // Sanitize query
  const sanitizedQuery = sanitizeQuery(userQuery);

  // Validate filters against user permissions
  const allowedFilters = validateFilters(userFilters, user.permissions);

  // Apply access control
  const results = await vectorStore.similaritySearch(sanitizedQuery, {
    filter: {
      ...allowedFilters,
      accessLevel: { $lte: user.accessLevel }
    }
  });

  return results;
}

---

Architecture Recommendations

Tiered Access Control

┌─────────────┐
│   User      │
└─────┬───────┘
      │ Query
      ▼
┌─────────────┐
│  Query      │  ← Sanitization & validation
│  Processor  │
└─────┬───────┘
      │
      ▼
┌─────────────┐
│  Vector DB  │  ← Access control filters
│  (filtered) │
└─────┬───────┘
      │ Retrieved docs
      ▼
┌─────────────┐
│  Content    │  ← PII redaction, sensitivity filter
│  Filter     │
└─────┬───────┘
      │
      ▼
┌─────────────┐
│  LLM        │  ← System prompt with guardrails
└─────┬───────┘
      │
      ▼
┌─────────────┐
│  Response   │  ← Output validation
│  Filter     │
└─────────────┘

Separation of Concerns

• Public Knowledge Base — General information, no access control
• Internal Knowledge Base — Requires authentication
• Sensitive Knowledge Base — Role-based access, audit logging

---

Common Vulnerabilities

---

Related

• Prompt Injection — Injection via RAG documents
• Agent Safety — RAG-powered agents
• Other Detectors — PII and data exposure