Prompt Injection Defense

Prompt Injection is a severe security vulnerability where crafted inputs hijack LLM control flow or override safety alignments, encompassing Direct Prompt Injection (Jailbreaking via user prompts) and Indirect Prompt Injection (embedding malicious instructions inside external web pages, PDFs, or RAG passages triggered when ingested by an Agent); defense architectures include strict XML delimiter sandboxing, formal Instruction Hierarchy enforcement, input pre-screening classifiers, and Dual-LLM privileged-isolated execution frameworks.

Autonomous web-browsing agents, enterprise pipelines ingesting untrusted third-party files (resumes, emails), and privileged API execution tools.

Indirect prompt injections can silently command agents to exfiltrate private DB records to external attacker endpoints (OWASP Top 1 for LLMs); multi-tiered defense establishes deep boundary isolation, neutralizing injected text as passive data.

Contrast the attack vectors and blast radiuses between Direct Prompt Injection and Indirect Prompt Injection?

Explain the security design of the Dual-LLM architecture (Quarantine LLM parses untrusted text, Privileged LLM executes actions)?

Explain how adversarial suffix attacks (GCG gradient attacks) break safety alignment and how input perplexity filtering detects them?

How to use dynamic cryptographic Nonces to tag input boundaries and prevent attackers from forging XML closing tags?

How does Human-in-the-Loop (HITL) mandatory approval intercept high-risk tool operations triggered by malicious injections?