M12-004M12: AI Security Red-Teaming & AI PM PlaybooksAI Security Red-Team & IsolationMedium
⚡Flashcard Drill➔Mastery:

AI Security & FinOps: Stopping Token-Exhaustion DoS & Sponge Attacks

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M12-005 →
📐 Mathematical Definition
Per-Tenant TokenBucket(Ccap,RTPM)∧Spend Cap  ⟹  Zero Bill Shock\text{Per-Tenant } \text{TokenBucket}(C_{\text{cap}}, R_{\text{TPM}}) \land \text{Spend Cap} \implies \text{Zero Bill Shock}
⚡ One-Sentence Core Answer
Protect shared LLM clusters from Economic Denial-of-Wallet (DoW) and long-output sponge attacks via Redis TPM token buckets, per-run dollar ceilings, and max-token clamps.

📌 Key Takeaways

  • •
    Denial-of-Wallet Vector: tiny HTTP concurrency sending 32K contexts asking for maximum-length trivial lists bankrupts token budgets fast
  • •
    Defense 1 (Pre-Reserve TPM Bucket): reserve worst-case input+max_output upfront in Redis Lua and reconcile actual usage cleanly post-stream
  • •
    Defense 2 (Hard Run Clamps): bound max_output_tokens, tool steps, and per-run dollar ceilings by subscription tier
  • •
    Defense 3 (Anomaly Circuit Breaker): trip automatic 8B-fallback throttling when 5-minute tenant burn rate spikes >5x baseline

📐 Mathematical Derivations

Enforce TPM token quotas alongside RPM counts and clamp client-supplied max_tokens server-side per subscription tier.

🏭 Production Trade-offs

Tighten anonymous/free-tier max_tokens (e.g. 512) and gateway rate limits to block automated proxy scrapers from draining paid compute.
⚠️ Common Interview Pitfalls
  • ✕
    Configuring standard HTTP request-per-minute limits without metering prompt/output tokens
  • ✕
    Trusting client-supplied max_tokens fields cleanly without server-side tier caps
🎯 Interviewer Follow-up Inquiries
  • ?
    How do burst credits and async priority queues smooth legitimate enterprise batch spikes without flat 429 drops?
  • ?
    How does an internal Agent orchestrator share live remaining budget signals cleanly with edge API gateways?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM12-003: AI Red-Team: Defending Base64, Homoglyph & Low-Resource Jailbreak Evasions📋 Back to BankNext →M12-005: AI Supply Chain Sec: Defending Pickle RCE, Weight Poisoning & Rogue MCP Servers