🏗️ System DesignID: sse-streaming-backpressure

SSE Streaming & Backpressure Control

Q: Compare SSE vs WebSocket across connection overhead, bidirectional capabilities, Nginx reverse proxy buffering, and reconnection primitives?

【SSE Streaming & Backpressure Control 核心考点 1】Compare SSE vs WebSocket across connection overhead, bidirectional capabilities, Nginx reverse proxy buffering, and reconnection primitives?。技术解析：Definition: Server-Sent Events (SSE) Streaming & Reactive Backpressure Architecture is the standard high-performance protocol transmitting token-by-token LLM auto-regressive completions to client browsers over persistent unidirectional streams; leveraging native HTTP/1.1 or HTTP/... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/sse-streaming-backpressure

Q: Explain why configuring `proxy_buffering off;` in Nginx reverse proxies is strictly mandatory to prevent token buffering in SSE streams?

【SSE Streaming & Backpressure Control 核心考点 2】Explain why configuring `proxy_buffering off;` in Nginx reverse proxies is strictly mandatory to prevent token buffering in SSE streams?。技术解析：Definition: Server-Sent Events (SSE) Streaming & Reactive Backpressure Architecture is the standard high-performance protocol transmitting token-by-token LLM auto-regressive completions to client browsers over persistent unidirectional streams; leveraging native HTTP/1.1 or HTTP/... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/sse-streaming-backpressure

Q: How do async Python generators (AsyncGenerator) in vLLM apply backpressure to pause GPU token scheduling when clients lag?

【SSE Streaming & Backpressure Control 核心考点 3】How do async Python generators (AsyncGenerator) in vLLM apply backpressure to pause GPU token scheduling when clients lag?。技术解析：Definition: Server-Sent Events (SSE) Streaming & Reactive Backpressure Architecture is the standard high-performance protocol transmitting token-by-token LLM auto-regressive completions to client browsers over persistent unidirectional streams; leveraging native HTTP/1.1 or HTTP/... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/sse-streaming-backpressure

Q: Design Layer-7 load balancing (Envoy / HAProxy) for long-lived SSE connections to prevent connection skew and hot-spotting?

【SSE Streaming & Backpressure Control 核心考点 4】Design Layer-7 load balancing (Envoy / HAProxy) for long-lived SSE connections to prevent connection skew and hot-spotting?。技术解析：Definition: Server-Sent Events (SSE) Streaming & Reactive Backpressure Architecture is the standard high-performance protocol transmitting token-by-token LLM auto-regressive completions to client browsers over persistent unidirectional streams; leveraging native HTTP/1.1 or HTTP/... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/sse-streaming-backpressure

Q: How to propagate client disconnect signals via HTTP cancellation tokens to immediately abort active GPU KV-Cache generation?

【SSE Streaming & Backpressure Control 核心考点 5】How to propagate client disconnect signals via HTTP cancellation tokens to immediately abort active GPU KV-Cache generation?。技术解析：Definition: Server-Sent Events (SSE) Streaming & Reactive Backpressure Architecture is the standard high-performance protocol transmitting token-by-token LLM auto-regressive completions to client browsers over persistent unidirectional streams; leveraging native HTTP/1.1 or HTTP/... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/sse-streaming-backpressure

SSE 长连接流式传输与背压控制

🎯Core Definition

Server-Sent Events (SSE) Streaming & Reactive Backpressure Architecture is the standard high-performance protocol transmitting token-by-token LLM auto-regressive completions to client browsers over persistent unidirectional streams; leveraging native HTTP/1.1 or HTTP/2 (`Content-Type: text/event-stream`), SSE is significantly more lightweight than WebSocket and features built-in browser reconnection using `Last-Event-ID`; under massive concurrency, API gateways enforce Reactive Backpressure control: if a client's network buffer saturates, the gateway signals upstream vLLM/TGI inference engines to pause token generation, preventing buffer overflow and gateway OOM crashes.

💡Use Cases

Real-time typewriter chat interfaces (ChatGPT), streaming Agent internal thoughts/reasoning logs, and live code completion.

⚡Key Problems Solved

Blocking HTTP requests force users into 10-30 second blank-screen waits (ruining Time-To-First-Token, TTFT); SSE delivers first tokens within 200ms, while reactive backpressure protects gateway infrastructure from memory exhaustion under heavy load.

🎯5 High-Frequency Exam Points

Compare SSE vs WebSocket across connection overhead, bidirectional capabilities, Nginx reverse proxy buffering, and reconnection primitives?

Explain why configuring `proxy_buffering off;` in Nginx reverse proxies is strictly mandatory to prevent token buffering in SSE streams?

How do async Python generators (AsyncGenerator) in vLLM apply backpressure to pause GPU token scheduling when clients lag?

Design Layer-7 load balancing (Envoy / HAProxy) for long-lived SSE connections to prevent connection skew and hot-spotting?

How to propagate client disconnect signals via HTTP cancellation tokens to immediately abort active GPU KV-Cache generation?

📖 In-depth Guide:📄 llm-rag-agent-system-design →

Updated 2026-08-14

🎯

Test Your Knowledge: Practice Questions for "SSE Streaming & Backpressure Control"

Single choice pitfall questions with instant feedback and mistake tracking.

🚀 Start Card Practice ➔

← Previous CardMulti-Tenant Partitioning & RBAC Next Card →Context Budgeting & Token Management

🔗 More System Design Knowledge Cards

RecSys Multi-Stage Funnel & 50ms SLA DSSM Two-Tower Retrieval YouTube DNN Candidate Generation Pre-Ranking Lightweight Architecture