SSE Streaming & Backpressure Control

Server-Sent Events (SSE) Streaming & Reactive Backpressure Architecture is the standard high-performance protocol transmitting token-by-token LLM auto-regressive completions to client browsers over persistent unidirectional streams; leveraging native HTTP/1.1 or HTTP/2 (`Content-Type: text/event-stream`), SSE is significantly more lightweight than WebSocket and features built-in browser reconnection using `Last-Event-ID`; under massive concurrency, API gateways enforce Reactive Backpressure control: if a client's network buffer saturates, the gateway signals upstream vLLM/TGI inference engines to pause token generation, preventing buffer overflow and gateway OOM crashes.

Real-time typewriter chat interfaces (ChatGPT), streaming Agent internal thoughts/reasoning logs, and live code completion.

Blocking HTTP requests force users into 10-30 second blank-screen waits (ruining Time-To-First-Token, TTFT); SSE delivers first tokens within 200ms, while reactive backpressure protects gateway infrastructure from memory exhaustion under heavy load.

Compare SSE vs WebSocket across connection overhead, bidirectional capabilities, Nginx reverse proxy buffering, and reconnection primitives?

Explain why configuring `proxy_buffering off;` in Nginx reverse proxies is strictly mandatory to prevent token buffering in SSE streams?

How do async Python generators (AsyncGenerator) in vLLM apply backpressure to pause GPU token scheduling when clients lag?

Design Layer-7 load balancing (Envoy / HAProxy) for long-lived SSE connections to prevent connection skew and hot-spotting?

How to propagate client disconnect signals via HTTP cancellation tokens to immediately abort active GPU KV-Cache generation?