SSE Streaming Gateway & Backpressure

The Server-Sent Events (SSE) Streaming Gateway Architecture & Adaptive Backpressure Flow Control handles millions of concurrent long-lived streaming connections while insulating GPU inference clusters from slow-client bottlenecks; key elements: 1) SSE Unidirectional Pipe: utilizing HTTP/2 chunked transfer with `Content-Type: text/event-stream` pushing incremental delta tokens formatted as `data: {...}\n\n`; 2) Reactive Backpressure: when a client on weak cellular networks lags and saturates TCP write buffers, the gateway's bounded async ring-buffers throttle consumption from GPU engines, while client connection drops instantly emit `Abort` cancellation signals to terminate the forward generation loop, immediately recycling GPU KV cache blocks.

Real-time typewriter interactive chat interfaces, streaming audio/text GenAI assistants, and enterprise API gateways.

Full-response buffering forces users to wait seconds before reading, while disconnected clients waste GPU cycles generating unconsumed tokens; SSE with backpressure delivers instant TTFT and stops zombie generations.

Diagram the end-to-end SSE streaming dataflow and Abort Signal propagation across Client, Envoy Gateway, FastAPI, and vLLM?

Why must `proxy_buffering off;` and `X-Accel-Buffering: no` be configured in Nginx to prevent buffering delay spikes during SSE streaming?

Compare WebSocket vs SSE vs gRPC bidirectional streaming across firewall traversal, auto-reconnection, and architectural simplicity?

How to use FastAPI `request.is_disconnected()` asyncio generators to cancel active vLLM generation tasks upon socket drop?

How do frontend streaming UI parsers handle partial unclosed Markdown code blocks gracefully without layout flickering?