🏗️ System DesignID:
distributed-tracing-observabilityDistributed Tracing & Observability
全链路分布式追踪与可观测性
🎯Core Definition
Distributed Tracing & LLM Observability (OpenTelemetry, LangSmith, Arize Phoenix, OpenLLMetry) provides the comprehensive telemetry and telemetry telemetry recording full lifecycles across production LLM and Agent pipelines; injecting a globally unique `Trace ID` and nested `Span IDs` at ingress, the framework instruments every execution stage: 1) Latency Telemetry: Time To First Token (TTFT), Time Per Output Token (TPOT), and total inference duration; 2) Cost & Token Auditing: tracking prompt tokens, completion tokens, prefix-cached tokens, and monetary cost attribution per tenant/model; 3) Runtime Payload Capture: recording fully rendered prompt templates, retrieved RAG chunk scores, Agent tool execution inputs/outputs, and safety guardrail verdicts.
💡Use Cases
Production LLM troubleshooting, debugging multi-step Agent reasoning loops/deadlocks, tenant cost chargebacks, and RAG hallucination auditing.
⚡Key Problems Solved
Classical APMs only trace raw HTTP spans, leaving LLM inference internals and multi-agent loops as opaque black boxes; LLM observability renders the full execution graph transparent from network ingress to individual token generation.
🎯5 High-Frequency Exam Points
1
Define the calculation formulas and hardware bottlenecks of TTFT (Time To First Token) vs TPOT (Time Per Output Token)?
2
Design OpenTelemetry-compliant semantic conventions for LLM tracing across prompt tokens, completion tokens, and tool invocations?
3
How does nested Span tree topology record Agent Thought-Action-Observation loops and trigger watchdog circuit breaks on infinite tool loops?
4
Design an inline PII masking interceptor scrubbing credit cards, passwords, and PII from prompts before persisting spans to ClickHouse/Jaeger?
5
How to integrate asynchronous RAGAS span evaluators computing live Faithfulness and Answer Relevance metrics without blocking user responses?
📖 In-depth Guide:📄 llm-rag-agent-system-design →
Updated 2026-08-14🎯
🚀 Start Card Practice ➔Test Your Knowledge: Practice Questions for "Distributed Tracing & Observability"
Single choice pitfall questions with instant feedback and mistake tracking.