← Back to Registry Hub/Site Search/L3 · capstone_project
⚡Flashcard Drill➔
L3S3 · Model & AgentS4 · UX & HITLS5 · Eval & LLMOpsVerified: 2026-11-17

Capstone: Stateful On-Call SRE Telemetry Triage & Root-Cause Copilot

Production SRE capstone pairing LangGraph cyclic loop guards, read-only OTel/K8s log diagnostics, interrupt-before rollback Diff UX, and deterministic sandbox incident replay scoring.

Best For

SRE, DevOps, QA, and Backend candidates showcasing fault-tolerant agent state machines, OTel observability, and safe HITL mitigations.

Human-AI Boundary

Telemetry diagnostics run read-only; mitigation commands pause cleanly at signed diff cards requiring on-call engineer approval.

Anti-Patterns to Avoid

  • Giving an on-call agent direct unattended `kubectl delete` or DB failover credentials in production
  • Feeding raw 50,000-line Pod logs verbatim into context without Map-Reduce log filtering or deduplication

Latency/Cost Profile & Eval Metrics

Initial RCA evidence summary < 12s after alert trigger vs 25min manual log hunting · Bounded max_steps = 8 + compressed Map-Reduce trace worker summaries

  • Top-1 Root-Cause Replay Accuracy & Zero Unsafe Mutations: >= 88% exact root-cause service identification on 40 replayed postmortem incidents; 0 loops > 8 steps (Evaluated over frozen historical Prometheus + Jaeger telemetry snapshots)

💰 Business ROI Model & Evidence Level

Reduces median Sev-2 triage MTTR from ~38 minutes to <9 minutes while zero-trust read-only gates guarantee zero unauthorized production mutations.

🏗️ End-to-End Architecture Pipeline

Step 1: Alert Deduplication & Isolated Subagent Log Triage

Correlate deploy commits, error clusters, and latency traces via isolated read-only worker subagents.

Step 2: Cycle Guard + Signed Mitigation Diff Approval

Abort repeated tool-call cycles deterministically and pause for on-call confirmation before executing mitigation steps.

📊 Benchmark & Alternative Comparison

  • Bounded Stateful RCA Agent vs Single-Prompt Log Dump: root-cause recall lifts from 52% to 89% on noisy microservice traces while staying under 12k tokens/incident.

📦 Project Deliverables & Portfolio Artifacts

  • LangGraph On-Call RCA Engine with Cycle & Idempotency Guards
  • 40-Incident Deterministic Replay Eval Harness & MTTR Benchmark Report

🎤 Interview Defense & System Design Deep-Dive Q&A

  1. How do you prevent an on-call debugging agent from endlessly polling broken log queries or hitting stale pod IDs during an outage?
  2. How do you evaluate an RCA agent offline when production alerts are non-stationary?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔

Bidirectional Taxonomy Links

Internal Tools & Primary Sources