M10-014M10: Engineering Deep-Dive & Technical BQAI Infra Deep-Dive & Postmortem BQHard
⚡Flashcard Drill➔Mastery:
AI Infra Optimization BQ: End-to-End Roofline Kernel & Serving Throughput Tuning
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Walk through concrete hardware profiling (Nsight Systems) exposing CPU launch overhead or HBM memory ceilings, resolved via CUDA Graphs, operator fusion, and FP8 quantization.
📌 Key Takeaways
- •Profile First: open Nsight Systems traces to see whether GPU bubbles stem from CPU kernel-launch latency or HBM bandwidth saturation
- •Fix 1 (Kill CPU Launch Bubbles): capture decode step execution via CUDA Graphs cleanly to eliminate ~40% host dispatch overhead
- •Fix 2 (Fuse HBM Round-Trips): fuse RMSNorm/Residual/RoPE in Triton + ship FP8/AWQ weights to halve memory traffic
- •Empirical Lift: demonstrate concrete pre/post tok/s, SM occupancy, and P99 TPOT metrics
📐 Mathematical Derivations
Respect Amdahl's Law rigorously: optimize host tokenization, IPC copies, or dispatch bubbles first whenever profiling shows idle GPU streams.
🏭 Production Trade-offs
Bucket decode batch sizes to discrete powers-of-two (1, 2, 4, ..., 64) for pre-captured CUDA Graphs cleanly.
⚠️ Common Interview Pitfalls
- ✕Tuning random kernels blindly without capturing an Nsight trace upfront
- ✕Breaking CUDA Graph capture invariants via implicit
.cpu()syncs or dynamic host allocations
🎯 Interviewer Follow-up Inquiries
- ?How does FlashInfer / vLLM reconcile dynamic Paged KV tables cleanly with static CUDA Graph capture?
- ?How do you diagnose register spilling cleanly when tuning Triton tile block sizes?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚