Context Budgeting & Memory Compaction

Context Window Token Budgeting & Progressive Memory Compaction treats the finite LLM context window as scarce RAM managed through strict partition budgeting and automatic memory compaction; the budget allocation matrix partitions total tokens (e.g. 8k/32k) into: Fixed System & Tool Definitions (20%), Dynamic Working Memory (50% for recent detailed tool/thought traces), Output Buffer (20% for generation), and Safety Margin (10%); Progressive Compaction triggers when working memory saturates: 1) Sliding Window Truncation (retaining recent $K$ turns), 2) Recursive Semantic Summarization (using cheap small LLMs to distill older turns into dense structured state summaries), and 3) Tool Output Pruning (stripping multi-page raw JSON responses down to essential fields while offloading full payloads to vector stores).

Multi-hour 50+ turn conversational agents, autonomous codebase refactoring, and continuous research systems.

Naively appending conversation histories causes context window overflow, 'Lost in the Middle' attention degradation, and skyrocketing token fees; dynamic compaction ensures constant-memory, bounded-cost execution indefinitely.

Diagram the 4-tier context window token budget partitioning and dynamic reallocation thresholds?

Explain how recursive summarization preserves structured task states, active constraints, and resolved sub-goals without data loss?

Explain the 'Lost in the Middle' attention bias and why placing critical system constraints at the beginning and end maximizes retrieval accuracy?

How should tool execution wrappers safely truncate massive 50k-token CLI logs or HTML payloads before passing them to the LLM?

How to structure conversation history sequentially to preserve static prompt prefixes and maximize cloud KV Cache reuse?