Conversation Context Management

As conversations get long, context fills up. Once context is exhausted, the model loses access to earlier information. This lesson covers the failure modes of long conversations and the strategies for keeping critical context intact.

The Three Failure Modes

1. Progressive summarization loses values. When conversations are auto-summarized, numeric values, dates, customer expectations, and other specifics disappear into prose. The summary captures the gist but not the binding details.
2. "Lost in the middle" effect. Models process the beginning and end of a long context reliably; middle sections may be omitted or compressed in attention. A fact buried in the middle of a 50-turn conversation may functionally not exist for the model.
3. Tool result accumulation. Tool outputs include 40 fields when only 5 are needed. They compound across calls, eating context disproportionately.

Strategy: Persistent Case Facts Block

For domain workflows (customer support, case management, multi-step transactions), extract the transactional facts into a separate block and include it in every prompt. The block is short — customer ID, account status, refund policy applicable to this customer, dates relevant to the case. It survives any summarization because it's reconstructed each turn.

## Case Facts
Customer: Jane Smith, Account #5432
Status: Active, Premier tier
Refund policy: 60 days from purchase
Order in question: #1234, purchased 2026-03-15
Time elapsed: 44 days (within policy)

Strategy: Trim Verbose Tool Outputs

If get_customer returns 40 fields and you only need 5, keep only those 5 before appending to context. The other 35 fields aren't useful to the model — they're noise eating context.

Strategy: Position-Aware Ordering

For multi-document inputs, put the key findings at the BEGINNING of aggregated content. The model processes the beginning reliably; the middle is where attention degrades. If you have 10 documents and 1 contains the answer, putting the answer-document first gives the model the best shot at finding it.

Strategy: Scratchpad Files

For exploration tasks across long sessions, persist key findings to a scratchpad file. The model reads the scratchpad at the start of each phase. Even if context gets summarized or fragmented, the scratchpad survives.

Strategy: /compact Command

The /compact command summarizes the conversation to free context. Use it during extended exploration sessions before context pressure becomes critical. The model can still answer follow-up questions; it just has less granular detail in earlier turns.

Tool Output Hygiene

Skills to Develop

1. Build persistent fact blocks for domain workflows so transactional details survive summarization.
2. Trim tool outputs to relevant fields before appending to context.
3. Place key findings at the beginning of multi-document inputs.
4. Use scratchpad files to persist findings across long sessions.
5. Use /compact proactively before context pressure becomes critical.