Domain 5 — Context Management & Reliability
CCA Foundations course · Page 6 of 8 · ← Back to all courses · Weight: 15%. Mark complete at the bottom to advance.
5.1 — Context Window Management · Core
- Case-facts block at the TOP, verbatim — and it must survive
/compact(don't let summarization drop exact amounts/dates). - Lost-in-the-middle effect: put key info at the beginning/end of long inputs.
- Trim verbose tool output to only relevant fields (e.g. 40 fields → 5) before it accumulates.
- Use scratchpad files to persist findings during long exploration.
5.2 — Escalation Patterns · Core
- Explicit human request = immediate escalation, no exceptions (don't attempt resolution first).
- Policy gap / silence on the request = escalate.
- Sentiment-based or self-reported-confidence escalation = anti-pattern (poor proxies for complexity).
5.3 — Error Propagation · Core
- Silent suppression AND full abort are both wrong.
- Return a structured error with
partialResults— preserve successful work and annotate the gaps.
⚠ Often-missed — Human Review & Confidence Calibration · Gap
- Raw confidence ≠ accuracy. Calibrate against a labeled validation set (e.g. via ECE), then route on the calibrated probability.
- Blocking review for irreversible actions; async review for reversible ones.
5.6 — Provenance & Conflicting Sources · Core
- Mandatory claim-source mappings (source URL + date) preserved through synthesis.
- Conflicting statistics → annotate BOTH with attribution; NEVER pick one or average them.
- A temporal difference is not a contradiction (require publication/collection dates).
Exam reflexes for Domain 5
- "Context filling up with exact values" → case-facts block at TOP of every prompt.
- "Explicit human request" → escalate immediately, no resolution attempt.
- "Conflicting statistics" → annotate both with source + date; never pick or average.
- "Timeout returns empty success" → structured error with
isError/partialResults. - "Confidence score for routing" → calibrate first (ECE), route on calibrated probability.