Validation, Retry, and Feedback Loops

Estimated time: 3 minutes

Retry with error feedback means the follow-up request includes the original document, the failed extraction, and the specific validation errors — not a generic "try again," which wastes a call by giving the model nothing new to act on.

The discrimination the exam actually wants — when retry helps versus when it's futile:

Retry succeeds Retry fails
Format mismatches The information is genuinely absent from the source
Structural output errors The value lives in an external document you didn't provide
Wrong field placement The source is genuinely internally contradictory

The discriminator: ask whether the information exists in what you actually sent. If it doesn't, no number of retries manufactures it — the correct move is surfacing the gap (a null value, a flagged field) rather than looping on a call that will fail identically every time.

Retry logic flowchart: did the extraction fail, is the information present in the source, yes routes to retry with error feedback, no routes to surfacing the gap One question decides the branch — everything else (how many retries, how detailed the feedback) is downstream of answering it correctly first.

Two specific design patterns worth knowing by name:

  • Self-correction validation designs — extract a calculated_total alongside the document's stated_total and flag any discrepancy; add a conflict_detected boolean whenever source data is internally inconsistent. These catch semantic errors schemas can't (§3).
  • Feedback loops — tag every structured finding with a detected_pattern field. When developers dismiss findings over time, those dismissals become an analyzable dataset showing which code constructs or extraction patterns generate false positives — turning routine dismissals into an improvement signal instead of just noise.