Mitchell Foutz All writing

Claude writes, Codex reviews: a Stop-hook review gate capped at two rounds per batch of changes

A Claude Code Stop hook in agent-controls runs the project's tests and a Codex CLI review of the uncommitted changes each time Claude tries to end its turn, and a change request sends Claude back to work. After 2 rounds on the same set of changed files the gate lets Claude stop and logs the open blockers. The original version of the gate logged 34 verdicts in two private repositories: 12 approvals, 18 change requests, 3 approvals at the round cap and 1 dispute.

Published
2026-10-28 (draft)
Topic
Agent engineering
Source
darblinko/agent-controls
Revision
fb209b2
Verification
The gate code is public at the revision above. The review-event counts are self-reported: the activity and decision logs are in two private repositories that are not published, so those counts cannot be checked independently.

The problem: the author decides when the work is done

In a Claude Code session, the model that wrote a change also decides when the turn ends. Without a check that runs outside the model, that decision rests on the author's reading of its own work. The agent-controls threat model lists the resulting failure as T4: Claude stops with failing tests, or with changes no second reviewer has seen.

Claude Code runs Stop hooks when Claude is about to finish. A hook that exits 2 keeps the session going, and its stderr becomes Claude's next input. A hook that exits with another non-zero code is reported to the user and does not block. The review gate uses these exit codes to put two checks in front of the stop: the project's test command, and a review by a second agent, Codex CLI, run with a read-only sandbox and a JSON output schema.

A blocking check brings its own failure, listed as T5: a review loop that does not converge, with the agent held in it. Most of the gate's limits exist because of T5. The code is in the agent-controls repository, described on its project page. This article cites it at revision fb209b2.

How the gate runs

flowchart TD
  stop(["Claude tries to stop"]) --> checks{"changes to review?"}
  checks -->|"no"| a0["exit 0"]
  checks -->|"yes"| tests{"tests pass?"}
  tests -->|"no"| b1["exit 2: test output to Claude"]
  tests -->|"yes"| codex["Codex review of the capped prompt"]
  codex --> verdict{"verdict"}
  verdict -->|"none"| e1["exit 1: no approval"]
  verdict -->|"APPROVED"| a1["exit 0: approval written"]
  verdict -->|"CHANGES_REQUESTED"| rounds{"rounds left?"}
  rounds -->|"yes"| b2["exit 2: review to Claude"]
  rounds -->|"no"| a2["exit 0: blockers logged"]
  b1 -.->|"stops again"| stop
  b2 -.->|"stops again"| stop
      

The diagram follows main() and folds the checks before the review into its first decision: a disabled gate, a lock held by another run, no uncommitted changes, and a working state that is already approved each lead to exit 0. A setup error leads to exit 1. An exit 2 sends Claude back to work, and its next attempt to stop runs the gate again. The three exit codes map to three outcomes in Claude Code:

ExitWhenEffect
0Gate disabled; live lock held by another run; no uncommitted changes; working state already approved; APPROVED; CHANGES_REQUESTED in the last roundClaude stops
2Test command fails; CHANGES_REQUESTED with rounds leftClaude continues, with stderr as its input
1Setup error; Codex exits non-zero; no review file; review cannot be parsedThe user sees stderr and Claude stops; no approval is written

An approval records a hash of HEAD, the short status, the diff against HEAD and the content of every untracked file. Any later edit changes the hash, and the next stop reviews again. A verdict of APPROVED that lists blocking issues is treated as CHANGES_REQUESTED.

Round cap: 2 rounds per batch

REVIEW_GATE_MAX_ROUNDS defaults to 2. Round 1 is a full review. In later rounds the prompt carries the previous review and tells Codex to verify only that its blockers were fixed, and to raise no new blocker unless the fix caused a regression. The reviewer instructions add a severity rule: only P0 and P1 issues block, and P2 issues are suggestions.

    CHANGES_REQUESTED)
      printf '%s\n' "$ROUND" >"$ROUND_FILE"
      if ((ROUND >= MAX_ROUNDS)); then
        write_approval true max_rounds_reached
        log_event AUTO_APPROVED "$blocking unresolved blocker(s) after $ROUND round(s)"
        log_decision AUTO_APPROVED "$blocking blocker(s) unresolved after $ROUND of $MAX_ROUNDS rounds"
        echo "Codex still requests changes after round $ROUND of $MAX_ROUNDS."
        echo "Round limit reached: stopping is allowed. Unresolved blockers are in $DECISION_LOG."
        exit 0
      fi
      log_event CHANGES_REQUESTED "$blocking blocking issue(s)"
      log_decision CHANGES_REQUESTED "$blocking blocking issue(s) in round $ROUND"
      echo "Codex requested changes (round $ROUND of $MAX_ROUNDS). Fix these, then stop again:" >&2
      cat "$REVIEW_FILE" >&2
      exit 2
      ;;

Failure it prevents. A reviewer that raises a new blocker in each round can keep Claude working without end. The threat model labels this rationale an inference; it is not measured.

What it costs. At the cap, Claude stops with the blockers still open. The approval file records auto_approved=true and override_reason=max_rounds_reached, and the decision log .ai/reports/qa_decisions.jsonl records the review. A person has to read that log; the gate takes no further action on it.

Design decision: a round counts only when Codex returns a verdict. The round counter is written in one place, the first line of the CHANGES_REQUESTED branch above. A failing test command exits earlier, and so does a Codex failure. Inference: if those runs used rounds, a Codex outage in round 1 followed by one change request in round 2 would end in an approval at the cap after a single review.

Claude Code also passes Stop hooks a stop_hook_active field, which is true when the session is already continuing because of a Stop hook. The gate does not read its hook input. Its loop bound is the round counter file.

Batches and the stale review

start_round() {
  local batch previous="" completed=0
  batch=$(project_changes | sort | git hash-object --stdin)
  [[ -f $BATCH_FILE ]] && previous=$(<"$BATCH_FILE")
  if [[ $batch != "$previous" ]]; then
    rm -f "$ROUND_FILE" "$REVIEW_FILE"
    printf '%s\n' "$batch" >"$BATCH_FILE"
  fi
  [[ -f $ROUND_FILE ]] && completed=$(<"$ROUND_FILE")
  [[ $completed =~ ^[0-9]+$ ]] || completed=0
  ROUND=$((completed + 1))
  log_event HOOK_START "round=$ROUND/$MAX_ROUNDS"
}

The cap applies per batch. The batch identifier is a hash of the sorted git status --short output. When it changes, the gate deletes the round counter and the previous review together and starts at round 1. An approval also deletes the counter and the batch file.

Design decision: delete the previous review with the counter. The fix-verification section quotes the review file only when the round is above 1. After a reset the round is 1, so that section is skipped in either case. Deleting the file as well means a review of a file set that is no longer in play cannot be quoted in a later round. Inference: this protects against a future change to the order of these steps; the current code does not need it.

Observation: staging starts a new batch. Each line of the short status carries two status letters. git add on an untracked file changes its line from ?? to A, which changes the hash. With a stub reviewer that always requests changes, the gate at fb209b2 reported round 1 of 2, then, after git add and no content change, round 1 of 2 again. The README and the threat model state both cases: a fix that touches a new file starts at round 1, and staging with no content change does as well. Both documents gained the staging case at b8fc30e. Inference: the cap bounds rounds per file set and status, and an agent that stages files between rounds resets it.

Prompt cap: 204,800 bytes

REVIEW_GATE_MAX_PROMPT_BYTES defaults to 204,800 bytes, which is 200 KiB. The prompt is assembled in this order: reviewer instructions, round section, project scope, Claude's summary, test output, git status, staged diff, unstaged diff and untracked files. Three bounds apply:

  PROMPT_BYTES=$(wc -c <"$PROMPT_FILE")
  if ((PROMPT_BYTES > MAX_PROMPT_BYTES)); then
    log_event PROMPT_TRUNCATED "original=${PROMPT_BYTES} bytes, cap=${MAX_PROMPT_BYTES}"
    head -c "$MAX_PROMPT_BYTES" "$PROMPT_FILE" >"$PROMPT_FILE.tmp"
    printf '\n\n[TRUNCATED: prompt exceeded %s bytes]\n' "$MAX_PROMPT_BYTES" >>"$PROMPT_FILE.tmp"
    mv "$PROMPT_FILE.tmp" "$PROMPT_FILE"
    PROMPT_BYTES=$(wc -c <"$PROMPT_FILE")
  fi

Failure it prevents. A prompt that grows with the diff without a limit. The repository does not state why 204,800 bytes was chosen or which Codex context limit it was sized against.

What it costs. The review runs on the cut prompt and can return APPROVED. Truncation is logged; it does not fail the review. The cut is taken from the end, so the unstaged diff and the untracked files are the first content lost (inference from the section order). The decision log stores the prompt size after truncation. At the default cap a cut prompt is 204,844 bytes, the cap plus the 44-byte marker, so that prompt_bytes value marks a review of a cut prompt.

Time bounds: the hook timeout and the lock

The gate sets no timeout on the Codex call. Its run time is bounded by the timeout field of the hook entry in Claude Code settings, which is 1,200 seconds in the example settings file.

The 1,500-second default in the script is a lock age. Each run writes its process ID and start time to a lock file and removes the file on exit. A stop that finds a lock held by a live process and younger than 1,500 seconds logs a skip and exits 0. A lock held by a dead process, or older than the limit, is cleared.

  local stale=${REVIEW_GATE_LOCK_STALE_SECONDS:-1500} lock_pid lock_time elapsed
  if [[ -f $LOCK_FILE ]]; then
    lock_pid=$(sed -n 1p "$LOCK_FILE")
    lock_time=$(sed -n 2p "$LOCK_FILE")
    [[ $lock_time =~ ^[0-9]+$ ]] || lock_time=0
    elapsed=$(($(date +%s) - lock_time))
    if [[ $lock_pid =~ ^[0-9]+$ ]] && kill -0 "$lock_pid" 2>/dev/null && ((elapsed < stale)); then
      log_event HOOK_SKIP "lock held by live pid=$lock_pid (${elapsed}s old)"
      exit 0
    fi

Failures it prevents. Two stops in one working tree starting two Codex reviews that write the same report files, and a lock left by a killed hook disabling the gate.

What it costs. The second stop ends without a review of its own. Inference: when both sessions share the working tree, the first review covers the same files, except for edits made after its prompt was built.

Observation. The 1,500-second lock age is above the 1,200-second hook timeout in the example settings, so a review that stays within the hook timeout keeps its lock against a second stop. The default was 900 seconds until a5d20b6. A test reads the timeout from the example settings, checks that a live lock one second younger than the timeout is kept, and checks that a live lock twice that age is taken over. Inference: a second review can start in parallel only if the process holding the lock outlives the hook timeout by 300 seconds, or if the hook timeout is set at or above the lock age and a review runs past the lock age.

A reviewer failure exits 1 and does not block

  if ! "$codex" exec --ephemeral --sandbox read-only --cd "$PROJECT_ROOT" \
    --output-schema "$SCHEMA_FILE" --output-last-message "$new_review" \
    - <"$PROMPT_FILE" >"$CODEX_LOG" 2>&1; then
    log_event CODEX_ERROR "see $CODEX_LOG"
    {
      echo "codex-review-gate: Codex review failed; no approval was recorded. Last output:"
      tail -20 "$CODEX_LOG"
    } >&2
    exit 1
  fi

When Codex exits non-zero, writes no review file, or returns a review the helper cannot parse, the gate exits 1 and prints the last output. Claude Code shows it to the user and lets Claude stop. No approval is written and no round is used.

Design decision. In the original gate a reviewer failure blocked the stop. The published version does not block, because Claude cannot repair a Codex outage from inside the session, and a gate that blocked on its own failures could hold Claude in a loop it cannot resolve.

What it costs. During an outage, sessions end with unreviewed changes and an error in front of the user. No approval exists, so the next stop with uncommitted changes runs the review again. Changes committed before that stop are not reviewed.

Turning the gate off: a marker file

The gate passes through when REVIEW_GATE_DISABLE=1 is set in the hook's environment or when the file .ai/reports/disabled exists. The example /review-gate command creates and removes that file.

Design decision. The switch is a file because an environment variable exported from a Claude Code Bash tool call does not reach the hook process.

What it costs. Claude can create the file itself. The threat model recommends denying writes to that path in Claude Code permissions where this matters.

What the tests cover

pytest collects 72 tests at fb209b2, and all 72 passed in a local run on 2026-10-28 (as reported from that run; docs/data.json line 19 at that revision records "of 72 in tests/, all pass"). 28 are in tests/test_review_gate.py, 26 in tests/test_codex_sandbox.py, 13 in tests/test_claude_profiles.py and 5 in tests/test_links.py. Each gate test builds a throwaway git repository and replaces Codex with a stub that records the prompt it receives and returns a fixed verdict.

No test covers these behaviors: deletion of the previous review on a batch change, untracked files over 51,200 bytes, a batch change caused by staging, and the real Codex CLI. The README lists the same four as untested. Inference: the stub returns JSON that matches the schema, so the tests do not show how often Codex itself returns a verdict the gate can parse.

Review records from two private repositories

The original gate ran in two private repositories. It overwrote one review file per round, so the history is in two logs: an activity log with one line per event, and a decision log with one JSON line per verdict. For this article, only the timestamp, round and event fields of the activity logs were read, and the outcome, round and timestamp fields of the decision logs. No review text was opened.

The two activity logs hold 54 and 366 lines, dated 2026-04-29 to 2026-05-10. They record 34 verdicts and 39 Codex calls.

3 of the 39 Codex calls exited with an error, and 3 have no completion or error line. Under the original gate those errors blocked the stop. One repository logs 10 Codex calls in round 2 against 6 change requests in round 1. Inference: the original counter could reach round 2 without a logged change request, either through runs that returned no verdict or through loops that began before the log did. The published version counts a round only on a verdict.

The decision logs hold 11 and 4 lines. The shorter one starts on 2026-05-02 and covers 4 of the 23 verdicts in its repository's activity log, so the activity logs are the more complete record.

The records do not show whether a blocker was correct, whether the blockers open at the cap were fixed later, or what a review cost. Durations were not tallied for this article.

What the gate cannot catch