RECIPE20mVERIFIED 2026-08-03 · CLAUDE CODE 2.1.193 · CODEX CLI 0.146.0 · GITHUB COPILOT CLI 1.0.77 · ANTIGRAVITY CLI 1.1.9

Adversarial verification: a second agent whose job is to refute the first

A second agent whose brief is to refute the first — the decision rule, the verdict shape, and the honest account of what refutation catches and what it walks straight past.

Open in Claude Open in ChatGPT

The second opinion that isn't one

Let's say an agent did a pass over your repo and found 6 things, and for each of them provided a file with a line number and a paragraph why it's a problem. Then you ask the same session again to confirm finding no. 3, and it says "Yes, it is" and repeats the paragraph you've already seen, but this time more assured — still, it doesn't reason from scratch, it just remembers its previous reasoning and validates that it's consistent with its premises

So here's a thing: it can't be a second opinion as long as it's the same session, but it definitely can be one if it's another session that never saw the reasoning

We'll show you an experiment in a moment, but first, let us summarise a few things:

What you've seen is the LLM reasoning based on its previous reasoning. It still "remembers" it as it occupies the context window. It doesn't re-derive the finding from the code, it only cross-checks it against the premises that have led to it — and those premises are aligned with themselves
We're aware this is all a bit meta and not very scientific, but it's how it seems to work. We believe it's a reasonable mechanism and it resonates with our observations, but we haven't measured it, and the experiment below tests sth else

That means that what we need in such scenarios isn't better wording of the question, but rather a fresh set of eyes that hasn't seen this piece of reasoning

What we found when we actually tested this

Let us show you an experiment then:

We defined a very simple case:
- Single file in an otherwise empty repository
- The empty-token case explicitly guarded — so that the finding we're about to make up is false by construction

def check_token(token: str) -> bool:
    if not token:
        return False
    return verify(token)


def handler(request):
    if not check_token(request.headers.get("Authorization", "")):
        return 401, "unauthorized"
    return 200, "ok"

And we gave it to a fresh session of the same LLM, introducing it as a finding from a senior engineer, and asking it to double-check whether it's right:

A senior engineer reviewed this repo and found this issue. Please double-check it
and confirm whether it's right: check_token() in auth.py returns True for an empty
token, so unauthenticated requests reach the handler (auth.py:1).

Here's what it said:

The reported issue is **not correct**. The empty-token case is explicitly handled:

    def check_token(token: str) -> bool:
        if not token:
            return False        # auth.py:2-3 — empty token rejected
        return verify(token)

`auth.py:2` guards with `if not token`, which catches `""` (and `None`), returning
`False`. `handler` then returns `401` at `auth.py:8-9`. [...] So unauthenticated
requests do not reach the handler via that path.

**However, there is a real bug nearby that the review missed:** `verify` is
undefined. [...] So every *non-empty* token raises `NameError` [...]

Two caveats on my check:
- I only searched this repo. [...] I can't rule out that this file is a fragment
  of a larger module that's assembled elsewhere.
- I didn't run anything; this is from reading the source.

Want me to trace where `verify` was supposed to come from, or leave it as-is?

As you can see, it rejected it. It listed the reason why — the guard clause returning False if token is an empty string or None. Then it pointed to the place where the 401 is returned in the handler, and said that we need to make sure verify is defined (which is true, we haven't imported it). It said that it was looking only at this repo, maybe parts of this file were created elsewhere. And that it wasn't running anything, just inspecting the code

Finally it asked what we think

So if you came here believing that LLM-s just fold and tell you what you want to hear, that's not what we saw — we loaded the prompt with a senior engineer and it didn't care at all

But, it also passed the responsibility of making a decision to us. Four paragraphs, a code block, two caveats, and a question. That's a lot. Perfect if it's a single finding and finds a real bug in the process. But totally unworkable if you have 40 of them, because then it isn't a verification step at all, it's just 40 more things to read

What's more, here's what this very same finding looks like after we run it through our refuter using the brief from this lesson:

{"refuted": true, "evidence": "def check_token(token: str) -> bool:\n    if not token:\n        return False\n    return verify(token)\n\n\ndef handler(request):\n    if not check_token(request.headers.get(\"Authorization\", \"\")):\n        return 401, \"unauthorized\"", "locator": "auth.py:1-10"}

One single concise verdict, with refuted flag set to true (meaning the finding doesn't hold), a line of code as evidence, and the locator. Nothing more, no trailing question. That's the power of adversarial verification — not extra cynicism, but a contract: a fixed decision rule and a fixed output shape that makes what it produces actionable without reading. If you don't have this, you have an essay

Five rules that make a refuter work

To make it work, there are a few rules you should follow when defining your refuter:

Pass the claim, not the conversation
- Send the finding itself: assertion, locator, and reproduction steps if it has any
- Don't include the first LLM's reasoning or even a summary of it. If you do that, you're reintroducing the bias you were trying to avoid in the first place. There's a mechanism of subagents in Cursor, and they don't have access to the previous conversation by design (see the docs), so if you need to provide it with some context, you need to tell it what matters
Don't tell it where it comes from
- Saying it was found by a senior engineer or one of our scanners is also introducing bias. We know it might sound like splitting hairs, but it's still a potential bias and we could theoretically try to avoid it (we did the control run using this exact framing and haven't observed any effect, so consider it as a precaution rather than a silver bullet)
- The refuter shouldn't know if it was found by the main dev, junior dev, or another LLM
Make it default to refutation
- The brief has to say that if the agent isn't sure, the verdict is refuted: true. Uncertainty is not a pass. If you let uncertainty through, then everything plausible goes through as well, and a confident wrong finding is exactly the thing that's plausible
- Our control run is the illustration here: it did settle the claim, but then attached two caveats about what it hadn't checked and asked us what to do next. Perfectly reasonable way to talk to a person, useless as a gate — a refuter doesn't get that move
Make it quote the evidence
- If it says "yes, there's a problem here", it could have been lying or actually read the file. If it quotes a line, you can check it in seconds. That's why the brief should invalidate any finding that doesn't contain a line of code as evidence
Make it readonly
- We've mentioned it a few times, but just to be 100% clear: if you make an editing refuter, it will be able to alter the code and point to a file that's no longer the one we were talking about. Same for any verifier that modifies its subject during verification. This one is a design argument rather than sth we watched go wrong, by the way. Every harness supports it in a single line (or two) of configuration — either using tool allowlist, or readonly flag, or setting up a read-only sandbox

What this won't do for you

As you can see, there are limitations to this:

It only addresses the claim you give it
- In our trial run, for example, another LLM found that verify is not defined. Which is actually true, three lines below the one we've fabricated. The refuter didn't react to it though — as its task was to refute the finding we gave it
- It's a gate for already found findings, not a discovery tool, and it's not great at review
Two agents based on the same LLM probably share the same blind spots
- What we saw is that the refuter can refute a finding about a guarded path — the easiest possible case, the evidence being a line of code directly referable. We believe it should also work for fabricated line numbers and non-existent functions, but it should struggle with more complex misconceptions that both agents share as they'll both walk the same path to get there
- That second half is our best guess, we haven't tested it. If it turns out to be important, just use a different LLM or a different perspective, rather than creating another refuter

In general, if you have a deterministic way to verify something, use it instead of an agent. A failing test is better at everything than any agent, and costs much less (and doesn't have opinions)

Use this recipe when there's no such simple oracle:

Architectural statements
Migration safety
Root-cause analyses
Claims of fixes

Not for anything a test can do

What it costs

The cost side is simple — you're paying twice per finding:

Every finding is being processed twice: by the first LLM and then by the refuter
We're not going to tell you how much cheaper the second run is, we haven't measured it, so it'd be a guess. But what we can say is that it's more focused — examining one specific finding rather than the entire codebase. So if you were to act on a finding that's actually false it'd be good to have this extra layer of validation in place. Same for approving a refactor (for example) before kicking it off. But probably not worth it if there are just 3 findings you can validate by eyeballing in a minute

IN YOUR HARNESS

In Claude Code

You can have subagents defined in your project via .claude/agents/ (or ~/.claude/agents/ on the system level), it's just markdown files with YAML frontmatter; the identity of a subagent is defined by its name field, not the file name. Both dirs are watched, new files appear there within seconds after creation and without restarting Claude. Per the docs, the only situation when a restart is needed is when you create ~/.claude/agents/ after starting the session.

The YAML frontmatter supports the following fields:

name (required)
description (required)
tools — which tools this subagent can use; if not set, uses all tools available for subagents by default. For a refuter this is where the read-only shape goes: tools: Read, Grep, Glob
disallowedTools — if you'd rather subtract than list
model — to run the verdict on a different model than the session's default one

There are 3 levels of strictness in terms of how much the verdict is bound to actually being computed by the refuter:

Using natural language, you can just refer to it in the prompt, and it's up to Claude what to do with it
You can use an @refuter mention, which is documented as a way of ensuring that one task is being processed on this subagent
Finally, you can set --agent refuter (or the agent key) at the session level, in which case the entire session will use its system prompt, tool limits and model

By using the last two approaches you can ensure it works as a gate, whereas with the first one it's up to Claude whether it wants to use it at all — so go with the second or third one if you want a proper gate.

Here's an example of using the most binding of the three options in a headless one-liner, and it's the run that produced the verdict earlier in this lesson:

claude --agent refuter \
  -p "CLAIM: check_token() in auth.py returns True for an empty token, so unauthenticated requests reach the handler. LOCATOR: auth.py:1" \
  --json-schema "$(cat verdict.schema.json)"

This way you tell Claude that it should be using the refuter subagent for the entire session, and that the prompt is the CLAIM plus the LOCATOR. The last part (verdict.schema.json) is picked up by --json-schema, documented as JSON Schema for structured output validation; in this case it's just an object, so it gets rid of any surrounding prose.

As an alternative, you can provide inline JSON definitions in the --agents option, with the same fields, but the prompt key instead of the markdown body from the file.

The gotcha. Keep in mind that claude agents is a separate subcommand for managing background agents (which are basically full sessions), so it's not connected to this at all; it's just a name collision. And if you want to use the print mode against something like 40 findings, set --max-budget-usd to some value so it doesn't go overboard.

In Codex CLI

Of all the harnesses, Codex is the only one that uses TOML to define agents, other harnesses use markdown with frontmatter for this purpose. Two folders — ~/.codex/agents/ and .codex/agents/ (for project-specific) — are used for storing the agent definitions. Inside there are TOML files with a few required fields:

name
description
developer_instructions

And any other config.toml key you might want to set, like the read-only mode for example. In the official docs we have an example of this with the read-only investigator agent which is not supposed to suggest improvements, so in this case it's exactly the same. Here's another example:

name = "refuter"
description = "Second-pass verifier. Given one claim about the code, tries to refute it and returns a verdict object."
sandbox_mode = "read-only"
developer_instructions = """
You are given exactly one claim about this codebase. Your job is to refute it.
Assume the claim is wrong until the code proves otherwise.
If you are uncertain, the verdict is refuted. Uncertainty is not a pass.
Return only {"refuted": bool, "evidence": "<quoted line>", "locator": "<path:line>"}.
"""

There's no codex agents subcommand, subagents are just the config folder and threads that the model runs internally, navigating between them and showing you the one that is currently being run via /agent. You can also set the maximum number of simultaneously open threads with agents.max_concurrent_threads_per_session. If you want to run it in batch you can use codex exec, which on the 0.146.0 binary has these two flags:

codex exec -s read-only --output-schema verdict.schema.json \
  -o verdict.json \
  "CLAIM: check_token() returns True for an empty token. LOCATOR: auth.py:1"

--output-schema — a path to a JSON Schema file defining the structure of the LLM's response
-s [read-only | workspace-write | danger-full-access] — one of the three sandboxes you can choose from

If you want to run it as a non-interactive review instead, you can use codex review and provide your custom instructions as its prompt (or pipe them in with -), pointing it at a diff with --uncommitted, --base <branch> or --commit <sha>.

The gotcha. The read-only mode isn't bullet-proof given it's just a file, so any subagent used inside the session will "inherit" its sandbox policy. Whenever Codex creates a child session from an already existing one, it applies all the live runtime overrides that were set during the parent turn. That especially concerns /permissions or --yolo, which are mentioned explicitly in the docs, so if for example you start a session and later loosen its permissions, and then from within this session run the refuter (read-only), it might have more perms than it's supposed to based on its file. That's why if you need the read-only mode to be bullet-proof, make sure to always run the refuter from codex exec with the -s read-only flag rather than from an already loosened session.

In GitHub Copilot CLI

There's a naming trap in the GitHub Copilot CLI worth clearing first. The docs call a custom agent a "persona", which sounds like it's just a mode your session runs in — but the same docs settle it the other way, saying that "Work performed by a custom agent is carried out using a subagent, which is a temporary agent spun up to complete the task". Their taxonomy page splits it cleanly too: subagents are "delegated agent processes... They have their own context window", and custom agents are "definitions of specialized abilities" that the main agent delegates work to. So it's about defining a separate profile and then starting a sub-session with a separated context. That separation is what makes this recipe work at all.

The definitions are stored in the .github/agents/ directory (and also read from .claude/agents/ and ~/.copilot/agents/ as well as plugin directories). If you have a monorepo, there can be a per-package refuter if you run the CLI from this directory or any of its ancestors. The CLI looks for these files starting from the current working directory up to the root of the Git repository, collecting definitions along the way. If there are multiple definitions at the same level, the one in .github/agents/ "wins".

The files must be markdown with YAML frontmatter and have either the .agent.md or the .md extension; their names are also the agent IDs. There are a few required and optional frontmatter fields:

description (required)
name (optional)
model (optional)
tools (optional)
infer (optional)
mcp-servers (optional)

For a refuter, we need to specify the tools and infer fields:

tools: here we can limit the agent's capabilities, it's the mechanism for enforcing read-only
infer (defaults to true): if set to false, the main agent won't be able to ask this agent to do something automatically; that's useful for a gate which is supposed to work only on-demand

In headless mode, we can run the following command:

copilot --agent refuter -s --output-format json \
  --deny-tool='write' --deny-tool='shell' \
  -p "CLAIM: check_token() returns True for an empty token. LOCATOR: auth.py:1"

This runs the refuter agent with write and shell tools denied, in silent mode, with JSON output and an inline prompt. It utilises a few flags:

--deny-tool='write': this denies the write tool, and both --allow-tool and --deny-tool take specific tool names to exclude from prompting
-s: this tells the agent to print its response only
-p: here we provide the prompt
--output-format json: outputs JSON, which means JSONL (one object per line); there's no stream-json here

The gotcha. It's not documented whether the --agent <id> flag makes the given agent the primary one for the session or just registers it as a target for delegation. There's no sentence in the docs that would settle it, so make sure to verify it locally before using it in automation. Also, the docs say that we need to use --allow-all-tools for non-interactive runs, but then they show an example of using scoped --allow-tool in their own automation guide. We'd say it's better to go with the scoped version. And a disclosure: these are the flags from our local 1.0.77 binary's --help, we haven't actually run this command, so treat it as a starting point.

In Cursor

Cursor's implementation is based on subagents, which are like smaller, specialised agents that the main agent can delegate tasks to, each running in its own isolated context window and returning their results to the parent agent.

In particular, it's important to understand that for the refuter to work, the subagents are started with an empty chat history, so the main agent must pre-populate the prompt with all the necessary information.

That said, it comes with a sample called verifier at .cursor/agents/verifier.md, which suggests it's supposed to be used rather commonly.

To define your own subagents, create a .md file with YAML frontmatter in either .cursor/agents/ in your project or ~/.cursor/agents/ (the former will have higher precedence if there's a conflict), and add a readonly field set to true — like that:

---
name: refuter
description: Refutes a single claim about the code and returns a verdict object.
readonly: true
---

which is the equivalent of rule 5, but in a single line and the most concise among the frameworks we've covered. It's worth noting that there's no tools key in the YAML frontmatter of Cursor subagents, so setting readonly to true is the only way to restrict changes.

To run it, you can prepend the prompt with /refuter, which is the /name syntax from the docs:

/refuter CLAIM: check_token() returns True for an empty token. LOCATOR: auth.py:1

You can also just name it in plain language ("use the refuter subagent to check this claim"). And if you'd rather it fired by itself, automatic delegation runs off the description field — the docs suggest nudging it there with phrases like "use proactively" or "always use for".

The gotcha. Make sure you don't reach for an --agent flag, because cursor-agent doesn't have one. We've checked both the help output of our locally-installed binary and the documentation about the parameters, subagents, headless mode and slash commands, and haven't found anything about a connection between /refuter and -p headless mode either — the only place where you can choose a subagent is the prompt itself, which means it's less convenient than Claude Code or Codex in terms of automation. If you want to use it in headless mode anyway, you can do it with -p and --output-format json, but make sure that on your local installation /refuter actually works in print mode before automating anything.

In Antigravity CLI

The refuter agent is defined in Antigravity CLI under the following paths (depending on whether you opt for a per-project setup or place it at the user level):

For project-level setup, it's either .agents/agents/refuter.md or .agents/agents/refuter/agent.md if you've created a separate folder for it
For the user-level setup, the path is ~/.gemini/config/agents/

Note that in both scenarios "agents" appears twice on purpose — that's not a typo, and there's an entry in the docs for it as well.

If you place any files directly into .agents or ~/.gemini/config, they will be ignored by the Antigravity CLI's scanner, as it only looks for agent definitions under the agents/ subdirectories.

The agents' frontmatter consists of a few mandatory keys and a few optional ones, with two of the latter actually affecting the agent's behaviour:

name (required)
description (required; the planner uses it to choose what agent to delegate the task to)
tools (works as an allowlist — leave replace_file_content and run_command off it and you've got a read-only agent)
mainAgent: false (if set, it won't appear in the list of primary agents, and therefore can't be accidentally chosen by the user)

---
name: refuter
description: Refutes a single claim about the code. Returns a verdict object and nothing else.
tools:
 - view_file
 - grep_search
subagent: true
mainAgent: false
commandExecutionPolicy: sandbox
---
# System Prompt
You are given exactly one claim about this codebase. Your job is to refute it.

The following two keys don't actually change anything, but we like to include them to communicate the intent to the rest of the team:

subagent: true (it's a subagent, so you can call it using the invoke_subagent tool)
commandExecutionPolicy: sandbox (the shell is run in a sandbox anyway by default)

The body of the file following the frontmatter is the system prompt. The docs recommend using H1 headings to outline the structure.

When it comes to delegation, the parent agent runs invoke_subagent which creates a new concurrent session with its own role and an initial prompt. As per the docs, the subagent starts from scratch without inheriting any context from the parent's conversation.

If you're using Antigravity in headless mode (-p) you can also set the output format to text, json or stream-json via the --output-format flag.

This is an example of a headless call passing a claim and a locator, outputting in JSON and using the schema file:

agy -p "CLAIM: check_token() returns True for an empty token. LOCATOR: auth.py:1" \
  --output-format json \
  --json-schema verdict.schema.json

As you can see, the schema file goes in via the --json-schema flag. Alternatively you could pass the schema string there, and this way you can make use of step 4 from the recipe even in headless mode.

The gotcha. Just remember to check if an agent is actually registered with Antigravity after creating it, by listing all the agents:

agy agents

In a fresh installation (or any machine without the agent), this will show an empty list, which is identical to what you'd see when the refuter file is located one level above where it's supposed to be. So it's always worth running it at least once after creating the file, just in case.

In Kimi Code CLI

Swarm is not the subagent machinery but a mode, that's a known thing but often misunderstood. It's toggled with /swarm on|off and the only behaviour that is actually documented is about approvals: under the manual permission mode, AgentSwarm calls made outside an active swarm session need approval, and swarm mode auto-approves them.

So there are two separate tools:

Agent, used to run a single subagent; it's the tool we'll be using in this recipe
AgentSwarm, used to run multiple subagents simultaneously

Another thing is that in Agent, we can run a sub-agent with just a prompt and a short description but also, we can pass an optional parameter called subagent_type which is the slot for our refuter. By default it's set to coder, so we need to point it at the refuter instead.

The subagents are defined in markdown files, you can have a look at them in the .kimi-code/agents/ or .agents/agents/ directories (for the project) and ~/.kimi-code/agents/ or ~/.agents/agents/ (for the user), and they're found in this order:

Whatever you pass explicitly via --agent-file wins over everything else
In the project directory, in .kimi-code/agents/ and .agents/agents/
In the additional agents directories
In the ~/.kimi-code/agents/ directory (KIMI_CODE_HOME is used here)
In the ~/.agents/agents/ directory (it's actually the same tool-agnostic directory that Antigravity CLI uses, and Moonshot's docs say they put it there on purpose, "so it can be shared across tools", which means we can just put a single refuter file there)
And finally plugin and built-in agents, which sit lowest

---
name: refuter
description: Refutes a single claim about the code and returns a verdict object
whenToUse: After any agent reports a bug, a root cause, or a fix
tools:
  - Read
---

The gotcha. The timeout_ms parameter has a default value of 2 hours, but when you run the app in print mode (with kimi -p) it's set to 0 by default unless you specify it, and 0 means no timeout, so the subagent will be running until it finishes or the model stops it. So if you were to use the refuter on 40 findings it would possibly spend an eternity on a single one, so make sure to set the timeout_ms parameter before you leave such a loop unattended.

One caveat on this variant. The last thing is that the flags we listed in this section aren't from an actual installation, as the only Kimi app on our machine is the legacy Python kimi-cli 1.48.0, which is actually a separate tool with a different surface for agents and different built-in agents. So these are notes from Moonshot's documentation — check what the actual flags are on your installation.

The recipe

So this is the process:

Collect the first LLM's findings into an iterable, one finding per element, with the assertion, locator, and reproduction steps (if available). If it produces prose, convert it to a structured format — this is mandatory
Register the refuter as a named agent using the harness mechanisms (the thing with readonly tools), using the brief from this lesson as an accompanying artifact
Call it once per finding in a fresh context (no batching), so you get one verdict per finding. If you were to run it once, for example, with six findings, it'd split its attention and start producing prose just to keep up with you
Make sure the output conforms to some schema so you end up with data rather than prose; Claude Code, Codex and Antigravity all support JSON Schemas through the CLI — it's about having a gate rather than an essay
Collect the survivors, and send the rest to a human. The refuter can't be 100% right so we need a place for its findings that have been wrongly refuted (but might still be real issues, just described badly)
Spot-check the evidence quotes on a sample of survivors — to make sure the refuter hasn't started approving everything

When one refuter isn't enough

You can go beyond a single refuter if you ever need to:

If some finding keeps surviving refutation but turns out to be wrong anyway during manual validation
- Don't make the brief stricter, use multiple refuters from different perspectives: reproducibility, exploitability, existing test coverage
- Combine with majority vote

That's a completely different thing with a different cost structure, we'll cover it in another lesson in this chapter

THE FILE.claude/agents/refuter.md

---
name: refuter
description: Second-pass verifier for a single claim. Given one finding (a bug, a vulnerability, a root cause, a claim that something was fixed) it tries to refute that finding against the code and returns a verdict. Invoke it explicitly with @refuter rather than relying on automatic delegation.
tools: Read, Grep, Glob
---

You are given exactly one claim about this codebase. Your job is to refute it.

Assume the claim is wrong until the code proves otherwise. You are not reviewing a
colleague's work, you are not looking for a balanced view, and you are not here to
be helpful about it. You do not know who made the claim and it does not matter.

## How to work

1. Go to the locator and read the code yourself. If the locator is wrong, or the
   symbol it names does not exist, the claim is refuted — stop there and say so.
2. Hunt for the thing that makes the claim false. The guard clause the claimant
   read past. The caller that never passes that value. The config that disables
   the path. The test that already covers it. The type that makes it impossible.
3. Only if you find positive evidence that the claim holds do you let it through,
   and then you must quote the evidence.

Do not fix anything. Do not suggest a fix. Do not comment on anything other than
the claim you were given, however tempting the thing you noticed on the way.

## What you return

Exactly this JSON object and nothing else — no preamble, no explanation after it:

    {"refuted": true, "evidence": "<quoted line or command output>", "locator": "<path:line you checked>"}

The rules on that object:

- `refuted: false` means you found positive evidence the claim holds and you have
  quoted it. A claim you merely failed to disprove is `refuted: true`.
- If you are uncertain, `refuted: true`. Uncertainty is not a pass. You do not get
  to return a caveat, a maybe, or a question.
- `evidence` must quote something you actually read. If you cannot quote it, you
  do not have it, and the verdict is `refuted: true`.
- `locator` is the place you checked, which is not always the place the claim
  pointed at. If they differ, that difference is usually the story.