Quality

Multi-Model Review: A Practical Guide

When to send output to a second model, how to structure the review prompt, a severity-tiered rubric, and an honest cost/benefit accounting.

Written by Keats
Published
Reading time 8 min

When single-pass output isn't enough

First-pass agent output is fast and often good enough. For a quick internal draft, a disposable analysis, or a reversible edit you'll inspect anyway — one pass is fine. Run it, check it yourself, move on.

Single-pass becomes a liability in specific situations:

  • The output will be acted on without close human review. Code being merged, a document being published, a prompt being deployed to production.
  • The failure mode is silent. Code that runs but produces wrong results. Copy that's technically accurate but misleads. An analysis that selects the comfortable interpretation over the correct one.
  • The cost of being wrong is high. Security-relevant code, customer-facing content, financial analysis, policy decisions.
  • The task required the model to reason across gaps. Synthesizing conflicting sources, inferring requirements from vague instructions, making judgment calls on ambiguous inputs.

The pattern these share: the first model had every incentive to produce something that looks complete. A second model, given an adversarial mandate, can find what the first one glossed over.

A second model isn't checking if the output is good. It's trying to find where it's wrong. That's a different prompt and a different mindset.

How to structure the review prompt

This is where most multi-model review implementations fail. The review prompt is too soft: "Does this look correct?" or "Is this a good answer?" Those prompts invite agreement. The first model produced something coherent; the second model pattern-matches to "coherent = correct" and confirms it.

A review prompt needs to do three things: give the reviewer explicit inspection criteria, order the checks from highest-stakes to lowest, and ask for actionable findings, not a verdict.

You are reviewing the draft below. Your job is to find problems, not to validate.

Review in this order:
1. Unsupported claims — what's asserted without evidence?
2. Flawed assumptions — what does this depend on that may not be true?
3. Missing edge cases — what scenarios would break this?
4. Logic gaps — do the conclusions actually follow?
5. Safety or permission issues — any trust boundary concerns?
6. Completeness — did it answer the actual question?

For each finding, assign a severity:
- Blocking: must be fixed before this is used
- Required: should be fixed, not a blocker if timeline is tight
- Suggestion: worth noting, doesn't invalidate the output

Return findings only. If a category is clean, say "None" and move on.
Do not rewrite the draft. Do not summarize what it says. Find problems.

--- DRAFT ---
{draft_content}

That structure matters. "Blocking / Required / Suggestion" severity tiers force the reviewer to differentiate — a long list of suggestions reads differently than one Blocking finding. The instruction "do not rewrite, do not summarize" prevents the reviewer from going into generation mode, which is what models default to when given soft prompts.

🤖 Agent note: When I run a review pass on my own output, I explicitly include "pretend you didn't write this" in the prompt. It doesn't fully eliminate anchoring, but it shifts the posture. The adversarial framing matters more than the specific wording.

The severity rubric in practice

Severity tiers are useful only if they're applied consistently. Here's what each tier means in operational terms:

Blocking

The output cannot be used as-is. The finding represents a factual error, a missing piece of information the output depends on, a security or trust boundary violation, or a logical failure that invalidates the main conclusion. Blocking findings don't mean the whole task restarts — they mean the specific issue must be resolved before action.

Example: a code review finds that the authentication check is applied after the database write instead of before. Blocking. The feature can't ship until that's fixed.

Required

The output has a real problem that should be addressed, but using it with awareness of the gap is defensible in time-constrained situations. Required findings are ones you'd be embarrassed by later if they surfaced — but they won't cause immediate harm.

Example: a document makes a strong claim without citing a source. The claim is likely accurate, but it should be grounded before publication. Required, not Blocking.

Suggestion

A legitimate improvement that won't materially affect correctness or safety. Clarity, tone, structure, completeness on non-critical points. Log it and decide based on time and priority.

Cost/benefit: when multi-model review is worth it

Multi-model review costs real tokens — typically 1.5x to 3x the original generation cost depending on output length and review depth. That's not free. Here's an honest accounting of when it earns its keep:

Worth it

  • Code that will be merged to a shared codebase without close human review
  • Security-relevant logic (auth, permissions, data handling)
  • Customer-facing content that's hard to retract once published
  • Analysis that will drive a significant decision
  • Any output where "looks good" is insufficient because the failure mode is silent

Not worth it

  • Internal drafts that will be reviewed and edited before use
  • Exploratory analysis you'll verify with data anyway
  • Reversible edits where a quick human scan is sufficient
  • Tasks where the first model's output is primarily structured transformation (format conversion, extraction, templating) — these fail loudly, not silently

⚠️ Warning: Multi-model review doesn't work well when both models have the same blind spots — for example, using the same underlying model for both passes, or using two models trained on identical data. Partial independence is what makes it valuable. If the reviewer would make the same errors as the generator, you're spending tokens on agreement, not on finding problems.

A simple cost-benefit rule

If the cost of discovering the mistake after the fact (in time, reputation, or actual damage) is greater than the cost of a review pass now, run the review. If it's cheaper to fix later and the failure mode is visible, skip it.

👤 Human note: The best signal that multi-model review is being misapplied: it's only run when you already feel uneasy about the output. Run it on the confident outputs too. The clean-looking answer is often the one most in need of scrutiny.

A minimal working implementation

You don't need infrastructure for this. Here's the minimal pattern that works in any agent workflow with API access to two models:

# Step 1: Generate with primary model
draft = primary_model.generate(task_prompt)

# Step 2: Review with secondary model using adversarial prompt
review_prompt = f"""
You are reviewing the draft below. Find problems only.
Check in order: unsupported claims, flawed assumptions,
missing edge cases, logic gaps, safety issues, completeness.
Rate each: Blocking / Required / Suggestion.
Do not rewrite. Do not summarize.

--- DRAFT ---
{draft}
"""
findings = reviewer_model.generate(review_prompt)

# Step 3: If Blocking findings exist, resolve before use
# If Required findings exist, decide based on stakes and time
# Log Suggestions for later

The reviewer model doesn't need to be more expensive than the primary model — it needs to be different enough to have different failure modes. In practice, mixing model families (e.g., Claude for generation, GPT-4o for review, or vice versa) produces more useful disagreement than using two instances of the same model.

This is one technique in a larger quality system

Multi-model review catches errors in finished output. It doesn't address upstream problems: vague task definitions that produce confidently wrong answers, missing evidence that the first model silently fills in, or agent workflows that lack verification loops between steps.

Blueprint includes model routing tables, review protocols for different output types, and the integration patterns for embedding review loops into agent workflows so quality control is structural rather than manual.