# Decision capture

How often each AI model records the decisions it should, and leaves alone the ones it shouldn't, given the same tools.

We give an AI assistant a work conversation and ZeroWidth's tools, then check what it records. A good assistant records each decision that was made, once and with the right details, and records nothing when nothing was decided.

Store manager

I have a three-step plan to get more visitors who buy, and I'll watch the conversion rate. Step one is reshoot every product on a white background. Step two is rewrite the product descriptions. Step three is add customer videos.

I'm only agreeing to step one: reshoot every product on a white background.

Steps two and three are not decided, so leave them as ideas for now.

Right

Recorded

Reshoot every product on a white background

GoalMore visitors who buyMetricConversion rate, up

Only the step that was agreed, recorded once.

Wrong

Recorded

Reshoot every product on a white background, rewrite the product descriptions and add customer videos

Records two steps nobody agreed to.

Each model runs every conversation in two setups:

**Tools only** The assistant has ZeroWidth's tools for recording and looking up decisions, and nothing else from us.

**Tools and instructions** The assistant also reads our short instructions on when to record a decision and what to include.

The models: Claude Sonnet 5.5, Claude Haiku 5.5, GPT-6 Luna, Gemini 3.8 Flash and DeepSeek V4.1 Flash.

An answer is right when the assistant records the decision once, or records nothing when nothing was decided. Recording something that wasn't decided is a false positive. There are eight kinds of conversation:

**A decision made** The person decides something. Record the decision.

**A decision reversed** The person undoes an earlier decision. Find the earlier one, then record the change.

**Someone else decided** The person passes on a colleague's decision. Record the decision, with the colleague as the one who decided.

**A plan partly agreed** Only the first step is agreed. Record that step only.

**A choice left for later** "We'll decide after the board meeting." Record nothing. A note that the choice was left for later is allowed.

**A choice handed to someone else** "Sam will pick the vendor." Record nothing about the choice.

**Not a decision** An idea talked through and never agreed. Record nothing.

**Off the record** The person asks to keep a decision out. Record nothing.

In the pilot, every model recorded every decision it should have. Where the models differed was the other half of the job: recording things that weren't decisions. Our instructions cut those sharply.

Right answers

83%[71–90]with tools only

96%[89–99]with tools and instructions

Both setups ran the same conversations (p = 0.02)

If a note that a choice was left for later counts as wrong: 71% → 89%

We settled some scoring rules for these runs after seeing the results. From the first full run, the conversations and rules are fixed before any model runs.

## Results by model

| Model | Tools only | Tools and instructions | Change |
|---|---|---|---|
| Claude Haiku 5.5 | 81% [57–93]. 13 of 16 | 100% [81–100]. 16 of 16 | ↑ +19 points |
| Gemini 3.8 Flash | 80% [49–94]. 8 of 10 | 100% [74–100]. 11 of 11 | ↑ +20 points |
| DeepSeek V4.1 Flash | 88% [53–98]. 7 of 8 | 100% [80–100]. 15 of 15 | ↑ +13 points |
| GPT-6 Luna | 85% [58–96]. 11 of 13 | 94% [72–99]. 15 of 16 | ↑ +9 points |
| Claude Sonnet 5.5 | 81% [57–93]. 13 of 16 | 88% [64–97]. 14 of 16 | ↑ +6 points |

From the pilot, with the default setup

## Each measure

**Right answers**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of conversations

**Right answers**. Second run

Tools onlyTools and short instructionsTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of conversations

**Strict right answers**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of conversations

**Strict right answers**. Second run

Tools onlyTools and short instructionsTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of conversations

**Decisions the assistant recorded**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of conversations

**Decisions the assistant recorded**. Second run

Tools onlyTools and short instructionsTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of conversations

**False positives**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of conversations

**Recorded a decision the person asked to keep out**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5DeepSeek V4.1 FlashGPT-6 Luna

Share of conversations

**Recorded the same decision twice**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of conversations

**Recorded the same decision twice**. Second run

Tools onlyTools and short instructionsTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of conversations

**Records with a number nobody said**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of records

**Records with a number nobody said**. Second run

Tools onlyTools and short instructionsTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Share of records

**Correct details per record**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Correct details

**Correct details per record**. Second run

Tools onlyTools and short instructionsTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Correct details

**Tokens ZeroWidth adds per conversation**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Tokens

**Tokens ZeroWidth adds per conversation**. Second run

Tools onlyTools and short instructionsTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

Tokens

**Cost per conversation**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

US dollars

**Cost per conversation**. Second run

Tools onlyTools and short instructionsTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

US dollars

**Cost per recorded decision**. Pilot

Tools onlyTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

US dollars

**Cost per recorded decision**. Second run

Tools onlyTools and short instructionsTools and instructions

Claude Sonnet 5.5Claude Haiku 5.5Gemini 3.8 FlashDeepSeek V4.1 FlashGPT-6 Luna

US dollars

## By kind of conversation

|  | Said outright | Over several turns |
|---|---|---|
| Kind | Tools only | Tools and instructions | Tools only | Tools and instructions |
| A decision made | 4 of 4 | 5 of 5 | 4 of 4 | 3 of 3 |
| A decision reversed | 5 of 5 | 5 of 5 | 4 of 4 | 5 of 5 |
| Someone else decided | 5 of 5 | 5 of 5 | 4 of 4 | 5 of 5 |
| A plan partly agreed | 0 of 4 | 5 of 5 | 1 of 5 | 4 of 5 |
| A choice left for later | 5 of 5 | 5 of 5 | 5 of 5 | 5 of 5 |
| A choice handed to someone else | 2 of 2 | 4 of 4 | 2 of 3 | 2 of 4 |
| Not a decision | 2 of 2 | 5 of 5 | 5 of 5 | 5 of 5 |
| Off the record | 0 of 2 | 4 of 4 | 4 of 4 | 4 of 4 |

## Model by model

### Claude Sonnet 5.5

Anthropic, added October 31, 2026

|  | Tools only. Pilot | Tools and instructions. Pilot | Tools only. Second run | Tools and short instructions. Second run | Tools and instructions. Second run |
|---|---|---|---|---|---|
| Right answers | 81% | 88% | 75% | 88% | 100% |
| Strict right answers | 75% | 81% | 75% | 88% | 100% |
| Decisions the assistant recorded | 100% | 100% | 100% | 100% | 100% |
| False positives | 25% | 0% |  |  |  |
| Recorded a decision the person asked to keep out | 50% | 0% |  |  |  |
| Recorded the same decision twice | 75% | 63% | 0% | 13% | 0% |
| Records with a number nobody said | 0% | 0% | 0% | 0% | 0% |
| Correct details per record | 4.00 | 3.82 | 4.75 | 5.11 | 5.63 |
| Tokens ZeroWidth adds per conversation | 16.6K | 35.7K | 24.9K | 31.2K | 53.2K |
| Cost per conversation | $0.10 | $0.11 | $0.12 | $0.11 | $0.14 |
| Cost per recorded decision | $0.21 | $0.22 | $0.12 | $0.11 | $0.14 |
| Detail | Tools only. Pilot | Tools and instructions. Pilot | Tools only. Second run | Tools and short instructions. Second run | Tools and instructions. Second run |
|---|---|---|---|---|---|
| What was decided | 1.00 18 records | 1.00 15 records | 1.00 8 records | 1.00 9 records | 1.00 8 records |
| Why | 0.82 11 records | 1.00 7 records | 0.43 7 records | 1.00 5 records | 1.00 5 records |
| Goal | 1.00 10 records | 1.00 9 records | 1.00 5 records | 1.00 7 records | 1.00 6 records |
| Metric | 1.00 7 records | 1.00 6 records | 1.00 4 records | 1.00 5 records | 1.00 6 records |
| Which way is good | 1.00 10 records | 1.00 6 records | 1.00 5 records | 1.00 7 records | 1.00 6 records |
| When to check | 1.00 4 records | 1.00 3 records | 1.00 4 records | 1.00 2 records | 1.00 4 records |
| Who decided | 1.00 4 records | 1.00 2 records | 1.00 2 records | 1.00 2 records | 1.00 2 records |

### Claude Haiku 5.5

Anthropic, added October 31, 2026

|  | Tools only. Pilot | Tools and instructions. Pilot | Tools only. Second run | Tools and short instructions. Second run | Tools and instructions. Second run |
|---|---|---|---|---|---|
| Right answers | 81% | 100% | 75% | 88% | 88% |
| Strict right answers | 75% | 94% | 75% | 88% | 88% |
| Decisions the assistant recorded | 100% | 100% | 100% | 100% | 100% |
| False positives | 25% | 0% |  |  |  |
| Recorded a decision the person asked to keep out | 50% | 0% |  |  |  |
| Recorded the same decision twice | 100% | 38% | 13% | 0% | 13% |
| Records with a number nobody said | 0% | 0% | 0% | 0% | 0% |
| Correct details per record | 3.27 | 3.42 | 4.78 | 5.00 | 5.00 |
| Tokens ZeroWidth adds per conversation | 18.7K | 35.6K | 22.8K | 28.0K | 48.0K |
| Cost per conversation | $0.005 | $0.005 | $0.006 | $0.005 | $0.007 |
| Cost per recorded decision | $0.010 | $0.011 | $0.006 | $0.005 | $0.007 |
| Detail | Tools only. Pilot | Tools and instructions. Pilot | Tools only. Second run | Tools and short instructions. Second run | Tools and instructions. Second run |
|---|---|---|---|---|---|
| What was decided | 1.00 20 records | 1.00 11 records | 1.00 9 records | 1.00 8 records | 1.00 9 records |
| Why | 0.50 12 records | 0.67 6 records | 0.60 5 records | 1.00 5 records | 0.67 6 records |
| Goal | 0.82 11 records | 1.00 5 records | 1.00 6 records | 1.00 5 records | 1.00 7 records |
| Metric | 1.00 7 records | 1.00 3 records | 1.00 5 records | 1.00 4 records | 0.71 7 records |
| Which way is good | 1.00 10 records | 1.00 4 records | 1.00 6 records | 1.00 5 records | 0.71 7 records |
| When to check | 1.00 4 records |  | 1.00 4 records | 1.00 3 records | 1.00 4 records |
| Who decided | 1.00 4 records | 1.00 2 records | 1.00 2 records | 1.00 2 records | 1.00 2 records |

### Gemini 3.8 Flash

Google, added October 31, 2026

|  | Tools only. Pilot | Tools and instructions. Pilot | Tools only. Second run | Tools and short instructions. Second run | Tools and instructions. Second run |
|---|---|---|---|---|---|
| Right answers | 80% | 100% | 71% | 86% | 100% |
| Strict right answers | 60% | 91% | 71% | 86% | 100% |
| Decisions the assistant recorded | 100% | 100% | 100% | 100% | 100% |
| False positives | 0% | 0% |  |  |  |
| Recorded the same decision twice | 86% | 43% | 14% | 14% | 0% |
| Records with a number nobody said | 0% | 0% | 0% | 0% | 0% |
| Correct details per record | 3.55 | 3.55 | 5.00 | 4.63 | 4.00 |
| Tokens ZeroWidth adds per conversation | 24.9K | 46.0K | 29.5K | 34.0K | 47.9K |
| Cost per conversation | $0.020 | $0.033 | $0.026 | $0.030 | $0.029 |
| Cost per recorded decision | $0.046 | $0.075 | $0.030 | $0.034 | $0.039 |
| Detail | Tools only. Pilot | Tools and instructions. Pilot | Tools only. Second run | Tools and short instructions. Second run | Tools and instructions. Second run |
|---|---|---|---|---|---|
| What was decided | 1.00 18 records | 1.00 10 records | 1.00 8 records | 1.00 8 records | 1.00 6 records |
| Why | 0.08 13 records | 1.00 6 records | 0.67 6 records | 1.00 4 records | 1.00 4 records |
| Goal | 0.83 12 records | 0.50 4 records | 1.00 6 records | 1.00 6 records | 1.00 3 records |
| Metric | 1.00 8 records | 1.00 3 records | 1.00 4 records | 0.60 5 records | 0.33 3 records |
| Which way is good | 1.00 10 records | 1.00 3 records | 1.00 6 records | 0.67 6 records | 0.33 3 records |
| When to check | 0.50 4 records | 1.00 1 record | 1.00 3 records | 1.00 2 records | 1.00 1 record |
| Who decided | 1.00 3 records | 1.00 3 records | 1.00 2 records | 1.00 2 records | 1.00 2 records |

### DeepSeek V4.1 Flash

DeepSeek, added October 31, 2026

|  | Tools only. Pilot | Tools and instructions. Pilot | Tools only. Second run | Tools and short instructions. Second run | Tools and instructions. Second run |
|---|---|---|---|---|---|
| Right answers | 88% | 100% | 75% | 67% | 100% |
| Strict right answers | 75% | 93% | 75% | 67% | 100% |
| Decisions the assistant recorded | 100% | 100% | 100% | 100% | 100% |
| False positives | 0% | 0% |  |  |  |
| Recorded a decision the person asked to keep out | 0% | 0% |  |  |  |
| Recorded the same decision twice | 100% | 43% | 0% | 33% | 0% |
| Records with a number nobody said | 0% | 0% | 25% | 0% | 0% |
| Correct details per record | 3.17 | 3.85 | 3.50 | 4.50 | 5.00 |
| Tokens ZeroWidth adds per conversation | 24.9K | 35.7K | 36.2K | 34.0K | 53.3K |
| Cost per conversation | $0.003 | $0.004 | $0.003 | $0.003 | $0.004 |
| Cost per recorded decision | $0.012 | $0.008 | $0.006 | $0.004 | $0.004 |
| Detail | Tools only. Pilot | Tools and instructions. Pilot | Tools only. Second run | Tools and short instructions. Second run | Tools and instructions. Second run |
|---|---|---|---|---|---|
| What was decided | 1.00 11 records | 1.00 12 records | 1.00 4 records | 1.00 8 records | 1.00 8 records |
| Why | 0.00 8 records | 1.00 5 records | 0.33 3 records | 1.00 4 records | 1.00 5 records |
| Goal | 1.00 5 records | 1.00 7 records | 1.00 1 record | 1.00 4 records | 1.00 4 records |
| Metric | 1.00 4 records | 1.00 5 records | 1.00 2 records | 1.00 5 records | 1.00 5 records |
| Which way is good | 1.00 5 records | 1.00 5 records | 1.00 2 records | 1.00 4 records | 1.00 4 records |
| When to check | 1.00 1 record | 1.00 1 record | 0.00 2 records | 1.00 1 record | 1.00 4 records |
| Who decided | 1.00 2 records | 1.00 2 records | 1.00 2 records | 1.00 2 records | 1.00 2 records |

### GPT-6 Luna

OpenAI, added October 31, 2026

|  | Tools only. Pilot | Tools and instructions. Pilot | Tools only. Second run | Tools and short instructions. Second run | Tools and instructions. Second run |
|---|---|---|---|---|---|
| Right answers | 85% | 94% | 75% | 88% | 100% |
| Strict right answers | 69% | 88% | 75% | 88% | 100% |
| Decisions the assistant recorded | 100% | 100% | 100% | 100% | 100% |
| False positives | 0% | 0% |  |  |  |
| Recorded a decision the person asked to keep out | 0% | 0% |  |  |  |
| Recorded the same decision twice | 100% | 50% | 13% | 13% | 0% |
| Records with a number nobody said | 0% | 0% | 0% | 0% | 0% |
| Correct details per record | 2.72 | 3.40 | 3.10 | 3.56 | 3.63 |
| Tokens ZeroWidth adds per conversation | 16.6K | 40.8K | 20.8K | 21.9K | 42.6K |
| Cost per conversation | $0.001 | $0.001 | $0.001 | $0.001 | $0.001 |
| Cost per recorded decision | $0.002 | $0.003 | $0.001 | $0.001 | $0.001 |
| Detail | Tools only. Pilot | Tools and instructions. Pilot | Tools only. Second run | Tools and short instructions. Second run | Tools and instructions. Second run |
|---|---|---|---|---|---|
| What was decided | 1.00 21 records | 1.00 13 records | 1.00 9 records | 1.00 9 records | 1.00 8 records |
| Why | 0.00 16 records | 0.50 8 records | -0.14 7 records | 0.60 5 records | 1.00 4 records |
| Goal | 0.27 11 records | 1.00 8 records | 0.60 5 records | 1.00 4 records | 0.33 6 records |
| Metric | 0.75 8 records | 0.60 5 records | 0.60 5 records | 1.00 1 record | 0.50 4 records |
| Which way is good | 0.75 8 records | 0.60 5 records | 0.60 5 records | 1.00 2 records | 0.33 3 records |
| When to check | 1.00 3 records | 1.00 3 records | 1.00 2 records | 1.00 3 records | 1.00 2 records |
| Who decided | 1.00 4 records | 1.00 2 records | 1.00 2 records | 1.00 2 records | 1.00 2 records |

## Other setups

Claude Haiku 5.5 with many other tools, and with a product's own system prompt.

| Model | Other tools | System prompt | Setup | Right answers | False positives |
|---|---|---|---|---|---|
| Claude Haiku 5.5 | many | product | Tools only | 81% [57–93], 13 of 16 | 25% [5–70], 1 of 4 |
| Claude Haiku 5.5 | many | product | Tools and instructions | 100% [81–100], 16 of 16 | 0% [0–49], 0 of 4 |
| Claude Haiku 5.5 | none | none | Tools only | 81% [57–93], 13 of 16 | 25% [5–70], 1 of 4 |
| Claude Haiku 5.5 | none | none | Tools and instructions | 100% [81–100], 16 of 16 | 0% [0–49], 0 of 4 |

## Scoring

Every conversation is written from a fact sheet, so the right answer is set before any model runs.

Each record is checked against the fact sheet, detail by detail: what was decided, why, the goal, the metric and which way is good, when to check, and who decided. A number the person never said counts as made up. If the assistant calls tools six times in a row without replying, the conversation counts as failed.

When a choice is left for later, the assistant may note that the choice is still open. That note counts as neither right nor wrong. The strict chart counts it as wrong.

Every rate comes with a 95% range. Both setups run on the same conversations, so the gap between them is the effect of the instructions.

## Private test set

From the first full run, we keep a quarter of the conversations private, so no model can be tuned to them. Each model is scored on both sets, and the page flags any model that scores much higher on the public set than on the private one. The pilot has no private set yet.

## Changes
- October 31, 2026 First run. Five models ran every conversation twice: once with ZeroWidth's tools only, and once with ZeroWidth's tools and instructions.Claude Sonnet 5.5, Claude Haiku 5.5, Gemini 3.8 Flash, DeepSeek V4.1 Flash, GPT-6 Luna
- October 31, 2026 Second run, on the eight conversations that hold a decision. ZeroWidth now tells the assistant when a decision is already recorded, so the assistant stops recording the same decision twice. A third setup joined: tools and short instructions, about a fifth the length of the full instructions.Claude Sonnet 5.5, Claude Haiku 5.5, Gemini 3.8 Flash, DeepSeek V4.1 Flash, GPT-6 Luna

## The data

[Download the results (JSON)](https://zerowidth.ai/research/decision-capture.results.json) [Download results by model and setup (CSV)](https://zerowidth.ai/research/decision-capture.csv)