The watch

Every decision is watched differently. A deploy that should cut errors can show its effect in minutes; a pricing change can take a quarter. Each decision's watch is set from what the decision changes and how the linked numbers have moved before.

Set for each decision

• Which number, and which part of it. Signups overall, or only signups from the country the change launched in.

• When to start and how long to watch. Right away for a fix that ships live, a few weeks later for a change that takes time to reach people.

• What to compare with. The weeks before, where the number was already heading, or the same weekdays before, for a number with a weekly rhythm.

Each choice comes with a one-line reason, shown on the decision. You can change any of them.

How big a move counts

A move counts only when it's bigger than the number's own ordinary swing over the same length of time. A number that jumps around day to day needs a bigger move than a steady one. For small counts, the bar is written in counts, such as "at least 6 more signups a week", not as a percentage.

Judging starts early

A week of readings is enough to start. Each decision shows how much history its bar rests on, and the bar gets sharper as more days come in.

A number that only starts when the decision ships is judged against what the team expected: a level or target they named, or else the first stretch after launch.

Fixed once set

The watch is fixed when the decision is kept, so a later change to ZeroWidth never changes an old verdict. A decision that couldn't be judged yet, because its numbers hadn't come in, is judged again once they have. A verdict, once given, is never replaced.