Research
Benchmarks and experiments on what AI models do, with the code to run each one yourself.
Decision capture
How often each AI model records the decisions it should, and leaves alone the ones it shouldn't, given the same tools.
Benchmark·Updated ·v0.1
Share of conversations recorded right - DeepSeek V4.1 Flash
- Claude Haiku 5.5
- Gemini 3.8 Flash
- GPT-6 Luna
- Claude Sonnet 5.5
Tools onlyTools and instructions
Shims
A shim is a small model that makes one decision inside your app, in the browser, and says so when it should not answer. Every demo on this page runs in your browser.
Experiment·Updated ·v0.1.2
Accuracy on CLINC150, 150 kinds of request - A frontier model, 8 examples per answer97.9%
- A shim, 32 examples per answer95.0%
- Keyword matching84.6%
- A zero-shot classifier52.0%
In the browserHosted