Highlights
Model Analytics Highlights
Distilled 2026-11-14 19:01 UTC from the published section write-ups alone, then checked by an independent reviewer. Each section’s own page carries the detail and the tables behind these figures.
What this shows
The Model Analytics area tracks what it costs to get a unit of work done by an AI coding tool. Each section runs its own fixed set of prompts over and over, prices those runs at the provider’s published API rates, and writes up what a task costs now, which way the cost has been heading across recent cycles, and what changed since the previous release. The two sections cover different tools and different prompt sets, and each is written up on its own terms.
Claude Benchmarks
This section measures what one task costs per session on the Claude Code CLI, running a frozen prompt catalog repeatedly and pricing it at published API rates, reported separately at the high and the medium reasoning setting. The section lists current per-session costs cheapest to most expensive, from $0.0016 for haiku-5-5 up to $0.0899 for opus-5 at the high setting; it notes separately that token capture for the Haiku models is incomplete, so Haiku cost and token figures are lower bounds, not exact amounts. Across the last 10 cycles at the high setting, opus-4-8 drifted up, staying within $0.0536 to $0.0602, while the other families with enough history showed no clear direction and fable-5-2 has too few cycles to say; across the last 10 cycles at the medium setting, every family with enough history showed no clear direction and fable-5-2 has too few cycles to say. This cycle has no percentage move to report at either setting, because at the high setting no family has sessions on both sides of the comparison with the previous CLI version, and the same is true at the medium setting; that does not say costs held steady, only that this cycle’s runs cannot be set against the previous version’s. The section also reports that comparing haiku-5-5 with opus-5 at the high setting on identical prompts, opus-5 costs 5430.7% more, a gap it calls statistically significant.
Codex Benchmarks
This section measures what one task costs per session on the Codex CLI, using its own frozen prompt catalog priced at OpenAI’s API list rates; this release ran medium reasoning only, and the section stresses that settings are not interchangeable, so every figure belongs to that setting. Current per-session costs at medium reasoning run from $0.0011 for luna-6 up to $0.1127 for astra, and across the last 10 cycles at medium reasoning luna, sol and terra showed no clear direction, while astra, luna-6 and sol-6-1 have too few cycles to say. Against the previous CLI version (0.164.0), the mean cost per session at medium reasoning moved like this:
- astra rose 22.3%
- luna fell 8.2%
- luna-6 rose 15.3%
- sol fell 6.0%
- sol-6-1 rose 18.4%
- terra fell 3.6%
The section flags that sol-6-1 moves in opposite directions depending on which summary you read: its mean per session rose 18.4% while its median fell 5.5%, and a mean and a median summarize the same sessions differently rather than correcting each other. Comparing the individual session costs of the two cycles, none of these moves can be told apart from ordinary run-to-run variation; something may have changed in each case, this measurement just cannot see it. That test is not controlled for cache state, because the study cannot tell which sessions started with a warm cache, so it is weaker evidence than a test that had removed that factor, and the section notes it is shaped like the Claude benchmark but uses different prompts, so the two should not be read as a like for like comparison.
Verification appendix
- Verifier verdict: PASS after 3 round(s).
- A first draft was rejected and revised; the objections were resolved.