Codex CLI 0.165.0 Analysis

Measured 2026-11-14 19:00 UTC from the benchmark runs this release recorded, written up from those measurements alone, then checked by an independent reviewer. Where the wording and the tables disagree, the tables are correct.

Verdict

At medium reasoning, one task costs this much per session in this release, cheapest to most expensive:

• luna-6: $0.0011
• luna: $0.0023
• sol-6-1: $0.0195
• terra: $0.0234
• sol: $0.0545
• astra: $0.1127

Across the last 10 cycles at medium reasoning, each model’s direction and the dollar band it stayed inside:

• astra: too few cycles to say, between $0.0921 and $0.1127
• luna: no clear direction, between $0.0023 and $0.0031
• luna-6: too few cycles to say, between $0.0009 and $0.0011
• sol: no clear direction, between $0.0490 and $0.0597
• sol-6-1: too few cycles to say, between $0.0165 and $0.0195
• terra: no clear direction, between $0.0225 and $0.0243

Against the previous CLI version (0.164.0), the mean cost per session at medium reasoning moved like this:

• astra rose 22.3%
• luna fell 8.2%
• luna-6 rose 15.3%
• sol fell 6.0%
• sol-6-1 rose 18.4%
• terra fell 3.6%

Comparing the individual session costs of the two cycles, none of these moves can be told apart from ordinary run-to-run variation; something may have changed in each case, this measurement just cannot see it. That test is also not controlled for cache state, because this study cannot tell which sessions started with a warm cache, so it is weaker evidence than a test that had removed that factor.

One more thing worth knowing: at medium reasoning, luna-6 costs 99.0% less than astra.

Which cost figure is which

The same sessions summarized three ways. They disagree on purpose, and none is the corrected version of another.

• Mean $/session: multiplies out to real spend, which makes it the budgeting number, and it is what the cost chart plots.
• Median $/session: shrugs off an unusually cheap or expensive run, but it does not sum to real spend.
• Suite cost: the price of one full pass over every prompt in the catalog, the per-prompt medians added up, so it answers what a whole run costs rather than what one session costs.

What these terms mean

New to this report? Here is the vocabulary used below, in plain words. Skip ahead if you already know it.

• Session (also “task”): one complete run of one prompt by one model, start to finish. Every cost in this report is per session.
• Reasoning setting: how much internal reasoning the model is asked to spend before answering. This benchmark runs each setting separately and reports them separately, because the same model at a different setting is a different cost.
• Mean cost per session: the ordinary average across every session. It multiplies out to real spend, so it is the budgeting figure and the one this report leads with.
• Median cost per session: the middle session once they are lined up by cost. It ignores an unusually cheap or expensive run, but it does not add up to real spend.
• Suite cost: the price of one full pass over every prompt in the catalog, found by taking each prompt’s median and adding those up. It is a total for the whole catalog, not an average per session, so it is a larger number than either figure above and is not comparable to them.
• Cache, and cache-read tokens: the tool reuses recent context instead of sending it again, which is far cheaper than sending it fresh. Most of what a session sends is reused this way, so how much of a session lands in the cache moves its cost.
• n: how many sessions sit behind a number. A larger n is more trustworthy.
• p-value, and “significant”: the same prompt run twice costs different amounts, so a p-value asks whether two sets of runs can be told apart at all. Line up every run from both cycles: if one cycle’s runs sit consistently higher, p is near 0 and the difference is called “significant”; if the runs are mixed together, p is near 1 and the two sets look like the same thing measured twice. A high p is not weak evidence of a change, it is no evidence either way.
• API-equivalent cost: dollars calculated at published API rates. It is not necessarily dollars billed, because a subscription plan may already cover the usage.

What a task costs

Each row is one model at one reasoning setting, measured over 90 sessions in this release. This release ran medium reasoning only, so every figure here belongs to that setting.

model	reasoning	n	mean $/sess	median $/sess	suite $
astra	medium	90	0.1127	0.1018	0.8447
luna	medium	90	0.0023	0.0023	0.0209
luna-6	medium	90	0.0011	0.0010	0.0099
sol	medium	90	0.0545	0.0536	0.4885
sol-6-1	medium	90	0.0195	0.0156	0.1385
terra	medium	90	0.0234	0.0237	0.2076

Compared with the previous version

The version before this one in the published order is 0.164.0. Every model has a same-setting comparison at medium reasoning, and each side of each comparison covers 90 sessions.

model	reasoning	prev CLI	mean $/sess	mean delta %	median $/sess	median delta %	suite $	suite delta %	n (prev -> cur)	Mann-Whitney p
astra	medium	0.164.0	0.0921 -> 0.1127	+22.3%	0.1007 -> 0.1018	+1.1%	0.8193 -> 0.8447	+3.1%	90 -> 90	0.088
luna	medium	0.164.0	0.0025 -> 0.0023	-8.2%	0.0025 -> 0.0023	-5.5%	0.0227 -> 0.0209	-8.1%	90 -> 90	0.327
luna-6	medium	0.164.0	0.0009 -> 0.0011	+15.3%	0.0010 -> 0.0010	+0.1%	0.0085 -> 0.0099	+15.7%	90 -> 90	0.120
sol	medium	0.164.0	0.0579 -> 0.0545	-6.0%	0.0563 -> 0.0536	-4.8%	0.5037 -> 0.4885	-3.0%	90 -> 90	0.401
sol-6-1	medium	0.164.0	0.0165 -> 0.0195	+18.4%	0.0165 -> 0.0156	-5.5%	0.1439 -> 0.1385	-3.8%	90 -> 90	0.883
terra	medium	0.164.0	0.0243 -> 0.0234	-3.6%	0.0238 -> 0.0237	-0.3%	0.2155 -> 0.2076	-3.7%	90 -> 90	0.710

The comparison asks, for each model, whether the individual session costs of the two cycles can be told apart at all. Every p-value here sits above the 0.05 line, the closest being astra at p=0.088, so none of these moves can be separated from ordinary run-to-run variation. That is not a finding that the costs held steady: a real change could be sitting underneath any of these rows without this measurement being able to see it.

One row moves in opposite directions depending on which summary you read: sol-6-1’s mean per session rose 18.4% while its median fell 5.5%. A mean and a median summarize the same sessions in different ways and are not corrections of each other.

Caveats

• The dollar figures are API-equivalent costs worked out at published list prices, not money actually billed. If you are on a subscription plan, that plan may already cover the same usage.
• Every model is priced at its standard published rate. Promotional pricing in force today is deliberately left out, because applying a temporary rate to the whole history would make every past cycle look cheaper than it was when measured.
• Whether a move between versions is bigger than ordinary run-to-run variation is tested by comparing the individual session costs of the two cycles. That test is not controlled for cache state: it runs across every session, because this study cannot tell which sessions started with a warm cache. Cache luck therefore sits on both sides of every comparison, so treat each result as weaker evidence than a test that had removed it.
• The two cycles being compared ran different CLI versions, so a move captures the effect of the version change together with anything else that differed on the day. It is not a statement about the models themselves.
• Each reasoning setting is reported on its own, and settings are not interchangeable. This release ran medium reasoning; the same model at a different setting is a different cost.
• The timings behind these runs are wall-clock and include the time the model spends on its own reasoning.
• These runs use their own frozen set of prompts. The study is shaped like the Claude benchmark, but the prompts are not the same prompts, so the two should not be read as a like for like comparison.

---

Verification appendix

• Verifier verdict: PASS after 2 round(s).
• A first draft was rejected and revised; the objections were resolved.