Diff your agent CLI, not its changelog: four checks that find what shipped

The changelog entry for Claude Code 2.1.258 is two lines long. The first one reads:

Fixed Claude Code failing to launch on macOS 12 (Monterey), a regression introduced in 2.1.255

Version 2.1.255 does not appear in that changelog. It is not on npm, where the published history jumps straight from 2.1.252 to 2.1.257. It has no git tag in anthropics/claude-code, which tags its published releases. Three public surfaces, and none of them has heard of the version that the vendor's own release note holds responsible.

That is a small thing on its own. It stops being small if you were sitting on 2.1.252 on a Monterey machine, trying to work out from the changelog whether the launch failure applied to you. The changelog cannot answer that question, because the release it points at was never published to you.

This is not one vendor being sloppy. A release note is a summary, written by someone whose priorities are not yours, filtered through whatever the release tooling happened to capture. If you pin an agent CLI to a tag in CI, or your prompts and evaluation sets assume a tool behaves a particular way, you need a way to read the release itself. Here are four checks that do that, with the commands and what they returned when run on 3 September 2026.

Start with the version list, because it has holes

Before reading any release text, get the list of versions that exist. For a project that tags every ship, gaps in that list are informative:

gh api "repos/anthropics/claude-code/tags?per_page=100" --jq '.[].name' \
  | sed 's/^v//' \
  | awk -F. '$1==2 && $2==1 && $3>=240 {print $3}' \
  | sort -n \
  | awk 'NR>1 && $1!=prev+1 {for(i=prev+1;i<$1;i++) print "MISSING 2.1."i} {prev=$1}'

Seven holes came back in that range alone:

MISSING 2.1.242
MISSING 2.1.244
MISSING 2.1.249
MISSING 2.1.253
MISSING 2.1.254
MISSING 2.1.255
MISSING 2.1.256

A gap means a build was cut and then pulled, or shipped to a channel that is not the one you read. Either way, "I will read the changelog from my version to current" quietly skips whatever happened inside it, and the 2.1.255 line is what it looks like when one of those versions leaks back into the record.

Run the same scan against npm and the two lists disagree, which is the more useful result:

npm view @anthropic-ai/claude-code time --json

npm has 2.1.240, 241, 242, 243, 245 through 248, 250 through 252, then 257 onward. The tag list has no 2.1.242 at all. One version was published to the registry without a corresponding tag, so neither surface is a complete record on its own, and the tag-gap scan above reports a hole that is not one. Treat a gap as a question rather than an answer, and check it against the other surface before concluding a version never existed. What makes 2.1.255 different from 2.1.242 is that it is missing from every surface and still named in a release note.

There is a second reason not to trust the shape of the list. The order the releases API returns is neither publish order nor creation order. Checked against openai/codex this morning, the first ten entries came back with at least two inversions: rust-v0.153.0 (published 3 September 01:37 UTC) sits above rust-v0.154.0-alpha.1 (09:29 UTC the same day), and further down rust-v0.152.0 (1 September 01:58 UTC) sits above rust-v0.153.0-alpha.4 (1 September 21:02 UTC). Check it yourself:

gh api "repos/openai/codex/releases?per_page=10" \
  --jq '.[] | "\(.tag_name)\t\(.published_at)"'

Sort by published_at yourself rather than reading the page top to bottom. The top of the list is not the newest thing.

When the release body is a placeholder

Codex publishes prerelease builds constantly, and their release notes are empty. Not thin, empty. Ask for the body lengths:

gh api "repos/openai/codex/releases?per_page=20" \
  --jq '.[] | "\(.tag_name)\t\(.body|length)"'

Every alpha comes back between 25 and 28 characters, because the body is the word Release followed by the version number and a line ending, and nothing else. The stable releases in the same window are 8,349 characters (rust-v0.152.0) and 13,139 characters (rust-v0.153.0). If you track alphas, the release feed tells you a build exists and nothing else.

For an open-source project the content is still recoverable, because the commits are right there:

gh api repos/openai/codex/compare/rust-v0.153.0-alpha.2...rust-v0.153.0-alpha.4 \
  --jq '.commits[].commit.message'

Thirty-three commits, with full bodies, between two releases whose combined release notes run to 50 characters.

One more thing that shows up here: rust-v0.153.0-alpha.3 has a git tag but no release. The tag resolves, gh api repos/openai/codex/git/ref/tags/rust-v0.153.0-alpha.3 returns refs/tags/rust-v0.153.0-alpha.3, while asking for the release by that tag returns a 404. Tags and releases are separate objects on GitHub and they go out of sync. Query tags when you want to know what exists, releases when you want to know what was announced.

The line that matters is filed under "Chores"

Codex 0.152.0 has a real changelog, 8,349 characters of it, under four headings. The first of the two lines under ## Chores reads:

The planning tool is disabled by default; enable it with tools.update_plan.enabled = true. (#41744)

A tool that was on is now off. Anyone whose workflow assumed the planning tool was available, including prompts and evaluation harnesses that check for it, gets different behavior after upgrading, and the notice lives under the heading reserved for things that do not matter.

Rather than trusting the author's choice of heading, search the release for the vocabulary of behavior change. The commit subjects work better than the curated summary:

gh api repos/openai/codex/compare/rust-v0.151.0...rust-v0.152.0 \
  --jq '.commits[].commit.message | split("\n")[0]' \
  | grep -iE 'default|opt-in|opt out|no longer|remove|disable|breaking'

Ninety commits reduce to six lines, and one of them is Make the update_plan tool opt-in (#41744). The same grep run against release prose (default, opt-in, no longer, removed, deprecated) is worth doing regardless of what the headings claim.

Documentation can ship before the release note

The reverse failure is worth watching for too, because it means the release feed is the slower surface.

Google's /boost command for Antigravity is a multi-agent reasoning pipeline for paid plans. The documentation page for it was live and complete on 1 September at 00:11 UTC, which is verifiable in the Internet Archive snapshot of antigravity.google/docs/boost/ from that timestamp, note on plan availability included. The Antigravity changelog first mentions the feature in the 2.12.0 entry, dated 2 September. The 2.11.0 entry from 26 August, which is the version that was current when the documentation went up, covers generative UI, @path/to/file in AGENTS.md, chart rendering, split terminals and the Darcula theme, and says nothing about /boost.

The release note caught up. It just did so a day later than the documentation tree, and by then the feature had already been discussed publicly. If your only monitor is a release feed, the documentation tree is where a feature can appear first, and a diff of the docs sitemap catches what the feed does not.

Where none of this works, and what to do instead

The compare trick depends on source being in the repository. It is not, for Claude Code. Running gh api repos/anthropics/claude-code/compare/v2.1.252...v2.1.257 returns a single commit touching two files, CHANGELOG.md and feed.xml. The repository is a distribution and issue-tracking surface, not the source.

So for a closed-source CLI the changelog really is the only narrative you get, which makes a missing 2.1.255 more consequential there, not less. What remains available is the tag list, the npm publish history, and one thing that is easy to miss: Anthropic publishes the changelog as an Atom feed at https://raw.githubusercontent.com/anthropics/claude-code/main/feed.xml, one entry per version, which is a better monitor than polling a markdown file. It inherits the same gaps, 252 followed directly by 257, so it delivers the holes faster rather than filling them.

For npm-distributed tools, publish times answer questions the changelog does not:

npm view @anthropic-ai/claude-code time --json

That is where 2.1.252 on 31 August and 2.1.257 on 1 September sit next to each other with nothing between them.

Turning it into something that runs

Four checks, in the order that costs least:

1. Pull the tag list and look for gaps before reading any prose.
2. Ask for release body lengths. Anything under about 100 characters is a placeholder and needs a compare.
3. Grep the commit subjects between your version and the target for default, opt-in, no longer, remove, disable, breaking, and read those regardless of which heading they were filed under.
4. Watch the documentation tree alongside the release feed, since either one can be first.

None of that needs to be manual. The whole sequence is gh and npm calls that return JSON, which makes it a natural fit for a scheduled job that runs before your weekly dependency bump and posts the result somewhere you read. If you already run a scheduled competitor watch, this is the same shape of job pointed at your own toolchain.

The check that pays for itself fastest is the third one. A default that moves is the change most likely to alter what your agent does, and it is the change least likely to be written up as important by the person who shipped it.

Related: Two model IDs change meaning on August 5 covers the same problem one layer up, where the thing moving under you is the model rather than the CLI. The tools themselves are at Claude Code, Codex and Antigravity.