The "use TDD" line in your AGENTS.md scored below writing nothing, across 80 runs per condition

Somewhere in your repository there is a file the agent reads before it reads any code. Claude Code loads CLAUDE.md at the start of every session and delivers it as a user message right after the system prompt. Codex builds an instruction chain from ~/.codex/AGENTS.md down through every AGENTS.md between the git root and your working directory, once per run, capped at 32 KiB by default. Cursor's rules files fill the same role. Whatever that file says about testing applies to every task the agent runs in that repo, whether or not you remember writing it.

A very common line in those files names a technique: "use TDD", "write property-based tests", "fuzz the parser". Until this month nobody had measured what such a line does to the result. Now someone has, and the answer for most of them is that the agent produces more test-shaped work and a less correct implementation.

The measurement

Dan Luu published the experiment this month. The setup: implement a Zstd decoder in Rust from the specification, with correctness graded by a hidden test suite. The base prompt was held constant and a one-line addendum was varied across 26 conditions, among them "Use test-driven development", "Use Lean 4", "Use QuickCheck", "Use property-based testing", "Use fuzzing", "Use mutation testing", "Audit first", "Make no mistakes", and "Default", which added nothing. Four skills were run the same way, three of them published and one the author wrote. Everything shown ran on Codex with GPT-5.6 Sol at two effort levels, medium and xhigh, 80 runs per condition per effort, and a second task, an IMAP server from the RFC, was run at 40 per condition. The metric is the fraction of runs that pass every hidden test, plotted against token cost.

Two caveats from the author come first, because they bound everything below. "Nothing really wildly outperforms", and he cautions "anyone against drawing any kind of strong conclusions from the ordering." What holds up is the shape of the result, not the rank of any single line.

The shape is this. Default, the condition with no testing instruction, finished well above average. "Use test-driven development" finished below average at higher cost. The formal-methods conditions (Verus, Alloy, Lean 4, Creusot, Kani, Spin, TLA+, ACL2, and an SMT solver with Z3, cvc5 and Yices installed) clustered around or below average, and the one that landed above Default, ACL2, did so with its many out-of-memory runs excluded from the count, which the author says would be surprising to read as causal. The property-based and fuzzing conditions did a little better than the formal ones at high effort and were mixed at medium. "Make no mistakes" was indistinguishable from random draws of Default at every level the author looked. And the author's own five-line skill, written in a couple of minutes, took the top score.

The author's one-paragraph explanation is worth quoting in full, because it is the whole finding:

Agents generally did things that were not useful when asked to use particular libraries or use particular test techniques. It stands to reason that not telling agents to do things that will make them do useless work does better than telling them to do things that will make them do useless work.

What the agent did with each line

The transcripts are where the value is, because they show what "useless work" looked like per instruction. Once you have seen these you will recognise them in your own sessions.

"Use test-driven development." Agents wrote twice as many tests. In 67 of 160 runs they had a failing test before any substantial implementation, against 0 of 160 for Default, so the instruction did change behaviour. The tests were of every kind, small and end-to-end alike. They were also worse. Zstd uses a jump table when a block carries four Huffman streams. TDD agents wrote more tests around it and were more likely to fail the hidden test for it, and the tests they wrote were more likely to skip the hard case, feeding four identical streams into a feature whose whole risk is confusing the four streams. A test that cannot distinguish stream one from stream four passes a decoder that transposes them.

"Use Verus" (or Lean 4, or Creusot). Agents wrote proofs. The proofs were about arithmetic bounds that were never where the bugs were, and a recurring form was a vacuous A => A. One real Verus block from the runs had a requires clause and an ensures clause that stated the same three inequalities. Verus agents wrote a test for the four-stream jump table in 89 of 160 runs, exactly the same count as Default, but were much more likely to encode a wrong expected value or make all four streams identical.

"Use fuzzing." Agents generated random bytes, which take one of a handful of invalid-input rejection paths and never reach the decoder. In 10 of 160 runs an agent built structured random inputs instead, and those found real bugs half the time. The capability is in the model. The word "fuzzing" does not summon it.

"Use differential testing." In 135 of 160 runs the agent did something that could be called differential testing. None built two full implementations to compare. Where a comparison might have caught a bug, the agent had written the same thing twice, with the same bug in both.

"Use the built-in Rust test framework." Double the tests at medium effort, 25 percent more at high effort, no gain in correctness. More tests of the same shape find the same bugs.

"Audit the code after implementing." 152 of 160 audited, 151 reported finding something and changed code. The audit usually ran in the same context that wrote the code and repeated the same mistake. 42 runs spun off a separate agent for the audit, and those scored worse, which the author flags may not be causal. On high effort, audit had the best correctness at a large cost increase. On medium it was below average.

What almost never happened, in any condition, was an agent using a technique the way a practitioner of that technique would. That is the pattern the naming line triggers. The agent produces the artifacts associated with the word and the work stays where it always was. Your AI tests are not testing anything catalogues what those artifacts look like in a JavaScript codebase. This measurement says that naming a better technique does not change the catalogue.

The line is everywhere, including in the skills you installed

The reason this matters beyond one experiment is how many files carry the line. GitHub code search on 10 September 2026 returns these estimates for files with those exact names in non-fork repositories:

Term in the file	AGENTS.md	CLAUDE.md
any content	931,840	770,048
"TDD"	22,656	24,864
"test-driven"	9,296	6,672
"property-based"	1,700	1,928

Somewhere between one in forty and one in thirty of these files mentions TDD by name. The counts include lines that say "do not use TDD", but that is not the common form.

The line is also the backbone of the two most-starred skill collections on GitHub. Superpowers (284,390 stars at the time of writing) ships a test-driven-development skill whose description reads "Use when implementing any feature or bugfix, before writing implementation code", and whose body states in capitals that there is to be no production code without a failing test first. The ECC collection (255,547 stars) ships a rust-testing skill, 11,798 bytes, whose description says "Follows TDD methodology" and whose cycle opens with RED, write a failing test first. That ECC skill is one of the four the experiment ran. 153 of 160 agents read it, and the earlier in the run an agent read it, the more its behaviour changed and the worse its correctness was. The runs in which the skill scored well were the ones in which the agent barely looked at it.

Superpowers was not in the experiment. One practitioner in the discussion thread reported the same shape from daily use: with the Superpowers set installed the agent "proactively adopts TDD for every new feature" and its understanding of testing "often stays superficial." That is a report, not a measurement, and it is consistent with the measurement.

AINews carries a Test-Driven Development skill with the same iron-law wording. It is a procedure for a task you pick on purpose, where you watch the failing test yourself. As a standing instruction to an autonomous agent, it is the condition that was measured, and the entry now says so.

Find the lines in your repo

From the repository root, this lists every line in an instruction file that names a testing technique or library. It is tested with the grep that ships with macOS.

grep -rniE '\b(tdd|test-driven|property-based|quickcheck|proptest|hypothesis|fuzz|fuzzing|mutation testing|snapshot test|snapshot testing|red-green)\b' \
  --include='AGENTS.md' --include='CLAUDE.md' --include='CLAUDE.local.md' \
  --include='.cursorrules' --include='*.mdc' .

On a demo repo with two files it prints:

./sub/CLAUDE.md:1:Always fuzz the decoder before committing.
./AGENTS.md:3:- Use TDD for every feature: write the failing test first.
./AGENTS.md:4:- Prefer property-based tests (proptest) for parsers.

The same pattern over the files that load into every project, and over the rules directory the repo loads at launch:

grep -rniE '\b(tdd|test-driven|property-based|quickcheck|proptest|hypothesis|fuzz|fuzzing|mutation testing|snapshot test|snapshot testing|red-green)\b' \
  ~/.claude/CLAUDE.md ~/.claude/rules ~/.codex/AGENTS.md .claude/rules 2>/dev/null

And over installed skills, restricted to the skill file itself so reference material and vendored dependencies do not flood the list:

grep -rliE '\b(tdd|test-driven|property-based|quickcheck|proptest|hypothesis|fuzz|fuzzing|mutation testing|snapshot test|snapshot testing|red-green)\b' \
  --include='SKILL.md' ~/.claude/skills ~/.codex/skills 2>/dev/null

Monday's post on auditing loaded skills covers what /skill-doctor reports about them and the one frontmatter line, disable-model-invocation: true, that keeps a skill out of the agent's automatic reach without uninstalling it.

For every hit, one question: does the line name a technique or library, or does it describe an operation the agent should perform on its own output? "Use TDD" names a technique. "Run npm test before committing" is a command, and it is the Claude Code documentation's own example of an instruction written well. The naming lines are the ones the measurement says to delete. Deleting is cheap and reversible, and the empty condition was the one that finished above average.

What to write in their place

The condition that scored highest was a skill the author describes as written in a minute or two, with no experience writing skills. It is five sentences:

Think about areas likely to have subtle bugs before implementing; for each, state likely mistakes and plausible alternative interpretations, then come up with a check where the results differ (prefer asymmetric / boundary examples on both sides of the boundary)

After implementing, for high risk areas, independently re-derive the result without context on production code and compare (fresh context, do not re-use helper functions)

When feasible, use property-based testing or randomized inputs to try to explore the space, minimizing effort on no-panic or no-crash randomization

When randomizing, lean towards inputs that will explore interesting state and code paths (don't just naively randomize inputs that all fall into the same error paths); this may require structured random inputs

If you're unsure about details, use independent reasoning to check what's correct (fresh context, do not re-use helper functions)

Read it against the transcripts above and every line is a direct counter to a failure the naming conditions produced. "A check where the results differ" is the four-identical-streams test, forbidden. "Minimizing effort on no-panic randomization" and "structured random inputs" are the random-bytes fuzzer, forbidden. "Independently re-derive" is the same-context audit, forbidden.

The author's own assessment of it is not triumphant, and you should carry the same caution. He writes that it did not work as intended: the fresh-context instruction was almost never followed, in at least one documented run an agent flagged bitstream reversal as risky, did the re-derivation, and still tested it with a palindromic input, and all the randomised testing was done by hand rather than with a library the model already handles well. He calls it a first draft. His principle for why it beat the tutorial-style skills is the transferable part:

The model is already going to have some kind of default behavior distribution, so I feel like the more natural thing to do is to give statements that will modify that behavior, not write instructions that would allow a human or non-knowledgeable agent to do the behavior at all.

Two more examples of that kind of line appear in the write-up, both from practitioners and neither measured. Jamie Brandon got agents to write real end-to-end tests with mocked IO only after moving tests into a separate crate and putting one line in AGENTS.md: keep tests in that crate and do not modify the public interface. That is a structural constraint, not a technique. Em Chu's two standing audit instructions are "read and understand the code yourself, do not spawn subagents" and "do not execute any of the code". If you already run the independent verifier subagent from July, note the tension: the measured audit condition did worse when it delegated, and the skill's fresh-context line was mostly ignored. A verifier you launch yourself, with only the claim in its context, is a different thing from a line asking the agent to remember to do that.

One more thing the transcripts show that the instruction can build on. When asked to audit and fuzz risky areas, agents at high effort correctly identified the risky areas: FSE tables, Huffman decoding, bit readers, state. They then fuzzed them with random bytes. The model knows where the bugs are likely to be. The useful line tells it what a discriminating check for that area looks like, and the most useful version of that line is specific to your codebase, which is a few minutes of your knowledge that no generic skill contains.

A third kind of line, and the cheapest

There is a category of instruction the experiment did not test because it cannot affect correctness either way: lines that shape the reply and nothing else. The clearest current example is i-have-adhd, a single-file skill at 36,542 stars, MIT, that runs in Claude Code, Cursor, Codex, Gemini, Kimi, Qwen and OpenCode. Its ten rules are all about form: "Lead with the next action", "Number multi-step tasks", "End with one concrete next action", "Restate state every turn", and a tenth rule banning preamble, recap and closing pleasantries, with the offending phrases listed. It changes what the agent writes back, not what the agent does.

That makes it the safest line you can add. A form rule cannot send the agent off to write proofs of A => A. The skill also ships with disable-model-invocation: true in its frontmatter, so it loads when you invoke it and stays out of the description budget otherwise, which is the pattern Monday's post recommended for anything you do not want auto-triggered.

So three kinds of line, in ascending order of risk: lines that shape the output, lines that describe a check the agent should perform, and lines that name a technique. The first is free. The second is where the measured gain was. The third is where the measured loss was, and it is the one most files contain.

What the measurement does not license

One harness, one model family, one task family. Codex with GPT-5.6 Sol on RFC implementations with unambiguous specs and hidden tests. The author notes that Claude and Codex "want" different prompting styles, and that much of what worked with GPT-5.5 stopped working or became unreliable with GPT-5.6, so a line that hurts here may be neutral elsewhere and the reverse. RFC tasks are also cleaner than what most people hand an agent, and his expectation is that the failure modes are the same or worse on ordinary work, but that is an expectation.

It also does not say TDD is bad. It says the addendum "Use test-driven development" handed to an autonomous agent produced worse code on this task. A person practising TDD who reads each failing test before the agent implements is a different process, and it was not measured. And it does not say skills are bad: the highest-scoring condition was a skill. It says that skills written like tutorials for a human who does not know the topic made an agent that already knows the topic do worse, and the author reports the same result in two further experiments on vendor-published skills he has not yet written up.

If you keep a technique line after reading this, keep it as a hypothesis with a number attached. You already have an eval set if you followed the July post; run the file with the line and without it, and let the pass rate decide. The one thing the measurement rules out is the assumption that the line is doing what its name says.