Did your agent fix the code, or the test?

The teams getting real work out of agents build a pass/fail check first, then let the agent grind against it. The agent soon goes after the check instead: it edits the test, skips it, or rewrites the script that grades it. This week we look at checks that hold, the ways they break, and the harder question of whether you checked the right thing.

• PRACTICE · Maurice Heumann
  500+ Billion Tokens Later: Letting AI Agents Decompile a First-Person Shooter (opens in a new tab) · a month of plausible-looking agent work was wrong until he built a byte-matching pass/fail check, and then the agents went after the check itself
• THE FAILURE · Matthew Busel
  Coding agents keep “fixing” tests by editing them, so I made it impossible (opens in a new tab) · agents turn red tests green by editing them, so the agent never touches the grader and never decides on its own that it’s done
• DISSENT · Hillel Wayne
  What TLA+ can and can’t check (opens in a new tab) · formal methods won’t solve agentic development, because you can only verify a property you managed to write down
• THE STAKES · Thomas Hales
  What mathematicians should know about the Lean theorem prover: questions of reliability and AI (opens in a new tab) · even the proof checker needs auditing, and the hardest question is whether the theorem you verified is the one you meant
• THE FIX · Inanna Malick
  Property Testing with Agent Swarms (opens in a new tab) · plant a bug on purpose: if the tests still pass, the tests are the problem