Showing archive results for 2026

Jul 8, 2026
Post comments count0
Post likes count0

The hidden variables in your agent eval

Waldek Mastykarz

This is the seventh article in a series about Agent Experience (AX): the practice of making AI coding agents work correctly with your technology. The series covers what you can and can't control in the agent stack, how to measure whether your extensions are helping or hurting, and how to iterate toward better outcomes. You build an eval. You run i...

AI
Jul 7, 2026
Post comments count0
Post likes count0

Don’t rewrite your CLI for agents

Waldek Mastykarz

There's advice making the rounds: replace your CLI args with a single payload so agents can use your tool more effectively. The thinking being, that agents already think in structured formats, and nested data maps cleanly to JSON. Flat args on the other hand, force awkward conventions like repeating to delimit multi-value groups, which is inheren...

AI
Jul 6, 2026
Post comments count0
Post likes count1

Not all model upgrades are upgrades

Waldek Mastykarz

A new model drops with lower per-token pricing and better benchmarks. You switch. A week later someone asks why the agent is burning 12x more tokens on the same task while producing worse output. We ran 150 agent tasks across 15 scenarios on two models, Claude Sonnet 4.6 and Claude Sonnet 5, using GitHub Copilot Chat in VS Code on Windows. The sce...

AI
Jul 1, 2026
Post comments count0
Post likes count0

What AI benchmarks are not telling you

Waldek Mastykarz

This is the sixth article in a series about Agent Experience (AX): the practice of making AI coding agents work correctly with your technology. The series covers what you can and can't control in the agent stack, how to measure whether your extensions are helping or hurting, and how to iterate toward better outcomes. We love benchmarks. A new mode...

AI
Jun 25, 2026
Post comments count0
Post likes count1

Your agent already has a plan

Garry Trinder

If an agent isn't doing the right thing, the obvious move is to make the docs clearer. Add a tip, spell out the correct command, describe the right approach more prominently. You do all of that, and the agent still ignores it. It does what it had already decided to do. The tip wasn't ignored because it was unclear, it was ignored because the ag...

AI
Jun 24, 2026
Post comments count2
Post likes count2

When the model has never seen your code

Waldek Mastykarz

This is the fifth article in a series about Agent Experience (AX): the practice of making AI coding agents work correctly with your technology. The series covers what you can and can't control in the agent stack, how to measure whether your extensions are helping or hurting, and how to iterate toward better outcomes. Everything we've covered so fa...

AI
Jun 22, 2026
Post comments count0
Post likes count3

Models don’t have preferences, they have context

Waldek Mastykarz

You open a fresh chat, type "What framework should I use for a web app?", and the model says "React." You screenshot it, share it, and write "Claude prefers React." It gets engagement. People nod along. A few reply with their own results. And now we have a consensus: Claude prefers React. Except it doesn't. The model doesn't prefer anything. You're...

AI