heka
Menu

Notes from the factory

Blame the codebase, not the agent

When an agent reviews or writes your code badly, look at the codebase first. It is usually missing context that lives outside the code, and the old architecture books show the fix.

Two factory floors, one behind the other, with the same three agents and the same work on each. On the far floor the code is eight blocks scattered across it with tangled threads between them; its label reads “Rules in many places”. On the near floor the same blocks are grouped into four modules named Domain model, Repository, Service and Adapter; its label reads “One place for each rule”. On the far floor two pieces of work in three are refused at the checks and go back to their benches, while on the near floor every piece passes and ships. A new engineer walks onto the far floor among the scattered blocks and stops with a question. Same agents, different codebase.

Here is a hot take: most agents are now better at code review than a human is. So if you point a review agent at your code and it cannot find mistakes that actually make sense, I do not think the agent is the problem. It is a sign that your codebase is not well architected, or not well organized, or both.

The same is true when agents write code. If you find an agent making mistakes you think another engineer would not make, you are probably wrong about the other engineer. Another engineer would make those mistakes too. They are a sign that the way the code is organized makes it hard to understand, even for something that runs on a supercomputer.

Most of the mistakes I see agents make are legacy problems, left by developers who did not want to put simple architectural patterns in place.

A review agent tests the codebase

A review agent reads the change, reads the code around it, and says what looks wrong. When its findings are vague, or point at things that are not problems, the usual reaction is to blame the model.

Look at what it was given first. If the rule that decides a price lives in three places, the reviewer cannot tell which one is right. If a rule only makes sense because of how a process ran years ago, the reviewer cannot know that. Neither can the person you hire next month.

When the review agent finds real mistakes, the shape of the code says what the code is for. When it cannot, it is showing you where that shape is missing.

Another engineer would make the same mistakes

All too often I see engineers complain that the agent is not doing what they would do. What they do not realize is why they would do it. They have context about the system that is not in the system.

They know how the process was done in the past. They know about the esoteric file that lives in some other system we need to talk to at the end of the day. None of that is in the repository, so none of it reaches the agent.

I have been shipping software for almost 30 years. Put me in that codebase today, without that context, and I would make the same mistakes the agent makes. So would any engineer new to it. The agent reads everything it is given and has nothing else to go on.

We have supercomputers analyzing our codebases. If the information they need is not in there, they are going to make the mistakes anyone would make without it. If we are blaming them for that, we are doing something wrong.

Go back to the old books

Now is the time to pick the rudimentary books on software development and architecture up again. These are the books we used to carry around and point at to say: this is how things should be architected.

In our experience, that is how agents think too. They expect a codebase built from those patterns. When we put them in a codebase that is not like that, they struggle, just like any engineer would.

What those patterns do is put knowledge into the code: a name for each idea in the business, one place for each rule, a clear edge where we talk to someone else’s system. That is exactly the context an agent is missing.

Put the context in the system

If an agent needs information that is not in the codebase, the fix is to put it there.

In our factory this is the loop: every miss becomes a fix in the factory, and a question an agent should not have had to ask becomes a guide (what Birgitta Böckeler calls a feedforward control). When an agent gets something wrong, we ask what it would have needed to know, and where that knowledge lives today.

Usually it lives outside the repository: in a person’s head, in a document nobody updates, in that file in the other system. Then the work is plain:

  • Write the decision down beside the code, as an architecture decision record.
  • Give each word the business uses one meaning, in a glossary the code follows.
  • Give each rule one home, named after the idea it holds.
  • Put the other system behind an interface that says what it expects and why.

Then run the review agent again. Its findings are a fair measure of how readable the codebase has become.

Where to start

The next time an agent makes a mistake you think an engineer would not make, ask the question you would ask about a new hire. What would they have needed to know, and could they have found it in the code? If the answer is no, the fix belongs in the codebase, and every engineer who comes after the agent gets it too.

If you would like help finding where that context lives in your codebase, talk to us.

Tell us what should exist.

Bring us something you want built, or something painful you want gone. Talk to our agent now, or email us and an engineer answers.

A voice call with our AI agent.

Email us