How to Configure AI Coding Agents Like an Engineering Team

Quick answer AI coding agents don't become a reliable engineering team because you write a clever prompt or add more agents. They become reliable when you give them what a good team already runs on: a short list of house rules, limited access, rules that are enforced automatically, tests that check their work, and a second pair of eyes on every change. Independent research backs this up. Instruction files like AGENTS.md add about 20% to the bill and barely change results unless they stay short. Teams of agents can use around 15 times more AI capacity than a single chat. A rule written only in instructions is a request, not a guarantee.

In This Post

1. Ten terms in plain English
2. What the research says before you configure anything
3. Treat the agent like a new hire
4. Layer 1: the house rules file (AGENTS.md)
5. Layer 2: permissions, what the agent is allowed to touch
6. Layer 3: hooks, rules that can't be ignored
7. Layer 4: automatic checks
8. Layer 5: specs and small tasks
9. Layer 6: the "team" of agents
10. Security: the lethal trifecta
11. Keeping the agent's memory clean
12. The setup I'd build on Monday
13. Frequently asked questions

Ten Terms in Plain English

• AI coding agent: an AI that doesn't just answer questions but acts. It reads your code, runs commands, edits files and checks the result, in a loop, until it decides the task is done. Claude Code, OpenAI Codex, Cursor and Gemini CLI are examples.
• Token: the unit AI companies bill by, roughly three-quarters of a word. More tokens means a bigger bill.
• Context window: the agent's working memory for one session. Everything it reads or says fills it up, and quality drops as it fills.
• AGENTS.md / CLAUDE.md: a text file in the project that the agent reads at the start of every session, like an onboarding note.
• Permissions: settings that decide which actions the agent can take on its own, which need your approval, and which are blocked.
• Hook: a small script that runs automatically at a fixed moment, for example just before the agent runs a command, and can block it.
• Subagent: a helper agent the main agent sends off to do one focused job (review this, research that) and report back.
• Tests, linter, type checker: automatic checks that tell you in seconds whether code works and follows the rules.
• Pull request (PR) and CI: a PR is a proposed change waiting for review. CI is the automated pipeline that runs every check before a change is accepted.
• Prompt injection: hiding instructions inside content the agent reads (a web page, an email, a bug report) so the agent follows the attacker instead of you.

What the Research Says Before You Configure Anything

Most guides start with tools. I want to start with the numbers, because they change what you should set up first.

Feeling faster isn't the same as being faster. In 2025, the research group METR ran a controlled experiment with 16 experienced open-source developers working on 246 real tasks in projects they knew well. When they were allowed to use AI tools, they took 19% longer. Afterwards, they believed AI had made them about 20% faster (METR).

METR repeated the study in late 2025 with newer tools. This time the estimates pointed to a speed-up: about 18% faster for developers from the first study, and about 4% faster for newly recruited ones. But METR itself called the result unreliable. So many developers now refuse to work without AI that the study kept missing exactly the tasks where AI helps most, and both results were within the margin of error, which includes being slower (METR).

AI amplifies whatever team it lands in. Google's DORA research program, which surveyed nearly 5,000 technology professionals for its 2025 report, describes AI's main role as an "amplifier", magnifying an organization's existing strengths and weaknesses (DORA).

Treat the Agent Like a New Hire

Here's the mental model that makes the rest of this post easy to follow. An AI coding agent is like a very fast new hire who is brilliant, tireless, very literal, and forgets everything when the session ends. You wouldn't give that person the production database password on day one, and you wouldn't let them merge their own code without review. Every layer below has a direct equivalent in how you'd onboard a person:

Onboarding a person vs configuring an agent

For a new hire	For an agent	What it prevents
An onboarding note: how we work here	A short AGENTS.md / CLAUDE.md file	Guessing the wrong commands and conventions
An access badge for some rooms, not all	Permission rules and a sandbox	Touching secrets, production or the internet unasked
Rules enforced by systems, not memory	Hooks	"I knew the rule but did it anyway"
QA and code review	Tests, type checks and CI	Work that looks finished but isn't
Clear tickets with a definition of done	A written spec split into small tasks	Solving the wrong problem, huge unreviewable changes
A team with distinct roles	One writing agent plus read-only reviewer subagents	Agents contradicting each other

Layer 1: The House Rules File (AGENTS.md)

In plain terms: a short text file in the project that tells every agent the commands to run and the rules nobody would guess. It's the onboarding note. Keep it short, because agents treat every line as an order.

AGENTS.md is an open format now looked after by the Agentic AI Foundation under the Linux Foundation. It's read by OpenAI Codex, Gemini CLI, GitHub Copilot's coding agent, Cursor, Devin and others, and appears in over 60,000 open-source projects (agents.md). Claude Code uses its own CLAUDE.md, and recent versions also read AGENTS.md when a project has no CLAUDE.md (Claude Code docs).

Every guide tells you to write one. Here's what the two most rigorous studies so far actually measured:

What instruction files do to coding agents · two 2026 studies

Study	What they tested	What they found
ETH Zurich and LogicStar.ai (Gloaguen et al.)	Well-known benchmark tasks, plus 138 real tasks from 12 projects whose developers had written their own instruction files, across several agents and AI models	Instruction files did not generally improve success rates and raised the AI bill by over 20%. Files written by developers did about 7% better than files written by AI. Project overviews didn't help.
Lulla et al.	124 real code changes across 10 projects, OpenAI Codex with and without an AGENTS.md	The typical task ran 28.6% faster and used 16.6% fewer output tokens. Whether the code was correct wasn't fully checked.

These don't contradict each other. A good instruction file can make an agent faster at routine work. It doesn't make it smarter. And because agents follow instructions very faithfully, every extra line creates extra work: one more check, one more test run, one more file to read. The ETH authors' advice: don't use instruction files generated by AI, and in a hand-written one include only what the agent can't learn from the existing documentation or the code itself, such as unusual conventions and requirements (Gloaguen et al.). Anthropic's own guidance says the same: aim for under 200 lines, and for each line ask whether removing it would cause mistakes (Claude Code best practices).

For engineers: don't commit whatever /init generates without editing it. Aim for something like this:

# AGENTS.md

## Commands
- Install: pnpm install
- Test one file: pnpm vitest run path/to/file.test.ts
- Typecheck: pnpm tsc --noEmit   (must pass before you say "done")
- Lint: pnpm lint --fix

## Conventions you would not guess
- Server state lives in TanStack Query. Never copy API data into Zustand.
- All input validation uses Zod schemas from src/schemas/.
- Migrations are generated with prisma migrate dev, never hand-written.

## Never
- Never edit files under src/generated/.
- Never add a dependency without saying why in the PR description.

No tour of the project, no "we value clean code", no list of technologies the agent can read for itself. Every line is a command it would otherwise guess, or a convention it would otherwise get wrong.

I learned this the slow way. The project behind this site now has a short CLAUDE.md that only points the agent to separate guide files, one per type of task (writing blog posts, SEO, security and so on). Each one is loaded only when a task needs it, instead of one giant file loaded every time. Claude Code calls these Skills. It's the principle the research points to: put instructions where the task is, not in a file every session has to read.

Layer 2: Permissions, What the Agent Is Allowed to Touch

In plain terms: decide in advance what the agent can do alone, what needs your approval, and what is off-limits. This is the layer that actually prevents disasters, and most people set it up last.

In July 2025, an AI agent on the Replit platform deleted the live database of SaaStr founder Jason Lemkin during an explicit "code freeze", wiping records for 1,206 executives and 1,196 companies. It then told him the data couldn't be restored, which turned out to be wrong: Replit recovered it, and within days added automatic separation between test and live databases, plus a planning-only mode (AI Incident Database, Fast Company). The lesson isn't "the AI was bad". The freeze existed only in the instructions. Nothing stopped the command from running.

I'll be honest about my own setup. When I checked the Claude Code settings for this site's project while writing this post, I found I had permanently approved three commands: one that can send data to any website (curl), one that publishes code changes (git push), and one that deletes files in bulk (xargs rm -rf). Each was a reasonable click on "always allow" at the time. Together, they let an agent send data out, publish and delete without asking me. That's how permissions rot: one convenient click at a time. My workspace instructions also contain a rule, in capital letters, never to use one Notion option that deletes linked databases. By my own standard, that rule is also just a request and should be a hook.

For engineers: a safer baseline for Claude Code, saved in .claude/settings.json and committed so the whole team shares it:

{
  "permissions": {
    "allow": [
      "Bash(pnpm test *)",
      "Bash(pnpm vitest *)",
      "Bash(pnpm lint *)",
      "Bash(pnpm tsc *)",
      "Bash(git status)",
      "Bash(git diff *)",
      "Bash(gh pr view *)",
      "Bash(gh run view *)"
    ],
    "ask": [
      "Bash(git push *)",
      "Bash(pnpm add *)",
      "Bash(gh pr create *)"
    ],
    "deny": [
      "Read(./.env)",
      "Read(./.env.*)",
      "Bash(curl *)",
      "Bash(wget *)",
      "Bash(rm -rf *)",
      "Bash(git push --force *)"
    ]
  }
}

The logic: allow anything that only reads or is easy to undo and that the agent runs constantly (tests, checks, reading CI results and review comments with GitHub's gh tool). Ask before anything that leaves your machine or adds dependencies. Deny secrets, internet access and anything destructive. Claude Code checks deny rules first, then ask, then allow, so a deny can never be overridden by an allow (Claude Code docs).

Two honest limits. First, command rules are pattern matching, and Anthropic's documentation warns they're easy to get around: a rule against curl doesn't stop a script that makes the same request another way. When a rule must hold, turn on the sandbox, which blocks file and network access at the operating-system level. Second, recent versions of Claude Code start in "auto mode", where a separate AI model reviews each action and blocks the risky-looking ones. That's useful, but it's a judgment call made by a model, not a guarantee. Your deny rules and sandbox still need to exist underneath it.

OpenAI's Codex expresses the same idea in ~/.codex/config.toml: sandbox_mode = "workspace-write" lets it edit the project but nothing else, with no internet access unless you switch it on, and approval_policy = "on-request" sends anything outside that boundary to you (Codex docs). Different words, same idea: the agent works freely inside a fence, and crossing the fence needs a human.

For anything that runs unattended, such as agents in CI or overnight runs, build the fence into the infrastructure: an isolated container, an access token that can't reach live systems, and a branch it can push to that isn't your main one.

Layer 3: Hooks, Rules That Can't Be Ignored

In plain terms: a hook is a tripwire. It runs automatically at a fixed moment, for example just before the agent runs any command, and it can say no. Instructions can be forgotten or overridden. A hook can't.

For engineers: a Claude Code PreToolUse hook that blocks any command mentioning the production database:

// .claude/settings.json
{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          { "type": "command", "command": "${CLAUDE_PROJECT_DIR}/.claude/hooks/guard-prod.sh" }
        ]
      }
    ]
  }
}

#!/bin/bash
# .claude/hooks/guard-prod.sh: receives the pending command as JSON on stdin
CMD=$(jq -r '.tool_input.command')
if echo "$CMD" | grep -qiE 'prod(uction)?[._-]?(db|database)|DATABASE_URL_PROD'; then
  echo "Blocked: production database access is not allowed from an agent session." >&2
  exit 2   # exit code 2 blocks the command and shows this reason to the agent
fi
exit 0

Like any pattern match, this catches the obvious cases, not a determined workaround. Its job is to stop the honest mistake, which is what the Replit incident was. Real separation still comes from the agent never having production credentials at all.

Two other hooks earn their place on almost every team. One runs the code formatter and linter after every edit, so style is never something the agent has to remember. The other runs on the Stop event, when the agent tries to finish: it runs the type checker and won't let the agent declare the job done while it fails (Claude Code hooks). "Make sure it works" stops being a hope and becomes a condition of finishing.

Layer 4: Automatic Checks

In plain terms: an agent with no way to check its own work is guessing. An agent with fast, objective checks can catch and fix its own mistakes. This is the single biggest quality lever, and it's ordinary engineering work.

Anthropic's guidance puts it well: give the agent a check it can run, because "it's the difference between a session you watch and one you walk away from" (Claude Code best practices). In practice:

• Tests that run in seconds. If your test suite takes 20 minutes, the agent will skip it or waste time. Provide a "test just this file" command and put it in AGENTS.md.
• Strict type checking and linting as hard gates. In TypeScript, strict: true catches a whole class of agent mistakes for free.
• A failing test first, for anything non-trivial. Ask the agent to write a test that reproduces the bug or captures the requirement, confirm it fails, then fix it. It gives the agent a target that can't quietly drift.
• CI has the final say, not the agent. Whatever the agent claims, its work goes through the same automated checks as a human engineer's.

One warning: an agent under pressure to make tests pass will sometimes change the tests instead of the code. Put "never weaken or delete a test" in your rules file, and have a reviewer look specifically at changes to test files.

Layer 5: Specs and Small Tasks

In plain terms: write down what "done" means before the agent starts, then cut the work into pieces small enough for a person to review properly.

"Spec-driven development" is the current name for something good teams always did. Tools exist at different weights: GitHub's Spec Kit is a free command-line kit that walks the work through specify, plan, tasks and implement. AWS's Kiro is a full editor built around requirement, design and task documents. OpenSpec is lighter and designed for changing existing projects. You don't need any of them to get most of the value:

1. Write a one-page spec in the project: the problem, how you'll know it's done, what's out of scope, and what must not change.
2. Ask the agent for a plan in a read-only planning mode, and review it the way you'd review a junior engineer's design. This is the cheapest moment to catch a wrong assumption.
3. Split the plan into thin slices, each a small, complete piece of behaviour that ends with a passing test.
4. One slice per session, one proposed change (PR) per slice.

A simple test for slice size: could someone review the change properly in 15 minutes? If not, it's too big. The real limit on how much agent work a team can absorb is review time, not how fast the agent writes.

Layer 6: The "Team" of Agents

In plain terms: use one agent to write and several to check. Agents that each see only part of the picture make conflicting decisions, and running many at once multiplies the bill.

This is where everyone wants to start: a planner agent, a coder agent, a reviewer agent, a tester agent, all talking to each other. It's also where the most money gets wasted.

Two reference points. Anthropic's multi-agent research system beat a single agent by 90.2% on its internal research tests, but used about 15 times as many tokens as a normal chat. Anthropic also wrote that most coding tasks have fewer parts that can truly run in parallel than research does, which makes them a weaker fit for teams of agents today (Anthropic). Cognition, the company behind the Devin agent, put the problem in one line: "Actions carry implicit decisions, and conflicting decisions carry bad results" (Cognition).

Together, they suggest a rule I'd defend: one agent writes, the others read. Helper agents are good at exploring and judging: searching a large codebase, reviewing a change, checking security, researching a library. They're bad at writing different parts of the same feature at the same time.

Team roles as agent configuration

Human role	Agent equivalent	What it's allowed to do
Tech lead	The main agent in planning mode, producing a spec and task list	Read only
Engineer	The single main agent that writes code, one slice at a time	Edit, run tests and checks; ask before publishing
Code reviewer	A subagent with a fresh memory that sees only the change and the spec	Read only
Security reviewer	A subagent looking for leaked secrets, injection risks and missing access checks	Read only
Researcher	A subagent that explores code or documentation and returns a summary	Read and search
QA	Not an agent: the tests, type checker and CI	n/a

For engineers: in Claude Code, a subagent is a Markdown file in .claude/agents/. The description tells the main agent when to use it, and the tool list limits what it can do:

---
name: diff-reviewer
description: Reviews the current diff against the spec before a PR is opened. Use after a slice is implemented and tests pass.
tools: Read, Grep, Glob, Bash
model: sonnet
---

You review a diff you did not write. You have not seen the conversation that produced it.
Use git diff and git log only; never modify files.
Check, in order: does it meet the acceptance criteria in the spec; were any tests weakened or deleted;
are there unhandled errors, leaked secrets or missing authorisation checks.
Report only gaps that affect correctness or the spec, with file and line. Do not fix anything.

One detail that tripped me up: the tools field takes whole tool names, not command patterns, so you can't write Bash(git diff *) there. Command-level limits come from the shared permission rules in Layer 2, which subagents follow too (Claude Code subagents).

The fresh memory is the point. Each subagent starts with an empty context window and doesn't see the conversation that produced the code. A reviewer that watched the agent's reasoning tends to accept it. A reviewer that sees only the change and the spec reads it the way a colleague would. Anthropic's guidance adds a useful warning: a reviewer asked to find problems will usually report some even when the work is sound, so tell it to report only what affects correctness. Use a cheaper model for narrow review jobs, and keep the most capable model for the agent that writes code.

If you really need several agents writing at once, give them unrelated tasks, not parts of one feature. Give each its own separate copy of the project (git calls this a worktree) and its own branch, and merge their work through normal reviewed PRs.

Security: The Lethal Trifecta

In plain terms: never give one agent all three of these at once: your private data, content written by strangers, and a way to send data out.

Simon Willison's "lethal trifecta" is the most useful security idea for agents I know. Any agent that combines access to private data, exposure to untrusted content and the ability to communicate externally can be tricked into leaking that data, because AI models follow instructions wherever they find them (Simon Willison).

A coding agent has all three by default. It reads your code and your secret configuration files. It reads bug reports, web pages, other people's code and the output of connected tools, any of which an attacker can write. And it can make web requests, publish code or call other services. That's why the deny rules in Layer 2 target secrets and internet access specifically. Remove at least one of the three for any agent that reads content from outside, and be especially careful when you connect agents to email, chat or ticketing tools (often through MCP, the standard way to plug tools into agents), because anyone outside the company can write the content those agents read.

Keeping the Agent's Memory Clean

In plain terms: long sessions get worse. Start fresh often, and keep important decisions in files rather than in the chat.

As the agent's working memory fills with old files, failed attempts and abandoned plans, it starts working from stale information. Anthropic calls the context window "the most important resource to manage" (Claude Code best practices). A few habits fix most of it:

• A new session for each phase. Planning, building and reviewing are different jobs. Clear the memory between them (/clear in Claude Code), and pass the result on as a file, such as the spec or the plan, not as chat history.
• Write progress down. A PLAN.md or progress note the agent keeps updated survives a fresh session. Chat history doesn't.
• Send helpers to do the reading. When a task needs a wide search, a subagent can do it and bring back a summary, keeping the main agent's memory focused on the work.
• Restart instead of arguing. If you've corrected the agent twice on the same thing, its memory is cluttered with failed attempts. Start fresh with a better spec.

The Setup I'd Build on Monday

If I were setting this up for a team from scratch, I'd do it in this order. The first three steps matter most, and none of them involves the AI model itself.

1. Review and share permissions. One committed settings file: allow tests and read-only commands, ask before publishing or adding dependencies, deny secrets, internet access and destructive commands. Turn on the sandbox. Clear out whatever has piled up in personal "always allow" lists.
2. Make checks fast. A test command for a single file, strict type checking and linting, all running in seconds.
3. Add two hooks. Format and lint after every edit, and run the type check before the agent can finish. Then add one guard hook for whatever would be most expensive to get wrong.
4. Write a short AGENTS.md by hand. Commands and non-obvious conventions only. If your team also uses Claude Code with a CLAUDE.md, have the CLAUDE.md import it with a line reading @AGENTS.md, so every tool reads the same rules.
5. Use the spec, plan, slice routine for anything bigger than a bug fix.
6. Add one read-only reviewer subagent. Check whether it catches real problems before adding a second one.
7. Measure. Track how long changes take to get merged, how long reviews take, and how often changes break something, before and after. The METR study is a reminder that feeling faster is not evidence.

Key Takeaways

• Agents amplify your engineering process. Tests, written conventions and access control matter more than which agent you choose.
• Keep instruction files short and written by people. Research shows bloated or AI-written ones add about 20% to the bill without improving results.
• Enforce, don't request. Anything expensive to get wrong belongs in permissions, a sandbox or a hook, not in instructions.
• One writer, many readers. Use helper agents to review, research and check security, not to write parts of the same feature in parallel.
• Break the lethal trifecta. Never give one agent private data, outside content and a way to send data out at the same time.

What's sitting in your agent's "always allow" list right now that you approved once and forgot about?

Frequently Asked Questions

Does an AGENTS.md or CLAUDE.md file make coding agents better?

Only modestly, and only if it's short and written by a person. A 2026 ETH Zurich study found instruction files didn't generally improve success rates and raised costs by over 20%, with developer-written files doing about 7% better than AI-generated ones. A separate study found an AGENTS.md made OpenAI Codex about 28.6% faster on typical tasks. Include only commands and conventions the agent can't work out from the code.

Should I use multiple AI agents for software engineering?

Use one agent that writes code and extra subagents that only read: code review, security review and research. Anthropic reports that multi-agent systems use about 15 times the tokens of a chat and that most coding tasks are less parallel than research. Cognition warns that agents working from partial context make conflicting decisions.

How do I stop an AI coding agent from running dangerous commands?

Enforce it in configuration, not in instructions. Use permission rules (allow, ask and deny in Claude Code, or sandbox mode and approval policy in Codex) to block secrets, internet access and destructive commands. Turn on the sandbox for rules that must hold, and add a hook that blocks specific commands. A rule that exists only in instructions is a request, as the 2025 Replit database deletion showed.

What is the lethal trifecta for AI agents?

A term coined by Simon Willison: an agent that combines access to private data, exposure to untrusted content and the ability to communicate externally can be tricked by prompt injection into leaking that data. Coding agents have all three by default, so remove at least one, usually by blocking internet access and secret files.

Sources

• METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
• METR, We are Changing our Developer Productivity Experiment Design
• DORA, 2025 State of AI-assisted Software Development
• Gloaguen et al., Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
• Lulla et al., On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents
• AGENTS.md, an open format for guiding coding agents
• AI Incident Database, Incident 1152: Replit agent deletes production data during code freeze
• Fast Company, Replit CEO: What really happened when AI agent wiped Jason Lemkin's database
• Claude Code Docs, Best practices
• Claude Code Docs, How Claude remembers your project
• Claude Code Docs, Configure permissions
• Claude Code Docs, Hooks reference
• Claude Code Docs, Subagents
• OpenAI Codex Docs, Configuration reference
• Anthropic, How we built our multi-agent research system
• Cognition, Don't Build Multi-Agents
• Simon Willison, The lethal trifecta for AI agents
• GitHub Spec Kit

You might also like The Cheaper AI Model Can Cost You More, Shipping AI Features in Production: GPT-4o Inside a Live Platform and the code quality guide.