We built an AI chief of staff. It could run code.

We're building an AI chief of staff — Maia Nexus Core, "Maia" for short: a Slack bot that will eventually run a fleet of agents on our own hardware. Phase 1 was deliberately tiny: answer direct messages, and hold exactly one tool, an emoji reaction. Nothing else. No files, no shell, no web.

It passed every test. Type-check clean, linter clean, a live smoke test that worked. Then we pointed our own review agents at it, and they found an AI agent security hole: the "one tool" bot was actually holding about eighteen — including tools that spawn subagents and execute code.

Here's what happened, because the mistake is one anyone building on today's agent frameworks can make, and the fix is one line.

"Allowed" does not mean "only these"

We built Maia on Anthropic's Claude Agent SDK. To lock the agent down to a single tool, we did the obvious thing: set allowedTools to our one emoji tool, and add a disallowedTools denylist of the dangerous built-ins — shell, file read/write, and so on.

That reads like a safelist. It isn't. In the SDK, allowedTools only controls which tools run without asking for approval. It does not control which tools exist. The SDK's own documentation says it plainly: to restrict which tools are available, you use a different option entirely.

So with our config, the agent loaded the SDK's full default tool set — minus the handful we remembered to deny. When we probed the model's actual startup handshake, it had been offered about eighteen built-in tools we never intended: tools that spawn subagents, run orchestration scripts, schedule recurring jobs, and load still more tools on demand. Our denylist was already out of date on day one: it blocked an older task tool but not its current replacement.

An "emoji-only" assistant was, in fact, carrying tools that run code — and nothing we controlled stood between it and them.

Why a denylist is the wrong shape

The deeper problem isn't the missing tools. It's the shape of the defense. A denylist is fail-open: everything is allowed unless you specifically block it. That fails in two directions at once.

It fails on the day you write it, because you can only block the dangerous things you thought of. And it fails over time, because every framework update that ships a new default tool lands in your agent automatically — until someone remembers to go add its name to the blocklist. You are one dependency bump away from handing your agent a capability you never reviewed.

This matters more for agents than for ordinary software because of what an agent process holds. Maia runs with its own Slack tokens and API key in its environment. An agent that can reach a code-execution tool and also holds live credentials is a secret-exfiltration path — and it's reachable by prompt injection. The moment a chief-of-staff agent summarizes an email or a document, hostile instructions hidden in that text are trying to steer it. "Only summarize things" is not a security boundary if the agent quietly has about eighteen other tools available.

The one-line fix, verified

The fix is to make the tool set an allowlist you control, not a denylist that rots. In the Claude Agent SDK, that means one option: set the tools list to empty. That disables every built-in, while your own intended tool still loads.

We changed it, then re-ran the same startup probe to confirm — not to assume. This time the agent was offered exactly one tool: the emoji reaction. Nothing else. We kept the denylist too, but only as a backstop, never as the actual perimeter. The real gate — a policy check on every single tool call — was the very next thing we built, before the agent gets any tool that can change the outside world.

That sequence is the whole point: the safe wall goes up before the dangerous door.

What the review actually was

We didn't find this by staring harder at the code. We ran an adversarial review — the same agents-checking-agents approach we used to vet our mascot — a set of AI agents reading the change from three angles (security, platform correctness, runtime robustness), and for every issue one of them raised, two more independent agents whose job was to disprove it. Only issues that survived the attempt to refute them counted.

Nine defects survived. The fail-open tool posture was the serious one, but the same pass also caught a container misconfiguration that would have broken every follow-up message, a logging setting that silently disabled all logs (including the security ones), and a race between two messages in the same conversation. All nine are fixed, with tests, before a single line shipped to a real deployment.

This is the same reason we didn't build Maia on the most popular open-source personal-assistant framework, tempting as its ready-made features were: an independent security analysis catalogued 470 advisories against it in about ten weeks, several of which chained into full remote code execution — because no single point in its design saw the whole picture. (That analysis is a preprint, not yet peer-reviewed, but its findings line up with vendor advisories.) One controlled gate beats a hundred scattered checks.

The honest caveats

This is Phase 1 of a longer build, and it isn't deployed yet. We proved the agent was offered the dangerous tools — a probe of the model's actual startup handshake showed them sitting in its context — and we stopped there: we never made it run one, and we didn't build a working exploit, because we fixed the hole instead. And our framework choice leans partly on that preprint. We'll say all of that plainly, because a build-in-public post that admits no limits is just marketing.

But the core lesson holds, and it's portable to whatever agent stack you're on: read what your agent can actually do, at runtime, not what you think you configured. "Allowed," "enabled," and "restricted" mean different things in different SDKs, and the gap between them is exactly where the security problems live. Make your tool set a short allowlist you own. Put the gate up before the capability. And have something adversarial read the result — an "only one tool" agent that quietly holds about eighteen will pass every test you write, because you didn't think to test for the tools you didn't know were there.

The takeaway for your team

We're building RawrTech itself on agents, in public, and writing down what breaks. If you want a straight read on where AI agents can safely take work off your team's plate — and where they can't yet — our free AI Opportunity Briefing is a 45-minute, no-pitch conversation: get in touch. We'll tell you what we'd actually automate, and what we'd leave alone.