Agentic Security Engineering

How do you do security engineering when the code is being changed in real time by a swarm of coding agents? We went all in and did the same.

At Synthesia we strive to move fast and be efficient. We recently released Syren, taking it from internal proof-of-concept to external release  in a very short timeframe, turning a prototype into a product-grade system that meets all our quality and compliance requirements.

The product was being built the way a lot of software is built now: by engineers directing coding agents. In its first couple days the main repository saw around 170 pull requests, all of them written by agents and merged by humans, noo threat model and no security review. There was, however, a very ambitious launch date set in stone.

In that context the traditional approach of carefully shifting security left doesn't work: there is no "left" to shift to when the design is changing faster than a review cycle. We had to do the security work at the same time the system was being built. Our guidance had to be real time, and ideally we had to contribute fixes directly, deploy continuously to a test environment for quick validation loops, and evaluate and tune every guardrail as it went in.

The engineering team had swarms of coding agents. So we built one for security engineering.

This post describes how we set it up, how we use it, how we kept the agents honest, and what it cost us in money and in human attention. We measured that last part.

Mirror the builders

We previously wrote about scaling vulnerability management after issues ship, and about automating code security reviews before changes merge. Both are pipelines: a known input goes through fixed phases. In this context we have no traditional fixed phases and the product is changing constantly as it's being molded from a prototype.

Here the focus point is to support the human operator in keeping a current and correct mental model of the system. Everything else follows from it: spotting the big structural issues and design assumptions, threat modeling iteratively to define and implement risk mitigations, and keeping those mitigations current as the system changes.

So instead of a pipeline we used a local agents orchestrator that we built in-house, which simply arranges agents in a hierarchy to form multiple small teams. There are two kinds of agents:

Smart long-running agents on the host: we call them captains. Each one owns an area of work. They can read our Slack, Linear and our observability tools, use the GitHub CLI and the local toolchain, start other agents, and open pull requests when we allow it. The operator, the security engineer running the fleet, can join any of these at any time to ask questions, steer them, or make a call. Captains can also spin up new agents reporting to them.

Disposable and sandboxed workers. Each one gets a single written brief with a task and returns a single result: a report, commits, a validation. They have a fixed network allow-list, no access to internal systems, no push rights and no access to production. They are cheap, they run in parallel, and their captain checks their output before anything is used.

                    operator (AppSec)
             joins any captain, gets pings
                         │
                         ▼
              captain: context gatherer
       watches the team channel and pull requests,
             keeps the map of the system
                         │
        ┌────────────────┼────────────────┐
        ▼                ▼                ▼
     captain          captain          captain         one per area
        │                │                │
    ┌───┴───┐        ┌───┴───┐        ┌───┴───┐
    ▼       ▼        ▼       ▼        ▼       ▼
 worker  worker   captain  worker  worker  worker
                     │
                 ┌───┴───┐
                 ▼       ▼
              worker  worker

captains: long-running, on the host, act on shared systems with a go-ahead
workers:  disposable, sandboxed, one brief each, reviewed by their captain

‍‍That gives two trust tiers: full autonomy where the blast radius is small, and supervision where an agent can act on shared systems. A few things nobody in the tree could reach: feature flags, the cloud console and a logged-in browser. That part stayed with us.

We also use multiple models: in general we try to right-size the model to the task, after having built a familiarity with the various models. But also current frontier models refuse to perform a lot of cybersecurity tasks so we had to delegate those to near-frontier open models.

Building the meat context

One of the operator's duties is to manage the agents' context window to ensure every one stays in the smart zone, before its reasoning starts to degrade. But one of the main goals of the orchestration system is to support the operator in building and maintaining a current mental model of the system in scope: that's the meat context.

So the first agent we started was a context gatherer.

Its job was to watch the product team's engineering activity every so often, watch the PRs and build a map of the system (what it does, which components, who owns what, which areas are moving fast) and ping when something security-relevant happened.

Then we were able to shift our attention elsewhere and turn back to the context gatherer when we needed more data.

Over the week the context gatherer flagged several risky changes as they landed. It wasn't perfect, e.g. sometimes pings were overstating issues, but as we kept working the system improved towards a decent signal/noise ratio. Once engaged, the median time from a ping to us acting on it was about 13 minutes.

Fan out by uncharted territory

As the operator focuses on building the mental model that maps the system in scope, each uncharted territory is assigned to a new agent that is tasked to bring back context first, and then later used to do the security work on that area.

The initial briefs are always short, for example:

spin up a new captain, focus them to look at how $COMPONENT-1 and $COMPONENT-2 are set up and interact together. Map the data flows and then I'd like a security review of the current implementation.

That captain knows how to split the work across parallel workers or sub captains and which model to use for each task, how to validate its agents' work, design a fix, implement it, support its deployment and monitor it once it lands in the test environment.

This is recursive: as the operator is building his mental model, new areas of the system are delegated to agents and those agents over time grow their own team. Their work shifts from context gatherers to designers and implementers as their scope grows in maturity.

From context, to design, to engineering, to operations

A system that is being built while you secure it goes through stages of maturity. As each area matures, the security agents working on it change their job: first they help the operator understand the code, then they help threat model it, then they work as full-time software engineers, removing security issues and implementing guardrails, and finally they help run what was shipped.

Understand. At the start everything is uncharted and moving. The agents' main job is to help us build the mental model and keep it current: explain how a component works, draw the architecture, and check our understanding. We did this through conversation far more than through reading. The agent does the reading, and we keep asking until the picture in our head matches the code. This is the phase where we validate the bigger issues and let the agent demonstrate them in our testing environment.

Design. Once the structural issues and design assumptions are clear, the agents become design partners. We threat model and brainstorm how to implement a control with a captain before anything is built: what it should do when it fires, where it sits, what is in scope, and when it fails open or closed. An hour of that conversation, making a decision every few minutes, was some of the highest-leverage time of the week. Only when the design holds up does the captain take over the implementation.

Engineer. Then the agents write the code. The captain writes a plan and splits it into tasks, workers implement in parallel and/or sequentially, and the captain reviews and integrates their work and opens a pull request. A design session in the afternoon could be a merged control by the evening.

Operations. Shipping a control is not the end of the job. The agents monitor the test environment once a change lands, root-cause what breaks, run bespoke integration tests where unit tests aren't enough, set up alerts, and keep the evals running.

Different areas sit at different stages at the same time, and one area can move through all four in a few days (even back and forth). Part of the operator's job is knowing which stage each area is in, and asking for the right kind of work.

Evaluating guardrails: measure, shadow, tune, iterate

The systems we secure are increasingly agentic themselves, and their codebases can change dramatically from one day to the next. A new tool, a reworded system prompt or a new kind of input can open a gap that no review would spot, or quietly degrade a guardrail that is itself an LLM. Running evals continuously, in development and against production, is how we spot newly introduced gaps and loss of effectiveness as soon as they appear.

In the orchestration, a guardrail is an area like any other. It gets its own captain, which takes it through the four stages, and once the guardrail ships the eval loop becomes that captain's standing job. Workers build the guardrail, the captain owns the evals and reads the production data, and the operator decides when a policy enforces.

We build two kinds of guardrails.

Deterministic guardrails: static analysis, input validation, sandbox and network policies. This is classic security engineering, and we evaluate it the classic way: against legitimate samples it must accept and adversarial samples it must refuse by having agents run integration tests.

LLM-based guardrails are, for example, LLM judges: classifying generated artifacts, moderating content, detecting prompt injection, or checking an agent's output. We need them because the systems in scope are agentic and most of their inputs are natural language, which deterministic rules can't judge. They are also non-deterministic, which makes the evaluation loop essential:

• Measure in development. Captains write an eval set for every policy before it ships. When the frontier model refuses to write adversarial samples, a worker on an open model writes them. A separate worker writes a blind held-out set that the builder never sees. Then the operator tries the guardrail by hand on a local stack.
• Ship in shadow. The guardrail goes to production logging its verdicts and blocking nothing, with alerts on spikes and on errors. The captain reads the production data itself, through our observability tools, rather than trusting reports that something is "enabled".
• Tune on real traffic. Production finds false positives no synthetic set will. The most useful question for every one of them is who authored the text being judged: often it's the system's own scaffolding, not an attacker. The captain proposes the fix and workers implement it.
• Probe every change. Aggregate scores are upper bounds. A change that keeps the eval perfect can still open a hole that a small probe, written for that specific change, will find.
• Enforce one policy at a time, and never stop evaluating. Enforcing is the operator's call, made for each policy when its own shadow data supports it. After that, the captain keeps the evals running before every change to the guardrail and against every change to the system around it.

This is also where the loop pays off in attention: the agents iteratively do the measuring and loop over implementing corrections. The operator's time can then go into a handful of decisions and into different areas.

Trust the proof

Agents produce confident claims based on reading the code. Most of the security value of the week came from proving those claims where the system actually runs.

Reproduce the finding. We don't take an agent's word for a vulnerability. We reproduce it end to end in a real environment, with agents guiding the operator step by step.

Test where users are. A mitigation can pass its whole unit test suite and still fail in a real browser. We used agents whose only job was to drive a real browser against the change. They found problems the tests couldn't see.

As we wrote in our post on automating code security reviews, this is just refusing to make decisions on vibes.

Decisions stay human

This whole orchestration is a pure extension of the human expertise. But to ensure we stayed in the loop when it mattered we followed a few principles:

Agents propose, humans accept risk. Anything that accepts or assigns risk stays with us: the severity of a finding, whether a gap is acceptable for launch, which architecture we bet on, and whether a guardrail enforces in production. The agents bring options and evidence; we make the call.

Delegate what is reversible and reviewable. Workers run fully autonomously because their blast radius is a sandbox and their output is reviewed before it is used. Captains are read-only on shared systems by default: posting in Slack, writing to tickets, pushing code or opening a pull request needs our go-ahead.

Accountability stays with people. Every merge needed a human approval, and our agents never approved their own work. Beyond that we delegated a lot. A captain can babysit a pull request on its own: validate every review comment, fix the valid ones, reply to all of them, keep CI green and the branch mergeable, and ping us only once another reviewer has approved it.

Decisions that involve people are made with people. When a decision affects other teams or how a system is built, the agents produce the options and the evidence, we validate them and we take them to the people involved. The decision is human and social: the agents' work is its input. When a better option emerges, we close our own agents' work, however green it is.

Close the context gap. From the context gatherer onward, our agents collect context on their own: Slack threads, pull requests, internal docs. They can't see everything. Decisions made in a meeting, a DM or a thread they weren't pointed at never reach them unless we bring them back deliberately. And when they retrieve context automatically, agents tend to fill the gaps with assumptions, so we instruct them to state those assumptions and ask us to validate them before acting on them.

Retrospective

We wanted numbers, so after the dust settled we ran a retrospective over the agent transcripts.

What we shipped

Over about a week we leveraged 82 autonomous agents: 10 captains and 72 workers. The fleet helped us ship six security controls, rework two major features so they are safer by design and implemented extensive operational security monitoring and alerting. The agents' merged pull requests added roughly 15,000 lines of code.

We spent  an aggregated $940 in inference costs on captains and $173 for workers, for a total of  $1,110 using various models for different tasks.

We have had no security incidents or bug bounties reports since the public launch.

Where the attention went

We counted every time a human had to step in, and labeled why: to add value, to babysit the system, or to do something the agents couldn't reach.

VALUE        56%   decide, learn, start work, comms, steer
BABYSITTING  27%   unblock, nudge, relay
REACH        18%   hands-on testing, flags, approvals

‍By day, "value" led every time: most of our time went into deciding, steering and learning, the part of the work only a human can do, rather than into keeping the agents running.

Where the attention leaked

The babysitting is where the attention leaked, and each major cause has its own fix:

• Model refusals on defensive security work, the biggest cause. Keep a second model on the bench and fall back to it automatically: we used a mix of GLM 5.3 and OpenAI Daybreak.
• Expired credentials, for example access to our cloud test environment. Check them once, when an agent starts, and refresh them for all agents in one go.
• Polling loops that woke a model every few minutes and rarely found anything to do. We soon moved to have a script poll, and wake the model only when something changed.

How many captains at once

We also looked at span of control. Eight to ten captains were alive for most of the week, but we never touched more than four in a single hour.

Day 0 is when we started the context gatherer, which then worked alone before the four working days in the table.

This is a sign that, with the current orchestration design, the operator's capacity extends to 3-4 workstreams at the same time.

The human cost

Running a fleet like this means holding two mental models at once: the system you are securing, which changes every hour, and the state of every agent working on it. Each switch between captains is a context switch, with its decisions, its open questions and its half-finished work. On the busiest day the median visit to a captain lasted less than a minute, and our messages shrank to a few words. Interestingly, this way of working is also very addicting: it feels like we are locked into the zone and everything is within reach and the only bottleneck is our capacity to work the problem.

The aftermath is a brain fog. Working like this burns brain sugar, and it's probably unsustainable as a steady state. It worked as a sprint for a launch. As a way of working every week, it needs less friction, less babysitting, better UX to support human context switching, and deliberate time to recover. Or just a slower pace.

What's next

We learned a lot working with a local orchestrator: it allowed us to run experiments in real time, and we will look into formalizing some of what we did in proper backend systems.

But the bigger thing to explore next is about our role as security engineers: our rituals and priors are based on assumptions that are not holding anymore when systems are built by agents. We need to rethink what a security review is when the design changes every day, how a threat model stays current when the code is written faster than anyone can read it, and which parts of our own judgment we can safely hand to agents.