From onsite to production: how Bretton takes an agent live

How Bretton takes a customized agent to production: validating it against your team’s past investigations while integrating it with your systems in parallel

In one of our latest blogs, we cover the Agent-in-a-Day process we run with prospective customers. Instead of spending weeks in discovery, we show up onsite with an agent pre-built from your SOPs and past cases and tune it live with the people who do the work. By the end of the day, your team has watched it work your cases the way you would.

But a working agent isn't a production agent yet. While it has been tested on a handful of cases and still runs in the Bretton AI sandbox, two questions remain: does it hold up across the work your team has already done, and can real cases flow through it from start to finish? We answer both at the same time, on two parallel tracks, which is how we get from that first day to a live agent in weeks, not months.

Validate against work your team has already done

The onsite session shows the agent can follow your workflow, but validation is where we test how it performs across the full range of cases your team handles. We run it on a larger set of closed investigations and compare its work with yours, looking beyond the final decision to the findings, sourcing, and reasoning behind it.

Those past investigations serve as a baseline rather than an answer key, so a disagreement does not automatically mean the agent is wrong. Your reviewers decide which call is right, or whether both are reasonable. Agent errors go into tuning, and every decision is documented, giving your model risk team a clear record of where the agent differs from your past work and why.

Fix patterns of failure

With the discrepancies identified, we look for what they have in common. A single error says little, but a recurring one points to something specific in how the agent works, such as consistently underweighting a category of adverse media. In our published validation, grouping results by field showed region precision for individuals trailing country precision, 94.2% vs. 98.6%, the kind of pattern this step is built to find.

Each pattern is then traced to the component responsible, whether an instruction, a skill, a source, a threshold, or a control that belongs outside the model. Because prompting alone cannot fix many of these failures, we make a specific system change and re-run the full validation set to confirm the agent improves without regressing. The result is a clear account of where the agent performs well and what must change before launch.

Integrate with your systems in parallel

Most integration dependencies, like API access, security reviews, and write-back, do not need to delay learning whether the agent works. The exception is approval to export historical cases for validation, so we start that request early. Integration runs alongside validation and answers four questions: how new cases reach Bretton AI, where it gets the data it needs, how records link across your systems, and where completed work is returned. The test is simple: real cases flow in, the agent has the data it needs, and results flow back to your system of record.

Running both workstreams together keeps the timeline short. While the agent improves against your historical cases, access, security, schema mapping, and write-back move forward on their own schedule. In recent deployments, our integrations have gone from first access to production in about a week. This is also why integration was never a prerequisite for the onsite session: we prove the agent works first.

One plan for both tracks

To keep validation and integration moving in step, both workstreams are governed by a Mutual Action Plan developed jointly with your team. Each item carries a named owner, a target date, and an explicit definition of done, so progress toward launch is measured against criteria both parties agree on at the outset. A representative plan is outlined below.

From there, the decision to go live is made jointly and rests on two conditions: the agent’s performance against your historical standard is documented, with material failure patterns addressed and remaining differences accepted by your reviewers, and real cases move through your production environment end to end without manual workarounds. Once both hold, the agent goes live, and measurement continues from there. The Bretton Evaluator, built into the Bretton AI platform, assesses, run after run, whether the agent is still meeting the standard it was given.

Proven, connected, and ready for production

By the time the agent goes live, you are not relying on our word for how it performs. Your reviewers have tested it against cases they already know, recurring failures have been traced to their cause and addressed, and your model risk team has a documented record of every material difference and why it was fixed or accepted. That is what makes it ready to carry real work.

If you would like to see what this looks like for your workflow, we should talk.